It's be hard to make a mistake starting off hunting small game ... I'd bet a lot of us cut our "hunting teeth" on hunting squirrels and/or rabbits. I did. A decent bolt action 22 LR or 20 ga. pump would be a fine small game gun.
About all one "needs" to go out and hunt besides the right firearm would be proper clothes for the temperatures likely to be encounter, sturdy foot wear ..... and the paitience and willingness to learn woodsmanship.
Beware !! There are gobs of gimmicks out there but there are no short cuts to becoming a top notch hunter. If there were, believe me about 75% of us would have a wall filled with big 'uns. Don't get caught up in the hype !!! Reading "how to" books or articles is a a decent place to start learning, but not nearly enough. (A huntsman that would serve as your mentor would be a heck of a lot better in my opinion). Add to taht spending hour upon hour in the field watching and listening .... looking for game trails, finding "sign" , observing the critters' normal habits, etc. all goes into making your self into a good hunter. Same goes for becoming proficeint at the basic stuff like easing along without being detected, how to play the wind direction, compass & map skills ... etc. and you are on your way.
As far as which fire arm, scope, bino, cammo pattern, pack, GPS unit, boots, rain gear, ammo ... and on and on and on .... there market is saturated with fine high value gear. About any brand will do as long as you stay with as much quality as you can afford.