Get a .22 first because you can get a good .22 for not too much money and you can get 400 rounds of ammo for the price of 20 rounds of other calibers.

Even though some states don't require a hunter safety course if you are over 18, take the advice of the other posters and take one anyways. You may find that the instructors or volunteers may be willing to give you more advice than just what they are required to teach.

Shoot your .22 lots and lots and then shoot more. You will want to spend time shooting targets but will want to do something more exciting like shooting soda cans that aren't always at a set distance and move around a bit when you hit them.

You might want to join a gun club (you might have to anyways to get range access) There's always someone at the gun club or shooting range willing to give advice. Just make sure you watch them a bit before asking or taking their advice because there's just as much bad advice as good advice out there if not more.

Squirrels, gophers (ground squirrels), and stuff like that make good starter game. You'll have to check your state regulations to see if you need a license for the animals you want to harvest.

Have Fun!