Well, I'd still recommend a good 22LR to start with. Again, maybe a Ruger 10/22, or some other 22 rifle. Lot's of fun, and very inexpensive for ammo.
Now, since you want something to shoot/have fun with, as well as for hunting, I'd pick
a rifle chambered to the 308Win. Ammo is readily available in a wide variety of
bullet styles from all the major ammo manufacturers. In addition, surplus military ammo
is also available at reasonable costs from many sources, as well as less expensive
factory reloaded ammo from outfits like Black Hills. A rifle in 308Win. will handle any game in the lower 48 with no problem, utilizing the right ammo. It's a great deer cartridge, but can also handle elk & moose, and smaller bears (black). A 308 is never wrong.