I started hunting a few years ago at 44...its lots of fun and a great father/son activity.
I went with a 30-06 because it seemed to me to be the most popular caliber. I am not recoil sensitive, and really enjoy feeling it unload.
As far as public land, there is good hunting on most public land, the key is to do some scouting and set up where others seldom go (if you get 1/2 mile form the nearest parking lot, you will most likely be alone. I also like to be up in a tree (using a climber) on opening day of rifle season on public land, as I feel safer up there:-)
Gutting is pretty easy...the only tricky part is freeing up the butt hole so you can get the inards out. Skinning also is not to hard, it just takes a little practice and a nice sharp knife.