you can shoot a .243 all day. Piece of cake for recoil and conventional wisdom is that it's adequate for deer.
Edit:
I am looking to get into deer hunting for sure, but the potential to go to bigger game could grow if I really get into the sport (I live in PA).
I didn't notice tis when I scanned your post. .308 will work great and gives you ability to move to bigger game. Recoil is manageable. Ditto for a .270
Most people can manage a .30-06 recoil. Maybe shoot one and see how you do. It opens more possibilties in terms of loads and shells.