No deer will ever know the difference between those 2 choices. They fire identical bullets at similar velocities. The 308 is a bit slower, maybe 150-200 fps.

The big difference here is the cartridge length. The 308 is shorter, so the gun's receiver can be shorter, the gun is therefore lighter, and the bolt-throw is also shorter. What that all means is a 308 bolt rifle can be "handier" that a 30/06. I like a handy rifle for woods hunting.

If you can, check out a Browning Micro Medallion, a Ruger M77 Ultralight, a Remington Model 7 and other similar guns for fit. I really like the Browning and the Ruger. Something like a 2-7, 1.5-6, or even a straight 4x compact scope fits these rifles well.But, if you don't like these there are plenty of other options in standard size rifles.

In my opinion, most people buy guns that do not fit them well, especially when you factor in the clothes you will wear when hunting. One part of checkinga gun for fit is, when you have the rifle shouldered properly and a good cheek weld to the stock, see whether your eyes would be in line with a scope. On a lever action gun, your eyes should line up with the iron sights. On a bolt gun that will be scoped, the stock should have less drop, putting your eyes about 1.5-2" over the bore. The bolt should be comfortable. With your palm facing up, raise your hand naturally under the bolt handle. The handle should contact the center of your palm, not your fingers.

Something most beginners do wrong is grab the bolt handle. You should not do that. Work the bolt with your palm. Much faster and more reliable when done properly.