If you are wanting one gun to hunt anything in North America, it's dang hard to beat the 30-06...When you get much higher than that then many shooters start learning to flinch...

As you gain experience, you will end up with more guns and speciality guns...I will say,I have seen a trend towards more experienced hunters moving to a lighter kicking round for deer than the 30-06...
It is a good all round caliber but it is also more than what is needed for deer...