While this might be moving ahead quickly in pursuit of your deer hunting desires. I'd suggest you get a 30-06 as your first rifle. In your research, you will find that this is one the most popular cartridges ever, and for good reason. You'll have more options as to brands of firearms and availability of ammunition. Through your reading, you will note that many of the offerings available today stemmed from this cartridge. Also, when your first start shooting, there is factory made reduced power ammunition that will help you to get comfortable shooting a high power rifle. Judging by your size, full power loads wont bother you at all. The 30-06 won't beat you up.
I'm of the opinion that a fella that owns a 22 rimfire, 30-06 rifle, and a 12 gauge shotgun can hunt about 95% of game on this earth.