Different calibers are able to handle different weight bullets well, but pretty much anything you buy off the shelf should shoot ok. Like you say there will be some that work better or worse for your rifle, but your contribution to the inaccuracy will probably be more than the contribution of the rifle/ammo once you're off the bench (I know it's true for me). Agreed on 150 or 165 grain for deer.

The .308, .30-06, and .300 Win Mag all fire the same diameter bullets with different case sizes, so they have different amounts of powder they can put behind it. If someday you were going for bear and wanted something bigger you could easily shoot 200 grain bullets, while a .308 really suffers when it gets up there. And a .300 win mag could go even higher.