For all you have listed that your rifle needs to do the one caliber that will fill your needs is the 30-06 rifle and even loaded with the lightest bullet it is still over kill on varmints, but ammo is certainly cheap for this caliber.
If you ever began to reload it will shoot all the light bullets the 308 does, but with more authority. The 30-06 will shoot 110 gr. bullets all the way to 220 gr for grizzly. Shooting 1000 yds will be boring using 208 gr AMAX bullets.
Biggest deer I;ve shot I was using 110 and 125 gr. bullets. But my go to bullet is 150 gr Remington Core-lokt. For elk you will be fine using 165 gr to 180 gr. bullets.