IMO the 300 mag is over kill for deer size animals but a great round for elk. The 308 is on the lighter side for elk, although it is plenty effective, and about perfect for deer.

The question you need to answer is what do you hunt the most. If you hunt elk as much or more than you hunt deer,or on a combination hunt where you have tags for both, then the 300 would be my pick. If you hunt deer more than elk then I' d probably opt for the 308. A great compromise would be a 30-06 .280 Rem., 338-06.

Having said that if you already have the 300 and are not having a problem with recoil and it shoots good for you then keep it. After all you can' t kill it too dead.