In your case I would get the 280 hands down. It has more than enough power for what you want with reasonable recoil. I might further suggest a 25-06. Mild recoils and good killing power out to 300-400 yards.
Yes the 300 WSM is a very good round. It is however much more gun than you need for deer.