First "long range" rifle I ever owned was a Remington 700 in 25.06 Rem. I had no issues with killing deer sized game cleanly. I was hunting a lot in western Alabama back then ... over and around the soy bean fields that were once wide spread across that area of the state. Shots in the 250-300-350 yard range were not uncommon. I used a hand load which incorporated a Nosler Partition bullet. You can certainly load up (or buy factory loads) blistering fps loads with the lighter weight bullets, but I would not suggest going below 100 gr. for deer sized game.
I sold that rifle many years ago and have regretted doing so many times!