If we are going to use that argument then why not just use the .243 with 100gr bullets? Or we could always just step up to a 150gr with the 30-06....the difference in drop at 500 yards is going to be somewhere around 2".
Again, the majority of people no matter what they tell you still only shoot around 100 yards so in the grand scheme it doesn't matter too much.