There is a lot of .243 lovers and all the power to you, it is indeed a great cartridge and I will not debate otherwise.
As far as the varmint side of calibers the .243 is one of the best choices when you are considering jumping up, to use it for Deer, and probably the minimum IMO.
I chose the .270 for the simple reason of misplaced shots on deer. The .270 will allow for some "Murphy's Law" that happens while hunting.

As an "example" of my claim... While hunting in TX back in 2007, a gentleman gut shot a deer with a .243. Now I'm not claiming that it would have died quicker with a .270, as mentioned it was a bad shot, so let's get that straight. However, after recouping the deer the following day, some 500 yards away, the guided said... In his 28 years of guiding, "the heavy majority of the deer that are wounded are from people that come to camp with a .243". He has always recommended @ least a .270 for this reason, but doesn't enforce it. As he too says "a well placed high percentage shot from a .243 will down a deer just fine".

So there you have it, my opinion why!

If your are a marksman that has never compromised a high percentage shot.
OR
If your deer has never moved / twitched while squeezing the trigger, then a .243 is your poison, a little extra can never hurt though???