You can kill a deer with a .243 or you can kill them with a 50cal. sniper rifle. Dead is dead. I just like having a heavier bullet. I also think that we owe it to the animals to use a weapon, bullet, shot placement that produces the least amount of suffering as possible.
All it amounts to is comfort and confidence in what you shoot and personally, I would not feel comfortable nor confident in shooting deer with a .243. That doesn't mean you feel the same.