That's kind of like deciding whether you want to buy a Corvette or a Peterbilt. Both of them can go fast and if either of them hit you at 90 MPH, you're dead. On the other hand, if you are hit by the Corvette at 30 MPH you have a somewhat better chance of survival than if hit by the Peterbilt at 25 MPH so all things are relative. Energy = Mass times velocity squared so the energy is dependent upon the velocity and (usually) the larger the caliber, the more mass you can put down range at (sometimes) a greater or equal velocity.
You don't give much information about where you will be hunting, what you will be hunting or if you are in to pain. The pain comes in when you factor in another law of physics, every action has an equal and opposite reaction, therefore the more "stuff" you put down range, the more "Stuff" you have coming back into your shoulder. This may be mitigated to an extent by using a rifle that is heavier in weight and with more surface area in the butt plate to spread the energy over a greater area, there are also systems of porting the muzzle to help alleviate some of the perceived recoil. Some folks (see elkampmaster) have a propensity for large caliber rifles and insist they can shoot one all week and never develop even the slightest notion that they had been shooting anything other than their kids Daisy BB gun. Along those same lines though, the Chinese, with a population well in excess of one billion, insist that their national past time is ping-pong, some folks will lie about anything.