Where rifle hunters after turkeys get a bad name for themselves is when they start taking potshots at birds they can't tempt in close with a call. Calling is most of the fun.
But that doesn't rule out the rifle. I know guys who use .22 WMRs, but I've probably seen as many birds hit and lost with those as I have with shotguns. I can't say that I've seen a turkey lost when hit with a centerfire rifle, though. My favorite turkey gun is a .22 Hornet. I've used both .223 and .243 as well. Another popular rifle chambering I've known people to use is the .221 Fireball.