How much corn does a supposed "corn-fed" deer eat? Between the time the corn is mature and when it is combined, what do they have? 2-3 months? And the other 10 months they're eating mast and browse.
I'd say it has to do more with how it's killed and how it's handled after the kill. The best deer are the ones that have bled out the most. Blood in the flesh will taint the meat and give it a gamey taste.