We seldom had any problem, ahem, "harvesting" --- ah, screw it --- "popping" either mulies or whitetails in fairly short order on the opening day.
EKM, I don't think anybody will argue the chances of harvesting a deer is very good in most places in NA. However you said Trophy and once you attach this tag all species become a formidable opponent IMHO. To me adding the word Trophy to the hunt "kicks it up a notch". Granted there will always be more deer entries in the record book but that is not due to being easier or less of a challenge to hunt, it has to do with population. Here the deer pop is pegged at over a 1 million animals and Elk less than 3% of that number (under 30K), it only stands to reason that more trophy animals will come from the deer side vs elk. Any way you slice it landing any animal in the book is a big accomplishment and one not every hunter will do.
As to smarts they all have their strong points, although the Elk would have to be tops IMO in this department. I use to think Mule Deer were stupid, until I started to hunt Trophies and found just how stupid they are not! I think it all boils down to personal experience and preference. Some find challenge in new experiences while others just need to change their methods, goals, viewpoints, area, etc for the game in which they have always hunted. Kicking it up a notch to me is extended myself past my past performance and is not just subject to the game I have chased...it is simply finding new ways to increase or maintain the challenge and try to be successful.