If QDM didn't improve herd health then you wouldnt see bigger antlers (keep in mind that herd health incudes a balanced buck age structure, so the argument that they're only bigger cuz their older is null and void).
All the research shows that as buck age their average rack size increases whether it is a QDM buck or not. Just letting more bucks live one year longer will result in an increase of the rack size of the average buck being harvested. But, in Miss. the average rack size of 2.5+ buck decreased across the entire state due to the effects of high grading.