If QDM didn't improve herd health then you wouldnt see bigger antlers
If you let young bucks live to be older bucks then they will have "Bigger Antlers". Especially when at the same time you are killing off does that would have competed with them for available local food sources. BUT, having more older bucks in a herd doesn't really do anything to make the herd more healthy. That is merely a more desirable outcome for people who like to kill bucks with bigger horns. Other aspects of a QDM program may benefit the herd in general but the above component is designed to do one thing only.
QDM is a very new concept compared to how long deer have roamed this earth. Who was here practicing QDM to keep the herds "Healthy" thousands of years ago? The deer did just fine before man was even hunting them at all much less selectively harvesting them in favor of optimum antler grow.
You seem a nice fellow and I mean you no insult. But at the same time I'm too old for you to pee on my back and tell me it's raining.