First, the average hunter has little control over genetics, unless they own and control 5000 acres of deer habitat.
Second, deer don't live forever and great antlers don't last forever.

Besides, hunters don't get to decide what deer is breeding what deer.

So, I never worry about it.