Genetics are an underlying factor to antler growth no doubt, however maturity level is the main factor in determining his potential. Other factors such as food, water, stress, late births, etc all have impacts on year to year growth. I have seen some pretty small 2 years old grow some decent headgear in a year or 2. If at 3-4 they are still lagging then you should think about harvesting them. If your worried about immature or inferior bucks breeding you have a choice, that is harvest more does. If you bring the doe to buck ratio in line it will allow more mature bucks to do the lion share, thus eliminating your worry. QDM is more than allowing trophies to sprout it is to also create balance on all sides.