There are trhree things that contribute to large antlers. Genetics-nutrition-age. That is all and all three must be in concert to attain maximum antler size. That is, maximum size for that area.
Now here is the major rub (pun intended). We have no way of knowing for certain how much the genetic input from the doe contributes to the antlers. We also have no way of selecting for certain, the best does.
Now, I'll complicate it even more. A buck begins to grow antlers as soon as he sheds the old pair and the testoerone is redirected. The antlers continue to grow until the velvet is shed and the testsoterone again redirected. Antlers may grow and inch or more per day. Therefore, would we not like to have an early shed with more time to develop hard antlers? Of course we would. How could we achieve that?