Actually, when comparing 1st year bucks that have nice racks compared to scrub 4's, I believe being born early or late plays a significant role.
Excellent point! I agree this along with nutrition play a big role. To take it a step further, the health of the doe could also be taken into consideration. A healthier doe will produce healthier offspring for a doe of lesser health.