I had the same thought when i saw the title...
I read an article once that tracked this very thing in a scientific study. They were saying that most mating actually is by the less dominant bucks.
It was mainly a matter of numbers. The big bucks simply weren't numerous enough to mate with all of the does, and that gave the smaller bucks plenty of opportunity.
It did vary though. Opportunities for the lesser bucks came in the peak of fertility.