You may not have to wait that long:
http://www.dnr.state.md.us/wildlife/antlers.asp
"A yearling buck can support a six to eight point rack by fall. This dispels the misconception that the age can be determined by the number of points. Age is determined by the replacement and wear of the teeth."
Dan O.