Even more unusual that it is a hard-horned deer. True antlered does stay in velvet. A deer needs a high testosterone level for the antlers to shed velvet which would require testicles. When you field-dressed it did you find the uterusor happen to check for undescended testicles? More than likely it was a buck with malformed reproductive organs or a hermaphrodite (both sex organs). Here's some info from a DNR website:

What about "does" with polished antlers? For the velvet to die and the antlers to become polished bone, a second surge of testosterone is necessary. Reproductively functional females will not get the second surge. Deer that appear to be does with polished antlers are almost always reproductively malformed males, which will have a second testosterone surge that causes the antler velvet to shed. Postmortem research on these deer shows most are cryptorchids, hermaphrodites with male organs predominant, or pseudohermaphrodites (animals with external female genitalia but internal male reproductive organs). Because its antlers were large and polished, Tom Schneider's 13-point deer likely was a pseudohermaphrodite.