IF you had tags for either sex, then brown and down and all legal fair and square. IF you shot a buck without a buck tag and used your father's tag, then let's not make a habit out of that.
Invest in some GOOD binoculars or a spotting scope (no spotting with mounted riflescopes) and no what your shooting.
If all you did was violate your own made up 'standard' of what is a shooter or not, well you did not shoot a small buck, you shot a doe, or so you thought. As long as no regs were violated, chalk it up to experience and a reason to go buy some more optics.