Your deer tag is your deer tag. Use it to take whatever deer you want as long as it is a legal animal. In my humble opinion far too many people are obsessed with amount of antler an animal has. I see no reason for that. I hunt for the sake of hunting and for meat. Size of antlers, or even the presence of antlers, means nothing to me.
As always, feel free to disagree.