Oh yeah, happens to me regularly. They are pretty hard to tell, unless they are REALLY close. Buttons are generally more curious than does, and they are also a little bigger than a yarling doe, which, I think is why we shoot them first, they are a little more valuable because they are bigger.