Hunt. Its the rut, you never know what will happen. It's not like your hunting deer in a "normal" pattern early in the season. Once the bucks start chasing, there isn't any rhyme or reason to a bucks, or does, pattern. I've had deer blow at me walking in, hear them run away, and then come back 20 minutes later completely oblivious.
I'd be a little more concerned if they made you in the stand after daylight. If its an older doe that sees you, you may have problems. I don't worry about bucks much, they have pretty short memories (unless they're with a doe that makes you).