That buck knows that something scared off a doe, could be a coyote, bad smell, loud mouse, etc. So my guess would be he will be back. The doe, however, will be looking for you in that tree. To move or not to move? What are the odds that the next time you see a buck come through the same doe will be there...I think that is what it will come down to...