I have a spot just like that, and it's my morning spot as well. Deer can (and do) come in from all directions. I'm always super careful about checking everything in all directions before getting down, but there's always the chance they are just out of sight in the pine thicket to the west. You will be fine to go back in after a while, and like has been said here, the best of the season is yet to come.