When I check my area's it's usually during mid day so as not to have the problem you have run into .Those deer were there probably because of your food plot and will likely remain there as well .I'm sure this has happened to them before with farmers and other's out for walk's .I wouldn't worry about it they'll be back they're going nowhere. One thing that I don't do is approach this area at all for about a month before season ,Some might disagree with me, but it work's for me .

nubo