Give it a try. You've been busting them out of there anyway. So walk in at a regular walking place and see what happens. If on busts you, don't even stop or look at it. Just keep walking towards your stand. Maybe it'll decide you're nothing to worry about.
I'd try to find a different approach if possible. Always try to avoid pushing deer out of your area on the way in/out.
-Jake