If it were me i would refocus my efforts on hunting deer to hunting that ___hole! There is no way in hell that he will spot you if you take the time to really conceal yourself into some brush or really high up in a tree. I would take a video camera and a gun with me and get the guy on film. I personally could not turn cheek and walk away. What might happen when the neighbor kids decide to go into the woods to build a fort or something, could you live with yourself if something bad happened?

Also what other kinds of problems did the officer say he has had in those woods?