Dude, I don't know where you live or what your situation is, but I would cut my losses and never go back there. Even if this deviant all of the sudden left for good, you would still have to put up with other hunters not considering you or your hunt(s). Bag that joint.

I am a bit biased as I have private land accessable to me. If you can secure something like that, try your best to gain rights to it. If you don't then just don't leave anything in the woods. Bring it in, take it back out with you. Also, look for a place way off the beaten path, although it sounds like you already have. There is no use in trying to reason with a person who is going to to act like that in the first place. If he doesn't consider you now, he may at best say that he will in the future. I personally wouldn't bet 5 cents on it.

Good luck to you.