When you catch them... beat them senseless.
Just kidding. LOL! Not sure how you know the trespassers are the same group that come and talk to you, but if it gets worse, may have to try something pro-active, like setting up cameras, etc. Or maybe instead of hunting deer, set up a stand to watch the road where they're coming from... if it's that bad.
If/when you do catch the culprits, I'd be just as ticked off towards them as I would be you if you didn't prosecute. Some people are too soft. When the party gets a ticket for trespassing, maybe then the word will spread and it will stop. ?? Be sure to prosecute!
Good luck,
iSnipe