I feel you man. The same thin is happening where I hunt. We have permission from the owner to hunt there. There are these hics that do the same thing...litter, piss, make noise, ruin the hunt for us legitimit hunters. These guys we deal with got permission to hunt there over 20 years ago, before the last owner (current owner's father) died. We approached the current owner and got in writing that we had permission to hunt/fish there. It also states that to his knowledge; no other people were granted permission to be there. These guys also shoot doe on a buck only day, and never even bother to look for them. This guy is disabled too..he has a heart condition. Are we talking about the same place...sounds like we have a similar problem. LOL. I wanna beat these guys a$$es too, but we maintain civil conversation. This hic even threatened my Dad..which deserves an a$$ beating in itself.
I would talk to the owner if I were you, and see if you can get these guys uninvited so to speak. They may not even have permission to be there...especially if he is bringing un-invited friends to tag along. You may also try talking to the guy to "persuade" him to locate another spot, or alternate hunting days...but if he is ruining it like you say...that will do no good. I feel your frustration!!! Try to remain nice as long as you can. Appear to be the bigger man and the more ethical, polite hunter, and you may get the place all to your self eventually.