I've had people start to setup 50-75 yards away from me on public in the past, then I whistle a little to get their attention and MOST move on waving sorry. Twice however I have had people blow me off and set up as they had planned which was fairly close, and at first I was very annoyed by it. Even more so when I noticed them moving around a lot... But then something cool happened. The first time it happened I watched a buck clearly see the guy moving around trying to get to his bow, so I grabbed mine and sat patiently waiting to see what happened. The buck stayed back about 70 yards from the guy and circled around the guy keeping an eye on him but placing him just 20 yards away from me. The buck caught a ride home in my truck that day, and the other hunter cussed me out for shooting HIS DEER that clearly avoided him like the plague and walked right by me. His loss, my gain.
The second time I had that happen the guy actually flipped me the bird and said out loud that it's public and he'll sit where he wants, with a few other adult works added into it... Ok... So about an hour into the hunt a few does came by between us. I wasn't going to shoot one but still had my bow in my hand for just in case. Well the guy decided apparently to be a jerk and said BOO which obviously spooked the doe right past me at a run. Unknown to both of us at the time, there was a decent 8pt buck slipping in towards the does from behind the other guy. When he yelled BOO the buck busted out running right at me. When he was 25 yards out I drew my bow for if I could stop him. He seen me draw and locked up the breaks...His mistake and he too went home in my truck.
My point... Yes it really REALLY stinks when people are so rude to setup by you if they know you are there, but at times it can actually help you and you'll also get to rub it in their faces a little bit too.