If the neighbor down the road is going on to property he doesn't own (50 acre plot you have permission to hunt along with the other two), then talk to the landowner about it. Feel him out on what he thinks about this neighbor. Be honest about the problems you have had and the problems this neighbor has created for the people who have permission to hunt. If the land owner doesn't want to do anything about it then you are better off finding a new place to hunt. At least that what I did in a similar situation - landowner, once he gave his permission told us "well he's my neighbor and I don't want any trouble between us because of hunting..."