Just fing the best trail the deer are using to travel accross the property you hunt, to get to his bait pile, and back up that trail some. Then when the deer are going to his bait, you shoot the best buck you can find, while it is still on your property. That way he helps to concentrate the deer for you, and you hunt them on your property. And if you are not so close to the property line, there is less risk of the deer going on to his property before recovery. You are just cutting them off before they get to him.

Use his being a slob hunter to your benefit, and relax.

God luck and good hunting.