I don't know what state you're in, but I believe that here in Ohio if you shoot a deer on someone else's property without permission, no matter if you're on the property or not, it's poaching. If he's set up facing property he doesn't have permission to hunt, you might want to get the wildlife officer involved.