Neighbors can be some of the nicest people on earth then deer season comes and they turn into the biggest A HOLEs on earth.
Michigan law states you can not trespass with out permission period. If you are tracking a wounded deer on my property you came and asked for permission to do so before the season started. If I see you I will have your picture and will prosecute you for trespassing. It is only neighborly to ask ahead of time no if and or buts.
No one is forced to trespass any way shape or form.
Of course isn't much can be done with what is done on their land. Is why I keep my blinds and stands away from the property lines, well away. I try to make all the hunting improvements in the center of my property and food plots with insight of the house most times. trail cams patrol the property lines too and the neighbors know also.

Al