In a least some states its who puts the tag on the deer. But in all I know of the land owner can have trespassers arrested.
I personally have told the neighbors that if they can show me a blood trail coming on to my property I will let them track down and take their deer. I do not consider it good sportsmanship to refuse to let some one track down a wounded deer or hog.