You're fretting over something that hasn't happened. I wouldn't sit on the guys line or he'll be checking you or harrassing you. Put the arrow in the right spot and the deer won't make it that far. If he does, I'd slip in like a shadow and get out of Dodge. It only takes a few minutes to drag a deer back across the line. Either that, or don't hunt anywhere near the guys place. You say you "hear he won't let anyone retrieve a deer". I don't trust anything I hear. Before the season starts I'd go see the guy, introduce myself and tell him I'm hunting the adjacent property. Tell him what you heard and ask if it's open to discussion. "I'll bring No weapons across the line and will ONLY cross to check on a hit deer. If I don't find it quickly, I'll leave. Would that be possible". If he says NO, well say "OK... I had to ask. I don't intend for such things to happen, I was just making plans before hand. We'll both stay on our sides of the fence. Have a nice day". THEN... if it does happen, get sneaky.