Well,I hate to say it but I've been in the exact same situation before.
Our land boarders the biggest Dhead in the county's land. I killed a nice 6 point about 2 years ago and it fell about 10 feet across his fence.
I went and asked him if I could get him and he said no in a very smartasss way.
Needless to say......I got my damn deer.