After I started talking with my neighbor I started understanding why he's so bitter at times. I've been living in my current home for about 2 yrs now but never really had a chance to talk with him before. He and his wife keep to themselves and it's rare to even see them out. Anyway, they own 4.5 acres of property which they section off and give 4 hunters (including the owner) each 1 acre to hunt. The problem is that there is alot of hunting pressure on nearby properties and other hunters often tresspass onto his property with no permission. In the past month they've had 2 ladder stands stolen from their property. After we talked for a while I explained to him where I had stands setup and proceeded to explain where my brother was hunting the day he shot his buck. He then said it was ok to retreive the downed deer and thanked me for taking the time to speak with him in person. He did reiterate that he still does not want people tresspassing on his property however if something like this should ever happen again not to hesitate to contact him.

Lesson learned!