I am fortunate where most of the neighboring landowners wouldn't care and would likely want the deer off their property.

That said, yes, I'd retrieve the deer and pay the fine if necessary. But I would try not to put myself into a situation where that could possibly happen in the first place. Meaning, I'd try to hunt at least a 200 yards from a "strange" property line.