it's tough on either side of the coin.......as said above , do it 'tastefully' and you should be ok --- kinda similar situation behind alot of homes i'm sure{mine included but not a mansion - lol}.....i would hunt carefully and get permission to recover!
as a side note you might want to check out the guys from team-overkill..............they have a dvd out about their suburban hunting{some with mansions around} - or contact them for further advice / i know those guys could shed some light in the delicate areas --- good luck