in this rare circumstance i would say it's certainly up to the landowner to make the final call.
i'm a big advocate for getting permission and respecting other peoples property as far as hunting is concerned but this situation definitely is an awkward one.
while it may rightfully be his to keep being it's his land, the landowner kind of took advantage of another hunters misfortune. i guess a lot depends on what extent the hunters went to in trying to obtain permission or if they just walked up to the house and with nobody home helped themselves.