I'd still call your local game warden and explain what happened. It can help with a few things. 1 - It will get your game warden familiar with this landowner. 2 - The warden can pay him a visit and see if anything is out in the open that shouldn't be. 3 - It gives you the opportunity to discuss what you should do in the future should this ever arise again.

Now, a couple of questions/assumptions.

Don't all game wardens have access to private land? I thought that was the agreement - that they protect all the lands, not just public lands.

Can a warden intervene and move the deer from one property to another?

I'm betting the farmer doesn't hate hunter's that much. What are the odds that he's enjoying some nice deer steaks right now?

Call the Warden...