The burden of prood would be on you to show he was trying to shoot a deer on your land. I don't see how this would be possible, he could always say he's shooting deer only on his land.
I'd try and catch him shooting a deer, video tape him, and press charges.