How big is the property? If it is a couple hundred acres and you have to use a short range weapon then there is plenty of room for both. If you had permission to hunt then only the land owner can tell you to leave.
I would have found a way to hunt regardless what he said.