I'm not entirely sure on this one. Without knowing all the details many ideas or pieces of advice may help but only you can determine what you want to do.

If neither of you own the land but you have been parking there and doing this for almost the whole 40 years you been hunting there then continue on.
If you just started to recently or within the last few years then working something out that works for the both of you might be ideal.

2 armed hunters getting into a dispute over something pretty trivial like this could escalate quickly. If your confident you haven't done anything wrong and your not gonna allow someone else to dictate how you hunt then I would definitely let that person know and even the land owner.