davidmill, I don't know what your state has for laws like this, but this is what Colorado law states. I would suggest that you get a video camera and tape this guy harassing you and the wildlife. As you can read it could cost him much more than the 100 - 500 dollar fine if convicted. He might think twice before he harasses you and the deer again.
Glad his plans backfired and you got your deer.

HOW TO HANDLE
HUNTER HARASSMENT


The Law On Hunter Harassment

33-6-115.5 Hunting, trapping and fishing – intentional interference with lawful activities.

1. No person shall willfully prevent or interfere with the lawful participation of any individual in the activity of hunting, trapping, and fishing in accordance with this article.

2. A person commits intentional interference with lawful hunting, trapping, and fishing activities if he:

a. Acts with intent to alarm, distract, or frighten prey and causes prey to flee by:

Use of any natural or artificial source of noise or light;
Giving chase to prey on foot or by use of any vehicle;
Throwing objects or making movements;
b. Intentionally harasses any person lawfully participating in the activity of hunting, trapping, and fishing by use of threats or actions;

c. Erects barriers with the intent to deny ingress to lawfully designated hunting, trapping, or fishing areas;

d. Intentionally interjects himself into the line of fire;

e. Engages in any other conduct with the intent to disrupt or prevent lawful hunting, trapping, and fishing activities.

3. Any person who violates this section is guilty of a class 2 petty offense and, upon conviction thereof, shall be punished by a fine of not less than one hundred dollars nor more than five hundred dollars and an assessment of twenty license suspension points.

4. Any person convicted of a violation of this section shall be liable for all damages incurred by the individual whose lawful activity was obstructed and for all court costs of prosecution.

5. Nothing in this section shall limit the actions of law enforcement officers and personnel of the Division of Wildlife in the performance of their official duties nor apply to landowners, tenants, or leaseholders exercising their legal rights to the enjoyment of land, including, but not limited to farming, ranching, and restricting trespass, nor will anything in this section be construed to prohibit any incidental interference arising from the lawful use of land or water.

(Editor’s note: Section 2 of chapter 241, Session Laws of Colorado 1988, provides that the act enacting this section is effective July 1, 1988, and applies to acts committed on or after said date.)

Avoid Confrontation

If you feel you’ve been harassed, as a hunter you must act reasonably and ethically. This will reduce conflict and minimize media coverage, a tool which may later be sued in your disfavor. Understand the hunter harassment stature and its application, and work to assure public safety.

When a person has or clearly intends to violate this wildlife law, avoid confrontation, then seek law enforcement assistance. Contact the Division of Wildlife, the Colorado State Patrol or local sheriff’s department.

As with any wildlife violation, if illegal activity continues after issuance of a summon, the offending party will be placed under arrest.

Some Tips On What To Do If Confrontations Occur:

Remember that your behavior is representative of all hunters in North America.
Strictly observe all gun safety rules.
Don’t do anything that could be considered threatening.
Show the utmost respect for all wildlife.
Avoid an argument.
Remain calm.
Report the incident.
Remember that your commitment to hunting is stronger than their opposition.
These are a few of the suggestions from the brochure published by the National Rifle Association of America entitled "Hunting’s Future?…..It’s Up To You."

GOOD LUCK AND GOOD HUNTING