If it were me I would rip the stand down and talk to the person who you think may have built it. In addition, it couldn't hurt to speak to the game warden and let him know what happened and to keep an eye out.

I definitely wouldn't leave it there.

With regards to the paint, I think that it should have dried relatively quickly and only affected deer movement for a short while. What is more likely is that who ever built it has been hunting it and spooking deer out of the area.

Just my thoughts...good luck.