I would just take it. He comes to the door complaining you can decide if you want to give it back or call the sheriff at that point. He had no business with it on your land. If he is sincere and HONESTLY made a mistake about the boundry line, than give it back. If he blatantly disregarded the property line...well tell him if you want to dance...u have to pay the fiddler.