A big question is, did you find the arrow, or did you for sure see the arrow miss him. A did when hit will often jump like that. If you didn't find the arrow, you need to make sure you didn't hit him.
If you know it was a miss, then it sounds like you could be ok to go back.