Bummer. Depending on your hunting locale, it may not affect him that much. I hunt farm country where there are plenty of humans in and around the fields and fences before and during hunting season, and the deer don't tend to change their routines if you happen to be spotted as they might in the deep woods. In hindsite, I would probably have sat tight after seeing the forkie and gotan arrow ready anticipating a bigger deer to follow as is often the case. I wouldn't fret over it too much.