It only takes me 6 minutes to forget that I spooked him....

I have spooked plenty of deer. I suppose if there were two or more deer and they went a couple different ways. They may try to come back and find the deer they were with. But I haven't seen them come back, unless maybe different deer all together.