You might have spooked them. Id let it cool off for a few days. It sounds like you had a swirling wind, or were in cover that caused the wind to swirl. A little more info on where you were located would make it easier to visualize. Try to find somewhere the wind will blow in one direction. Try to set up where the deer will have difficulty getting behind you and the wind in your face.