I assume he was either a deer you didn't want to shoot or didn't have a shot? Perhaps he caught you off guard?

Deer can pin point the exact location of a sound, as you witnessed. After going to the sounds source, he didn't see a deer. He knew there wasn't one there, so why come back?

If he was deer you wanted to take, the only advice I could give would be be ready all the time , or use a decoy.