I've learned 2 things throughout the years that helped me reduce my "deer hallucinations".
One... I've learned to tune out 90% of the noises I hear in the woods. My head doesn't turn with every snap of a twig or leaf crunch. I can tell the little varmits and the difference in the sounds they make as opposed to a deer. There is just something about the sound of a deer that is pretty much unmistakable to me when it happens. Like someone else said too, they just seem to appear out of thin air without being heard most of the time. I rely on my eyes more than my ears.

Two.... It is also unmistakable when a deer is actually in your view. I've reduce examing every branch, piece of swail, or stump trying to make them into a deer [8D]. When a deer is in your view, you will find yourself not looking half as hard to try to identify it as you do with the "hallucinations" of deer.

It's good to be alert and constantly study the woods, but you can use your senses to the best of your ability and help save some sanity.