If the deer was looking at you, he was on alert status to begin with. Next time try to hold off on letting go until he turns his head. You failed to mention your equipment so I can't tell you if speed is really a problem. Even with moderate speed, you should just be able to aim for double lungs on an unsuspecting deer at 20 yards. They can move pretty darn quick so your best chance is waiting for a better shot.