Sounds to me like you are shooting at the "whole deer". It's very hard to do, but try not to look a those antlers!!! Pick the exact spot on the deer where you need to aim, and stay with that spot all the way... It takes some practice, especially when it's a decent buck.
I think everybody gets buck fever, and the day that I stop getting that feeling will be the day that I give up on bow hunting. (I hope that day never comes!)
One final note, it'a a good idea when you see one deer is to scan around the area to make sure there isn't more than one deer that can discover you, and blow your cover!