I had that happen one year. We had a "Earn a Buck" program, where you had to shoot a doe first. And that first night out, sitting in a blind at under 50 yards, in walks a monster buck. I sat there with my scoped Black Diamond XR looking and drooling and could not pull the trigger.
I did later shoot two does, but never saw that buck again.