That's the curse of the trail camera. It will second guess your every move. For as many times as you miss a buck by sleeping in, I would venture to guess you see plenty at 4:35 on a .....Tuesday afternoon while you would have been at work. I know it happens to me alot.
I always feel guilty for the hunts I miss when the season is gone unless it was a productive season. I always say I'll hunt real hard next weekend and before you know it, the season is over.