Well that's what happened to me last season. Had a nice 8 point come in about 50 yards away in some thick brush. Just when i thought he was coming right to me, he beds down. So i wait, and wait and wait for over FOUR hours! In the meantime a big doe comes in and hangs with him and a spike walks right by me about 10 yards away, i let em pass. The bigger buck never even looked to see what was walking by. I was tempted to climb down and trying to get a bit closer, but stuck it out. Finally he jumps up, don't know if he got a whiff of me, but he was nervous. And he walks out the other way! With a rifle he would have been grilled backstraps, but i was with shotgun and handgun. I was pissed but what else could i have done?
Btw, this 10 point non typical made the mistake of not bedding down the next morning.