You would have been busted. You did the right thing by watching them and resisting pulling back.

I've learned over the years to watch the squirrels and the deer. You can learn a lot from watching those first set of does come in and if you looked closely enough you might have seen them give signs and glances to the others that snuck up on you from behind.

One year I was hunting a small opening in the woods with thickets all around me. I watched the squirrels feed anc chase one another all afternoon. Then, one of the squirrels bolted to a tree and started squawking. I knew it did not see me. What it saw was a few does making their way thru the thickets that I did not hear myself. However, watching that squirrel gave me the heads-up I needed. Dead doe about 2 minutes later!