I usually only worry about the adult does. For example if there are 2 does and 4 fawns together andI am after an adult doe, I only pay attention to the two adults. The fawns seem to stand and look at you, but they arent experienced enough IMO that they are going to start snorting or stamping and you can usually draw even if they are looking at you. Not always, but usually.

If it is all adults, I try to keep an eye on everybody and draw when everyone is looking the other way or behind something. When thats not possible, remember even if you spook one of the deer, you should have a second or two before the one you want spooks to.o So try and get a good quick shot off.