Sometimes they spook when they expect a verbal reply or movement from the hen decoy they see, but don't get the response they are expecting. Same as when they are looking for the hen making all the noise but don't see her anywhere when you don't have a decoy out.

Solution: Catch a real hen and tie her leg to a string and stake her in front of you (kidding).