Are you shooting into a target or through a target. If you are shooting into a target(braodhead is in the foam) when you pull it out they will normally always close. Shoot through a thinner target into another target or shoot something softer so you can see the exit hole.

Good Luck!