No, it's not. I have to pry mine off. Some brand of caps are too short for some brands of nipples. You need to find a good match.

Are you using Remington caps? They're on the short side.