Do you pick the nipple between shots? That will help sometimes. Could be the nipple. One reason I cringe when I see people dry fire their traditional muzzle loaders. They eventually flatten the top of the nipple and it can make seating the cap difficult. Also, when you pull the nipple for cleaning, be sure to check the drum for fouling.
I actually have a rifle that does the same thing.