Why does the primer blow out of the breach plug when you fire
the gun. Not every time, about 4 out of 10 shots.
I use shot shell primers.

brand and model, what type of powder, how many grains of the powder, what brand of primer and what type of bullet bullet?

that would help figure out what is going on.

Connecticut valley Apollo 50 Cal.
Thompson Center 240 grain XTP Sabot Hollow Point
Pyrodex FFG 80 Gr.
Win. W209 Primers

Is the Apollo a plunger style muzzle loader?

If I remember right wasnt the Apollo the gun that had many injury claims filed against CVA?

Yes it is plunger style

95 and 96 apollos were the recalled guns.

It is a plunger style set up so the primers popping out could just be normal.

Even if normal it wouldn't be desirable in my book

I run the S/N on the recall and its not one of them.
The gun shoots great, I dont know if its normal for the primer
to kick out like that. I just dont want to damage the gun because
of that. Do they make a stronger spring to help with this?

In all honesty, that's not something I would live with. Possibly the flash hole is eroded allowing excessive blow back pressure. There may be a fix by modifying the breech plug, but I'm not familiar with that model or its breechplug.