My experience has been a bit different, though you are no doubt a better and more practiced shooter than most hunters. I' ve worked rilfe sight in clinics for several years and have seen litterally thousands of shotguns sighted in. I have seen a few really accurate shotguns, but at best it would be about 1 out of every 20 I see. The few Ithacas and Brownings I see typically shoot really well, and the mossberg bolt action slug guns seem to shoot pretty well also. The few muzzleloaders I see tend to shoot much beter than the shotguns, my own smokepole shooting roundballs will outclass just about every shotgun I' ve seen.
I often wonder if its the shotgun or the shooter behind the inaccuracy, I' m leaning toward the shooter. 12 guage slugs kick hard, and for a person who dusts off the shotgun and sights it in once per year may waste his best few shots, trying to get the holes adjusted into the bullseye. What usually follows is shots start creeping all over the paper- maybe because the thin barrel of the shotgun is starting to warp from the heat or the shoters shoulder is screaming for mercy after 3 or 4 shots- maybe a bit of both. I' ve also seen a guy bring out a TC encore type muzzleloader and put 10 shots in an area that would easily be covered with the palm of my hand at 100 yards- probably as accurate if not more accurate than alot of hunting rifles out there. Less recoil, thicker barrel less prone to warping, and a more experienced shooter probably make the difference there.

Now, I have to put in a disclaimer- I don' t shoot shotgun slugs and have no interest in them, but have observed many others shoting them, with me occasionally finishing a sightin if the shooter' s shoulder gets to sore.