Doegirl75 - powder could very well be the problem. It would be one place I would look to. Also you did not mention how much and what kind of Pyrodex you are using. Pellets or loose, and the amount is important. Many times people try to push the projectiles too hard.

I would get a brand new jug of loose powder. Pyrodex RS is fine. Get a powder measure and set it at 80 grains. Start your shooting on a clean oil free barrel and no bore butter please. Load the 80 grains and a 240 grain XTP would be a good starting load. Start your target at 13 yards if possible and have it hit dead on the bull. When you can group them there, then you can move to bigger distances. If you can not because of range restrictions shoot at 13 yards, then move back to 25 yards.

If you are swabbing between shots as you say, and the rifle will not hold any kind of a group at those distances with new powder and proper barrel care... I would suspect you might have a BAD rifle. It might be one that has something wrong with the barrel and for this you would need to send it back to CVA and see if they can get the rifle to shoot accurate.

Open sights are usually the best indicater if the rifle is bad or the scope is bad. If you can hold a group with the iron sights, then start checking scope mounts and the scope itself.. like you have done.

Can you feel any flaws in the barrel when you swab, like a bulge or ring?