Hey shooter, both barrels should hold zero. What you would be doing is no different than taking a barrel off of a Renegade to clean it.

If you don't mind an un-asked for opinion, I would go just the opposite of what you plan. Why not use iron sights on the .54 with balls. Perfect for your 50 yard shots, but also goodfor 100 yards easily. Put the scope on the .50 barrel and work up the hottest load you can with something like Hornady's FPBs and get it dialed in for long shots.