Well, I've seen wads that shoot 357 bullets out of a 45 cal and you can get .350" balls.
For the 50 cal using the right sabot you can get balls in 40 cal (.395"), 44 cal (.433") and 45 cal (.451")
All the above are listed on Hornady's site.
IMO the .433" would probably work the best out of a Harvester CR sabot. But I think this is bigger than you had in mind.
I know you got Semisane thinking now.....and he's going to try something along this line. I just know it.