With my deep cut round ball barrels velocity is not a problem...I've used from 50-120grs of FFF Goex in my .54 without the groups opening up...Once you get to the higher powder charges you might have to put hornet's nest over the powder to keep from burning through the patch...

With real black powder and the proper ball/patch combo in a round ball barrel you aren't going to push the ball too fast...