Basic rule of thumb.
For a rifled barrel, use sabots.
For a smooth bore, use rifled slugs.
Each type ammo perfoms best, for several reasons, in specific types of barrels.
I just put a rifled barrel on my kids 20 gauge. With Winchester Partition Gold Sabots, you get 1900 fps with a 260 grain bullet. Zeroed at 150 yards gives -3.0 inches at 175. So, from muzzle to 175 yards you have a bracket of 2.9 inches high and 3 inches low. Not bad at all and pleanty of energy, over 1260 at 125 yards. You can probably find similar ballistics info at
www.winchester.com/products/catalog if you do a search.
With the best choke you have and a 3" mag buck shot round your range is probably close to 70 yards max (some might get a tad more). But with a good cantalever scope and a rifled bore, you should be able to do 125 to 150 easily.
Good luck
Hank