The setup you describe would have the scope mounted to the receiver, as opposed to the barrel. The barrel and receiver are not really locked together precisely and the barrel can move a bit between shots. This will drive you crazy when sighting in if you are expecting much accuracy. Some guns are better than others though. To minimize movement of the barrel, you need to tighten (but not overtighten) the barrel nut. A friend found he needed to use vice grips and a rag over the nut to get it tight enough, but I don't recommend that.

Also, realize that even if you're able to get the gun tight, 20 ga rifled slugs are about a 75-100 yd proposition anyway. If you find a slug that will give even 6" groups at 100, that's good enough for deer. Sight in at 50 yds to minimize the effects of wind. You'll want the group about an inch or two high at 50.

You do have the option of getting a smoothbore barrel with rifle sights. This will allow you to use rifled slugs which are far cheaper than sabots. With my smoothbore Mossberg 500, I could shoot deer to 75 yds no problem. Sights that attach to a smoothbore vent rib barrel are a cheaper option still.

Another option, and probably the best option, is to get a rifled barrel with a cantilevered scope mount. This will extend the effective range to 150 and beyond depending on the shooter.