Drill presses have a rack-and-pinion set-up for lowering the quill, compared to a direct lever drive for a loading press. I'm not sure what the pressure requirements for swaging bullets, but it could be enough to mess up your drill press.
Paul