I shoot the REAL 320 grain in my Trade Rifle and do not size them. In fact, I only size the bullets I cast that are to be cartridge loaded. If I'm shooting a ML, I want the lead to fill up the grooves upon loading so that it engages the rifling as much as possible.
With pure soft lead, you will be filling the grooves of the rifling, and sizing may rob some of that.