I cast from a LEE mold. The spur is not that bad out of the mold and I never saw a reason to tumble them. I do know of one person that sends his through a rock tumbler. And to be honest, they shine nice, but look no better them mine. Also he's no more accurate with his balls (I have shot a few of them and they are fine) then I am with my fresh out of the mold.
I do weigh my cast balls and separate them by weight. If they are too far off they go back to the melting pot. But normally they are pretty good.