I sort to a standard deviation of three grains high. If they are two grains or more low, then they are picked out also. Also as I sort I check for blemishes. If any of them have scratches, or pits in the spur (they usually show it in weight too) they are also set aside for recasts. The beauty of casting your own means you can always recast your mistakes.
I did once when sorting keep ten real heavy ones. And shot them just to see if they would shoot different. Now granted I am not a long distance shooter like MD. But out to 50 yards I found no difference in accuracy.