I've found the little .32 to be very finicky. Before you go out next time why don't you try weighing a volume of powder (say 20 grs) and then weigh a few more charges with that weight. Then do the same for a couple other charges. Remember, you're only dealing with low charges here so a 2 grain difference in a 20 grain charge is a 10% difference whereas with a big 50 cal using a 100 gr charge, that same 2 gr is only a 2% difference. I really think weighing your charges may improve your grouping.