My 2 cents worth is this. The small change in energy your talking about at 40 yards is very small. I think out to 40 yards it dosn't matter if you shoot 4's, 5's or 6's. I do think a lot of people feel better seeing more shot in the head area than if they had less shot. I have used all of them & go back & forth between 5's & 6's, I will say I mostly use #6's now and have never had any problems with them nor did I ever have a problem with #5's. I think it is more important to shoot the shot that patterns best with your gun & choke.