I'd be asking myself, what did I do wrong to drop that arrow at 20. AND, I do that sometimes. If I go out and shoot a half dozen groups, I'd want them all to be on at 20. Not having shot since 1993 I'd say you're still working on muscles and mechanics. Depending on your bow, a little too much hand or torque or dropping your arm or canting the bow can cause these fliers. The 40 yard group tells me you best not be shooting at deer at 40 yards. Your occasional flier could also be a bad arrow. Mark them and see if it's the same arrow all the time. See the spot and don't fight the pins. Work on a smooth release. Don't be punching that release, get the finger tip off the releasee. Relax. Bowhand is the tough thing for lots of folks to get down. I'd also be investigating why my 20 yard group seems to be leaning toward the left side of the bullseye and my 40 is left.