Shoot from 20, 30, and 40 yards (even further if you want) to check for slight adjustments that needs to be made. If your arrows drift left or right, your centershot isn't just quite perfect.

Target panic is what you're describing. It's a mental thing, and confidence in your abilities and equipment will help that. It's an extremely difficult condition to overcome, but blind bale shooting a TON will help.