First thing I would do is move in to 50 yards and shoot a five shot group. check and see if you have fliers there. Then hit the 75 yard line and do the same thing. Again, if it is making fliers at the close distances they should only get worse the further back you get.
Then at 100 yards shoot that same five shots. It looks like you have good accuracy. Also make sure you are swabbing the same for each shot.