I've seen too many shooters, including my own grandfather, try to anticipate the shot and jerk the trigger or flinch. But there are just too many variables here without seeing the rifle or the shooter. Like Hardcast stated it could be so many things or a combination of a few causing this erratic shooting. If everything else checks out then I would take the rifle to a gunsmith and have him bed the action.