Touching the bbl with anything is one problem. Scopes could be loose or broken, gun might not like the load, fouling can sometimes change how a gun will shoot (more of a muzzleloader thing though), operator error

, stock bedding or loose mounting screws. etc...