does your first shot go where you want it to?... if so don't worry about it... it's probably a bedding/barrel pressureproblem that could be fixed with a little work, but you say this is a hunter, not to worry then...

heat build up no matter how little will compound a bedding/barrel pressure problem quickly...