First I'd check to make sure your scope or rings aren't moving.

Then I'd try a different brand ammo. I know tons of people on here swear by the cheap Remington ammo, but it generally won't shoot like premium (Federal or Hornady) ammo in most of my rifles. You said you tried two different bullet weights, so I'm guessing it may not be the ammo though.

From a cold barrel you should able to shoot a three shot group before heat starts doing too much damage.

If after trying all that it still won't shoot, I'd start leaning towards you just have a rifle that won't shoot. If you shoot enough rifles, you will run into that now and then.