Will it group with the first handful of shots and then stray?

Don't forget, if you shoot too many shots to quickly your barrel will get hot and your shots will start to stray. My Tikka T3 Lite is good for 4 or 5 shots in approx 1" @ 100 yards.

first 2 shots were off the paper, then a large group then nothing and so on.

Try the Federal Premium 85gr HPBT, I've killed about 35 deer with it, excellent bullet...

You don't have to give up on heavier bullets though...I've tested about 12 factory loadings in my .243...Some bullets just don't shoot well in some rifles...For some reason my rifle loves the 100gr Remington CoreLokts, the Federal Premium 85gr, the Federal Premium 100gr Nosler Partition and the Federal 100gr Hotcore...It doesn't like Hornadys, in the Lite Mag or standard loading, nor does it care for the Federal Premium 100gr Sierra GameKing SPBT or the Winchester Supremes 95gr Combined Technology...Some of these grouped up to 4 inches, whereas theCoreLokts will hit a quarter at 100 yards...

Keep trying until you find what your rifle prefers...