Boy, if it were me, I'd shoot whatever factory ammo gave me the best group. When you say remington factory ammo are you just talking about corelokts? Or have you tried the premium grades? For example, the Remington Safari Grade has the Swift A Frame bullet which I believe will be similar in performance, in terms of the bullet, to what you're shooting. There is a Swift Scirrocco too which is a premium bullet. I'm only familiar with Remington because they seem to shoot best in my gun, and I don't know what is available for your caliber or what you're lookign for in a bullet.

But that still doesn't address the huge discrepancy between your first shot and second. That isn't a group size issue, imho.