Bed job required. Likely requires to have some of the stock hawged out after to ensure it isn't getting any pressure points upon heating up. I am big fan of pillar blocks when bedding just makes for better mating.
The trigger doesn't have to break at 2 lbs in fact I would argue for hunting purposes that is a bit light. My big game hunting rifles all break clean at 2.75 to 3 lbs. The secret to a good trigger isn't where it breaks but rather the no travel or creep...it should be a wall!
When results are repeatable I thing your aok in regards to optics, mounts, rest and even shooter. Though the mind can creep into the later...lol.
Ammo...JMO but your wasting money if you can't get anything to group at this point. Its a barrel/stock touch issue once she gets heated up it strings. I bet if you shot 2 groups let the rifle return to ambient and fired then next 2 shot group you'd find better results on target. Not saying thats the fix...bc it isn't but until your ready to tackle the bedding issue thats what I would do for the time being. Get your mind focused on that cold barrel accuracy, pick the right load and dial it in. The first shot is always your best when it comes to hunting... Best of luck