Sure it would be be alot of work, but after having some quality barrels, on some of my rifles, its hard for me to go back to factory barrels. They are smooth as silk. And as long as you get a good gunsmith to do your throat and crown for you and set your headspace, you are almost guaranteed a shooter.

Getting into custom built guns should never be a investment. You will never get your money out of them you put in. It all depends on what you want. I mean all you need is 2MOA accuracy or even less for deer/elk but most of us (including you) demands more. And we are not as lucky as you getting a sub .5MOA shooter.