Philnie - I've always pulled the barrel from the stock as well. It allows me to wipe the fowling off the edges of the stock along the barrel, clean the trigger assembly, and makes for easier cleaning. For instance, do not pull that trigger and put in the breech plug and fianally the end cap locking nut. Then, take the trigger assembly off (one screw holds the bold trigger in place) and put the breech plug in, etc... much nicer. Plus I have seen fowling along the top of the trigger assembly on the under side of the barrel along the hammer area.. If you leave it in the stock, you miss that.

So far, I have not seen a decrease in any accuracy. Now this would be different if the rifle barrel was pinned to the stock like in some of the old flintlock rifles, but it's not. If the barrel is made to come out.. it comes out.