This is true. However in your post you stated that you used hot soapy water to clean. Although this is fine and will clean your rifle, with the new substitute powders like the White Hots you're using, you can use the modern black powder substitute cleaning solvents. T/C's T-17 is one good example.
But you should remove and clean and lubricate (protective lube) any part which comes in contact with burnt powder residue. IF you don't you will have a problem later on.