Yes, you lose some velocity with a shorter barrel. This varies from caliber to caliber, but can be quite noticeable between an 18" barrel and a 24" barrel.

A shorter barrel will also subject you to more muzzle blast, and perhaps a bit more felt recoil, since it is usually lighter than a longer barrel.

Accuracy is not so much a problem, since you can usually develop a handload and make bedding adjustments to obtain satisfactory accuracy. A lightweight barrel is probably not going to be quite as accurate as a heavier barrel due to increased effects of heat warping the barrel, reduced stiffness, etc. In a mountain rifle accuracy vs weight is one of those trade-offs you make.