My experience -- Drawbacks are:
Loading is more complex than a cartridge rifle or shotgun.
Not as consitent grouping as smokless due to slight variances in powder burning and bullet seating.
Top limit on range is, for me , 150-165 yards(In line) ... 75 yards (Round Ball-Cap Lock)
Relatively easy to experience hang fire due to contamination of either or both the primer and powder, especially in wet weathwer.
Usually one shot and only one shot.
Smoke can often obscure whcih where the critter went after being hit.

But the fun is in the challenge. Especially when I use my Lyman "Plainsman".