I talked to a guy years ago who was into black powder cannons. He told me that there are two "sweet spots" in theory, but only one that is most effecient, the other usually cannot be obtained safely (even with a cannon). He said the most accurate charge with the least amount of powder is what matters, however, if you keep loading in powder, eventually you start to blow unburned powder out of the muzzle. Eventually, you will blow so much unburned powder out of the muzzle that the projectile will be forced out at the same velocity as it did with the first (sweet spot) load. Because even though you are loading in more powder, the powder that isn't burned begins to pile up and slow the projectile down. Kind of like the law of diminishing returns.

This was one guy's theory, so if it's true, it probably wouldn't matter if you tried 120+ grains of powder or not, if you get your best accuracy with 100 grains of powder, that's the most effecient load for your gun and that bullet.