I think some had primitive scopes - I saw a special on the history channel that describes it. Very interesting stuff. Here is the link for the history channel special posted on youtube:

https://www.youtube.com/watch?v=-XJmWXsZKNk

The Civil War sharpshooter bit starts around the 11 minute mark.

Still, shooting 5-10" groups at 200 yards with open sights or a primitive scope is pretty impressive in my book.