That rule applies to any weapon when you are adjusting the front sight.
If you ever adjust the rear sight, like on a rifle with iron sights, it's the opposite.

I remember it by the fact that I always follow my arrows downrange when I shoot them (to get them back), but not when I shootbullets.