If you are adjusting a scope's reticle or the rear sight of any weapon, move the sight in the direction you want the point of impact (POI) to move. So, if the bullets are hitting high and left, you move the reticle down and right.

If you are adjusting the front sight, like on a bow, then it is opposite, and you have to move the sight in the direction the arrows are going. I remember it by 'chasing the arrows" with my front sight. So, if your arrow are hitting high and left, you move your front sight up and left.