You really shouldn't be pulling into the wall hard enough to move your peg. A good way to really scratch up that cam and be inconsistent is to have that draw stop adjusted too short. I'd try moving it a bit before adding another peg, personally.