you can take your nipple out and use a little sand paper to take it down slightly in size. I honestly thing the nipple has a little flare at the tip and the first hammer blow just seats it.

Is the face of the hammer clean?