Sounds like you're losing your focus without a hard aiming point.

At 25 yards, you're trying to split the difference between your 20 and 30 pins, with neither pin actually anchored on a point of aim. (Assuming you have 20 and 30 yard pins.) Rather than hold the spot floating between the pins, go ahead and split them, check where the 20 pin is, then hold the pin on that exact spot.

It's called 'reading the gap.' Here's how I do it. In my mind, I visualize a scale between my pins, in 1 yard increments, with 10 increments between my 20 and 30 pins. So, for 23 yards, I hold my 20 pin 3/10ths high, mark where the pin sits and aim at that exact spot. 27 yards, I aim the 20 pin 7/10ths high and mark the spot. If the 20 pin is completely off the target, then I switch to holding the 30 pin 3/10ths low, mark that spot and aim at it. Of course, 25 yards is right in the middle.

You have to aim at a definite spot.