I have to agree with falcon. I've shot thousands of Barnes bullets and never suffered from any plastic buildup from sabots. However as I mentioned, burning out the flash hole (enlarging it) has always been an accuracy killer.
Shooting the Encore platform rifles, this I recognized and that was that each BP reacted differently. Some BP's lasted longer than others for some reason. However, when the groups started to suffer, replacing the BP brought the groups right back together again. A good thing to check. Assuming there's no scope, ring or mount issues.