real simple, every bag of seeds sold will tell you there is a percentage of unknown seeds in it!
so some will always come from this!

BUT< more likely, when your re doing your clover plots, IF your turning over the dirt, what your doing is turning up dormant seeds that were in the soil, , now there closer to the top, get sun light they then start to grow, and MOST all weed seeds, grasses, grow faster than clover, so, they soon spread through out the plots and then take over the plot if allowed to!

what your doing wrong here, honestly, is KILLING the plot, over doing weed control
there are chemicals you can add to a plot that will kill grasses and other things, yet NOT kill clover!, you first need to find out what the invasive planting is, and then treat it by spraying with correct chemical!

NOW another possible issue here could be incorrect soil PH and not enough or correct amounts of needed minerals/fertilizer
if your soil isn't at a correct PH to GROW clover, and you don't have the right amounts nutrients in the soil for clover to strive , your also wasting your time, as weeds and lesser plants will also, strive in poorer soils, and clover will never have the ability to grow well, even with treating with chemicals to kill off competing crops!

SO< rather than just killing your plots to get clover to strive, maybe you should be treating the plots to make clover strive in them!
I have 30 yrs of doing food plots and more in part time farming for experience here!

also keep in mind, its a never ending process, as weed grass seeds will always get into a plot, as things go to seeds, and the wind carry's them off, birds eat deeds and many other things and then they transfer them into your clover plot!
your clover plot attracts critters and them same critters carry in the grass seeds at times!
nature of the beast LOL