Unless, of course, you notice that your POI starts to shift dramatically after more shots, which may be due to fouling.

I don't clean my rifles, but I do scrub my rifled-barrel shotguns, because the fouling tends to stay in the grooves and effects slug performance, IMO...