Fouled barrel during hunting season if longer range shots are expected. Clean barrel the rest of the year.
For deer hunting at closer ranges it will not matter if the barrel is fouled or clean. In that situation I keep them clean.
Barrels that are not properly cleaned and oiled for storage often show the effects eventually with rust and pitting.