Copper (guilding metal from most bullets) builds up very quickly in some barrels due to roughness in their interiour finishes. The roughness can be caused by various problems, merely a rough rifling job with minor tearing, or pitting in older rifle barrels.

As others have noted, metal fouling merely continues to build up - and up, and up, making the bore quite inconsistent one end to the other as it is not even. Tight spots get tighter, while loose spots might not foul at all, thus making even larger differences and inconsistencies within the tube.

If the rifle regulates with clean bores, you'll probably want to keep them spotlessly clean.

Some products, as John says clean only one of the foulers, powder or copper, thus need to be alternated to get it all.
Some products, like an overnight soak with Wipe-Out foaming bore cleaner, do it all - but, I always use Kroil after drying the bore in the morning. Many cleaners do not leave any oil in the pours of the steel, thus the bore is quick to foul (fouls excessively) with the first shot through a clean dry bore.

Some shooters use EEZOX as an after-prep and bore preservative. It is one of the best, if not the best bore preservative - even better than RIG, which is second best, but is a storage grease more than anything.

I'm picking up some Lock-Eze today from Aukland's who ordered it in for me. This is an after-clean prep as well, that is actually a rust preventative as well. Many of the small bore shooters use it so I am going to try this 'stuff' too.

So - bottom line is I would clean the bores if they copper foul excessively. I would be very careful of stroking the bores with a rod, as Doug noted - they are quite easily damaged with a rod - which is why I really like foaming bore cleaners like Wipe-Out- fill the tube, let it sit overnight and push out the fouling with one patch - clean! If that doesn't work, it's REALLY a bad fouling - do it again. There is NO multiple stroking of the bore to wear the copper out (or the throat's oval) with a bronze brush on a rod - let the chemicals do the work and just a couple drying patches, then a 'light oil' patch and you're done done.
1-piece rods only.