Brass brushes are a hell of a lot softer than steel,
what do you think wears ?????

.423 - .458 isn't that big a jump (0.035 only), considering
that the brushes are flexible.

Also, I often use plastic brushes when over size
just to limit any POSSIBLE wear and tear.

I would say a bullet, especially a Steel Jacketed Solid puts a lot more wear on a bore than a soft copper brush with plenty of lubricant (bore cleaner) on it.