Nitro - sounds to me as if you should be using plastic or paper cases for a better bullet fit. I know brass is nice to use, but that's a lot of excess space on the sides.
With a wad over the slug, I would be concerned about accuracy - not an obstruction if it is in contact.

What about cleaning the brass as well as you can then spin cleaned steel wool inside the cases. Tin the inside of the brass cases, then solder tinned & coiled .010" to .012" brass sheet inside to increase the thickness of the case (full length), in a similar manner that paper tubes were used to reduce the capacity of some military BP brass. That would be a permanent 'fix' to the fit problem.