My 1903, while not a take down, suffered from similar inconsistent accuracy. From the description of your bore, mine sounds about the same. If you have checked these items please disregard the post but, if not, they might be worth taking a look at.

What I discovered on mine was 1), the wood around the less than robust recoil lug was soft, evidently just from age and perhaps an over-exuberant application of oil. 2), the screw for the front sling stirrup was putting a bit of tension on the barrel as the barrel heated.

Remedy was to glass bed the action in the typical manner and drill the hole in the barrel band a size or two larger to allow for expansion. The mortise in the stock for the front sling stirrup lug also showed evidence of the lug rubbing on the bottom as well as back face of the mortise. Those areas were relieved and the rifle now shoots consistent groups hovering slightly above or below 2 inches, (mostly above with my old eyes), benched and bagged with the original open sights.

I chased the demons in that rifle for well over a year before I finally got them all exorcised!!! Now watch something else pop up.