Josh-

This is a typical problem with 100+yr old BPE double rifles shooting NFB loads.
My Tolley 500-450 3-1/2" Magnum Express (NFB load = 350gr SNRN 2025fpsMV) does the same thing.
I've seen and heard of numerous other BPE double rifles with this issue.
What do you have for a rear sight? My Tolley has a single express sight, flanked by the "Charge 5-1/2 Drams" prescription engraved on the top of the chambers -- true express, Point Blank to 175 yards!
The standard course of action is to either live with it as it is or have a new front sight made.
The cost of a front sight isn't peanuts compared to the cost/value of an AsNew H&H Royal (of any configuration), and it doesn't detract from the value of the rifle in any way.

There's another thing that can contribute to this -- stock fit. The angle of the buttplate and the cut of the stock can affect vertical POI.

Something else you should try is Black Powder.
Get a pound of Swiss 1-1/2F and some hard cast bullets of appropriate weight and shoot the load it was built for.
See how it prints with coal and lead.

--Tinker