Welcome blackbearhunter.
The truth is that your double is crossing its shots and BOTH barrels are crossing the line of sight, not just the left one!
The reason that the right barrel is shooting close to point of aim is that the sight/s have been aligned that way.
If you draw a line between the holes in your target for shots fired from both barrels then cut the line in half, that is (approx) where the sight should be set.
If the line was say for example 4 inches between the two barrel groups, and you cut the line in half with another mark, this would show that the groups are crossing the line of sight by 2 inches each.
If you are shooting reloads you may be able to bring the groups together by reducing the powder charge and the velocity slightly and this may also raise the point of impact slightly as well.
Once the two individual barrel groups come together you can re adjust the sighting to align with the composite group.

Do yourself a big favour and buy a copy of "Shooting The British Double Rifle" by Graeme Wright as this book explains the details on the whys and howto's of getting doubles to shoot correctly.
Hope this helps.