Keith,
Are you sighting with the bead resting right at the bottom of the vee or can you still see some of the foresight blade in your sight picture?
The bead needs to be resting in the vee with no blade visible or else the gun is likely to shoot high.

Filing rearsight vees requires expert filing ability to keep everything correct.
You need to be able to retain the correct angle of the vee, keep the vee centered, and also retain the correct angle of the vee's top surface as this is usually relieved toward the front (muzzel) edge so that you only see the face of the rearsight itself.
This filing job can be quite tricky to say the least.
If all you have are flat files with "safe edges" the task is not easy.
If you can get hold of a file which is made to cut the correct vee angle it will make things easier but you still need to be patient, and take great care to achieve a result which appears similar to original.

If it was my rifle, and I was certain that my sight picture was correct and that the rearsight was the cause of the high shooting, I think I would be a little uneasy about going at that original Jeffery sight with a file.
I'd maybe machine a new sight from scratch but just have the one standing shallow vee regulated to say 75 yards and dont worry about the longer range folding leaves.
That way you can keep the original sight "original" and re-install it at any later time.