My first thought - will a bullet drop freely into a fired case? If it won't fit easily, then tight necks are the problem, likely (brass is too thick) with THAT brass.

Hornady Handbook printed in 1967 shows with 160gr. RN in the 6.5x54mm 18" bl.:
Moderate load 35.6gr. IMR4350 - 2,000fps, with max at 39.0gr. 2,200fps
Moderate load 37.5gr. H4831 --- 2,000fps, with max at 40.4gr. 2,200fps.

H4350 and IMR4350 might or might not be interchangeable in that ctg. with that bullet in that rifle.

While N160 might be close in burn rate to IMR4350, that does not mean it is a grain for grain replacement for IMR4350. Now, IMR4350 has changed a bit over the years as well. It produces more pressure and energy per grain than it used to produce. As well, different lot numbers produce different results. Energy per grain is most important when comparing 2 different powders.

It is possible that Viht powders or some Viht powders produce higher energy per grain than other slow burning powders. How that effects pressure is perhaps the stickler.

Blowing primers at 35.0gr. N160 sounds very suspicious to me. I am sure something else is going on.

Without chronographing the loads, the field of possibilities is very small indeed.
I would check the cases first. Long loading, impinging/touching the lands is also a pressure booster. Put a couple of these possibilities together and pressure spikes result.

First step is to check a bullet in the neck of a fired case. Does it fit easily?

If easy fit, load a smoked or felt-pen marked bullet into an empty sized case at your current oal. with primer and chamber it. Then extract it and look for land marks. If so, you need to see the bullet deeper. I suggest at least .020" jump.

Now, chamber that more deeply seated(if necessary)empty bulleted ctg. and fire the primer. Does it back out? If so, that is the amount of headspace with THAT brass.

After this, what did you find out?