I think it's a mistake to assume that shooters who weigh less have more difficulty with recoil than their more hefty collegues. My friend John Buhmiller probably weighed 150 pounds soaking wet, but he killed more elephant and buffalo than any other American I am aware of, using a .460 Weatherby necked up to .50 caliber, which significantly improved on the performance of the .500 NE.

My theory is that smaller shooters don't have the amount of inertia created by the greater body mass of heavier shooters, which means that they tend to recoil with the rifle, instead of resisting the force of the recoil for an instant before yielding to it.

Shooting my .577 has taught me to do a little dance, right foot, left foot, right foot, in response to the push of the 750 grain bullet at 2050 fps on the 12 pound rifle and my shoulder. It's nothing like the jab of my 8 3/4 pound .505, although I have never noticed the .505's recoil when shooting at game.

In recoil the factor to be concerned with is recoil velocity of the rifle. My .577 and my .505 have about the same amount of recoil, reflected in foot pounds. Where they differ is in recoil velocity.
20 fps is, in my opinion, the limit for bearable recoil.