I'm sure the screw driver is part of the problem, but the real problem is the soft steel of the nut.

I have take enough of these apart to know that some clamp nuts are soft and deform. Some are hard and do not deform. On the rifles I worked on that wouldn't hold zero, these soft clamp nuts showed up.