Well, here is my opinion. I have both types of doubles.
If I was in a reloading contest I think I can reload an ejector double a little faster. However in the field, when not under duress I spend more time with an ejector gun trying to catch my brass.
With an extractor gun it is an easier, faster reload, with an extractor gun to open the gun, pull out the brass and place it in your pocket, then reload the rifle. With an ejector gun you have to be sure you hand is positioned to "catch" the brass as it is ejected out of the rifle.
With an ejector gun IF you are under duress, you simply open the gun, flip the chambers toward the ground, OR give the gun a quick backward jerk, OR simply pluck out the brass dropping it at you feet, [which ever method you like] then reload the gun.

When I was in Zim I fired 17 rounds at game with my 450 No2[extractor].
I only dropped 3 rounds on the ground, the rest of the time I felt I had the necessary time to put the empties in my pocket.

With my 9,3x74R double [ejector] I "caught" all the brass and put it in my pocket except for a round or 2 when shooting my kudu at a little over 300 yards.

Bottom line for ME is, I have both types of doubles. It does not make ANY difference to ME whether a double is extractor OR ejector. I can change between the two types with out problem.

I have read where elephant hunters/poachers prefered an extractor gun because they are silent when opened and thus can be reloaded silently.

I say use what you like best.