Ok but if you let the cases fall from the chambers of a non ejector gun in rocky country they are just as likely to hit a rock and make unwanted noise as shells thrown from an ejector gun.
Imo a quality gun will have quality ejectors and therefore they should be just as reliable as the other working parts of the gun.
The stiffness when closing an ejector gun will only slow the closing of the gun if the wrong technique is used.
If you use the same technique with an ejector gun as with a non ejector gun, yes, it most likely will slow you down.

Not all fast reloads come about as a result of a charging animal.
Fast reloading is often needed in hunting circumstances which do not involve any charge situation but still require more than 2 shots to get the animal down and dead.