I have seen failures in the field in both a double and a pump action rifle. The double failure was a light primer strike due to a weak firing pin spring. The first barrel went "click" but the second went "bang" resulting in one dead deer. The pump action rifle went click or sometimes bang for the same reason. Turns out the owner would clean the rifle at the end of deer season and then leave it cocked (unloaded) in the closet until the following year. I have yet to see or to experience a bolt action rifle failure that wasnt related to bad ammo.

Getting back to the speed question through, if one were to time me with a double or a bolt, I'd be faster with a bolt for 3 or more shots, at least until I ran it dry. The simple fact at least in me case, is that I grew up shooting bolt guns and my affliction (some might call it an addictive sickness) is for doubles. I just think they're way more cool and it has little to do with speeding.