Reading Adrian Edmondson’s excellent autobiography, he mentioned Sturgeon’s law which is: “Ninety percent of everything is crap”. Adrian is a comedian and was applying the law to his creative work. Sturgeon was using it talk about science fiction, but I think it also applies to scientific research, and Sturgeon’s number is strikingly similar to the estimate from Chalmers and Glasziou that 87.5% of health and medical research is wasted (which they rounded down to 85%).
Layers of crap
Labeling 90% of research as “crap” is hyperbolic and unfair as there are layers to the crap.
At the pit is the research that is fraudulent.
The cream of the crap includes studies that did not go to plan, but where the researchers still learnt something, even if that was only how to do the next study better. This research still has value, even though it failed to answer its target question.
In the middle there’s a lot of stuff where the researchers made an avoidable error, including tackling an already answered question, a simple mistake in their design or analysis, or simply failing to publish their work.
The 10% is worth it
The Wikipedia page on Sturgeon’s law has some other interesting quotes, including on books from Benjamin Disraeli:
“Nine-tenths of existing books are nonsense, and the clever books are the refutation of that nonsense.”
And a more general observation from Rudyard Kipling:
“Four-fifths of everybody’s work must be bad. But the remnant is worth the trouble for its own sake.”
In both these quotes the good makes up for the bad, and this is the same in health and medical research. The truly wonderful and remarkable stuff, such as amazing vaccines, and life-saving technologies, is what makes health and medical research amazing.
Is 90% of my own work crap?
Am I brave enough to label 90% of my own career as crap? Perhaps I’ll go for Kipling’s figure of 80%, but I am honest enough to recognise that most of my published research is humdrum.
I tell young researchers that they will have a remarkable career if they finish with two or three real breakthroughs.
Not every joke that Adrian Edmondson wrote was funny and not every study that I’ve worked on has had impact, but we’ve both tried to make the world a better place.