Article

Retire Statistical Significance

Valentin Amrhein, Sander Greenland, Blake McShane and more than 800 signatories call for an end to hyped claims and the dismissal of possibly crucial effects.

Valentin Amrhein, Sander Greenland and Blake McShane

Nature, 2019

References: Mc Shane et al. (2019) , Gelman & Loken (2019) , Amrhein et al. (2019)

Read the article

Read offline

1×

Editorial Rating

Qualities

Analytical
Scientific
Hot Topic

Recommendation

Researchers commonly refer to the differences between sets of data – measures from different treatment groups, for example – as being (or not being) “statistically significant.” The sets are considered significantly different only if an arbitrary statistical threshold is exceeded. If it isn’t, people often conclude that there is no difference between the sets of data. This assumption, however, is not necessarily valid. A sub-threshold finding may still be relevant. And, as Carl Sagan put it, absence of evidence is not evidence of absence. A group of over 800 scientists is now calling to “retire statistical significance.” The article will engage and inform anyone concerned that statistical analyses can be misleading but were never quite sure exactly how or why.

Summary

Drawing a hard line between “statistically significant” and “statistically non-significant” results leads to invalid claims about scientific findings.

It is common practice for researchers to use statistical tests to interpret the findings of research studies. In statistical hypothesis testing, “significance” – or P value – refers to the probability that a given hypothesis is true.

If a certain statistical threshold isn’t exceeded, differences between two sets of data – measures from different treatment groups, for example – are often disregarded as “statistically non-significant,” i.e. interpreted as not relevant or even nonexistent...

About the Authors

Valentin Amrhein is a professor of zoology at the University of Basel, Switzerland. Sander Greenland is a professor of epidemiology and statistics at the University of California, Los Angeles. Blake McShane is a statistical methodologist and professor of marketing at Northwestern University in Evanston, Illinois.

Comment on this summary

Sign in to share your opinion
R. E. 4 years ago

And replace with what ? If these 'scientists' are proposing talking about the data sets then we are going to have this infuriating descent in post-modern 'narratives' where facts are replaced by opinions and irrelevant variables are examined so as not to be excluded. Just because the right-wing populism of our age decries expert reasoning, it does not mean that science should yield a single millimeter to the epistemological hooligans

On the other hand, the use of statistics in finance and market modelling has led to 'bad science'. The 'mathsness' perpetuates the fallacies of atomised self-interested individuals, maximising their own utility at the sake of others - and 'magically' out of this chaos of egoism all arrive at a utoptian and mathematically derived equilibrium.

Maybe if stats were not something that were lorded over people, then they would be respected and understood.