Replication, preregistration, and open science – what’s all the fuss about?

The so-called “replication crisis” engulfed psychology over the last 10 years, with numerous failures to reproduce canonical studies from the biggest names in the discipline like Dweck’s growth mindset, Baumeister’s willpower as a muscle, and around half of Kahneman’s Thinking Fast and Slow. Interrogation of this failure of replicate led to discoveries of p-hacking, publication bias, a huge disconnect between the theories psychologists were supposedly testing and the cute little studies they were using for that purpose. Eventually there was even evidence of outright fraud, notably in the case of Harvard’s Francesca Gino, Duke’s Dan Arielly, and others. Hearing all this in the news, you might wonder: why is replication so crucial to the progress of science? Is there anything we can do improve the credibility of scientific practice? Does preregistering our intended analysis and expected results have an effect? Here to answer all these questions and more is Patrick Vu, Assistant Professor of Economics at the University of New South Wales in my hometown of Sydney, Australia. Fresh off a PhD from Brown investigating the statistical side of replication and publication bias, Patrick is at the bleeding edge of these issues.

Patrick’s website:

https://www.patrickhvu.com/

Study where multiple research teams use the same data to test the same hypothesis:

https://eprints.lse.ac.uk/117278/1/pnas.2203150119.pdf

The paper that launched the replication crisis – Why most published research findings are false:

https://pmc.ncbi.nlm.nih.gov/articles/PMC1182327/

Karl Popper’s The Logic of Scientific Discovery:

https://philotextes.info/spip/IMG/pdf/popper-logic-scientific-discovery.pdf

My favourite paper from Andrew Gelman:

https://sites.stat.columbia.edu/gelman/research/published/gelman_hennig_full_discussion.pdf

Open Science Collaboration - Estimating the reproducibility of psychological science:

https://www.science.org/doi/10.1126/science.aac4716