The Replication Network
Furthering the Practice of Replication in Economics
Archives
Doing More with Confidence Intervals
[Excerpts taken from the article “In Praise of Confidence Intervals” by David Romer, posted at the American Economic Association’s 2020 annual conference website] “…most modern empirical papers concentrate on two characteristics of their findings: whether the point estimates are statistically…
Read MoreIt’s Not A Problem, It’s an Opportunity
[From the blog “The replication crisis is good for science” by Eric Loken, published at The Conversation] “Science is in the midst of a crisis: A surprising fraction of published studies fail to replicate when the procedures are repeated.” “Is…
Read MoreDo Not Abandon Statistical Significance
[From the article “The Importance of Predefined Rules and Prespecified Statistical Analyses: Do Not Abandon Significance” by John Ioannidis, published in JAMA] “A recent proposal to ban statistical significance gained campaign-level momentum in a commentary with 854 recruited signatories. The…
Read MoreSpecial Issue of The American Statistician: “Statistical Inference in the 21st Century: A World Beyond p < 0.05”
[From the introductory editorial “Moving to a World Beyond ‘p < 0.05’” by Ronald Wasserstein, Allen Schirm and Nicole Lazar, published in The American Statistician] “Some of you exploring this special issue of The American Statistician might be wondering if…
Read MoreFailure of Justice: p-Values and the Courts
[From the abstract of the working paper, “US Courts of Appeal cases frequently misinterpret p-values and statistical significance: An empirical study”, by Adrian Barnett and Steve Goodman, posted at Open Science Framework] “We examine how p-values and statistical significance have been interpreted…
Read MoreVASISHTH: The Statistical Significance Filter Leads To Overoptimistic Expectations of Replicability
[This blog draws on the article “The statistical significance filter leads to overoptimistic expectations of replicability”, authored by Shravan Vasishth, Daniela Mertzen, Lena A. Jäger, and Andrew Gelman, published in the Journal of Memory and Language, 103, 151-175, 2018. An open…
Read MoreSignificant Effects From Low-Powered Studies Will Be Overestimates
[From the article, “The statistical significance filter leads to overoptimistic expectations of replicability” by Shravan Vasishth, Daniela Mertzen, Lena Jäger, and Andrew Gelman, published in the Journal of Memory and Language] Highlights: “When low-powered studies show significant effects, these will…
Read MoreBLANCO-PEREZ & BRODEUR: Progress in Publishing Negative Results?
[From the working paper, “Publication Bias and Editorial Statement on Negative Findings” by Cristina Blanco-Perez and Abel Brodeur] Prior research points out that there is a selection bias in favor of positive results by editors and referees. In other words,…
Read MoreMore Evidence on How Control Variables Affect Statistical Significance
[From the working paper “Achieving Statistical Significance with Covariates and without Transparency” by Gabriel Lenz and Alexander Sanz] “An important yet understudied area of researcher discretion is the use of covariates in statistical models. Researchers choose which covariates to include in…
Read More
