The Replication Network
Furthering the Practice of Replication in Economics
Archives
GOODMAN: When You’re Selecting Significant Findings, You’re Selecting Inflated Estimates
Replication researchers cite inflated effect sizes as a major cause of replication failure. It turns out this is an inevitable consequence of significance testing. The reason is simple. The p-value you get from a study depends on the observed effect…
Read MoreThe Problem Isn’t Bad Incentives, It’s the Ritual Behind Them
[From the article, “Statistical Rituals: The Replication Delusion and How We Got There” by Gerd Gigerenzer, published in Advances in Methods and Practices in Psychological Science] “The “replication crisis” has been attributed to misguided external incentives gamed by researchers (the…
Read MoreChoosing the Right α: What You Need to Know
[From the article “The quest for an optimal alpha” by Jeff Miller and Rolf Ulrich, published in PLOS One] “The purpose of the present article is to show exactly what is necessary to provide a principled justification for a particular α…
Read MoreMILLER: The Statistical Fundamentals of (Non-)Replicability
“Replicability of findings is at the heart of any empirical science” (Asendorpf, Conner, De Fruyt, et al., 2013, p. 108) The idea that scientific results should be reliably demonstrable under controlled circumstances has a special status in science. In contrast…
Read MoreRedefining RSS
[From the blog “Justify Your Alpha by Decreasing Alpha Levels as a Function of the Sample Size” by Daniël Lakens, posted at The 20% Statistician] “Testing whether observed data should surprise us, under the assumption that some model of the data is…
Read MoreHow do Psychologists Interpret Nonsignificant Results? Evidence from the Journals
[From the abstract to the article, “Quantifying Support for the Null Hypothesis in Psychology: An Empirical Investigation” by Aczel et al., recently published in Advances in Methods and Practices in Psychological Science] “In the traditional statistical framework, nonsignificant results leave researchers…
Read MoreHow to Get Something from Nothing. Or, “Yes, Virginia, You Can Do Ex-post Power Analyses”.
[From the article, “The effect of the conservation reserve program on rural economies: Deriving a statistical verdict from a null finding” by Jason Brown, Dayton Lambert, and Timothy Wojan, recently published in the American Journal of Agricultural Economics] “This article suggests…
Read MoreREED: Why Lowering Alpha to 0.005 is Unlikely to Help
[This blog is based on the paper, “A Primer on the ‘Reproducibility Crisis’ and Ways to Fix It” by the author] A standard research scenario is the following: A researcher is interested in knowing whether there is a relationship between…
Read MorePARASURAMA: Why Overlapping Confidence Intervals Mean Nothing About Statistical Significance
[NOTE: This is a repost of a blog that Prasanna Parasurama published at the blogsite Towards Data Science]. “The confidence intervals of the two groups overlap, hence the difference is not statistically significant” The statement above is wrong. Overlapping confidence…
Read More
The Replication Network 
You must be logged in to post a comment.