The Replication Network
Furthering the Practice of Replication in Economics
Archives
To Your List of Biases in Meta-Analyses, Add This One: Accumulation Bias
Posted on by replicationnetwork
[From the preprint “Accumulation bias in meta-analysis: the need to consider time in error control” by Judith ter Schure and Peter Grünwald, posted at arXiv.org] “Studies accumulate over time and meta-analyses are mainly retrospective. These two characteristics introduce dependencies between…
Read MoreWhy So Many Insignificant Results in a Meta-analysis?
Posted on by replicationnetwork
[From the blog “Where Do Non-Significant Results in Meta-Analysis Come From?” by Ulrich Schimmack, posted at Replicability-Index] “It is well known that focal hypothesis tests in psychology journals nearly always reject the null-hypothesis … However, meta-analyses often contain a fairly…
Read MoreWhen Once Is Not Enough
Posted on by replicationnetwork
[From the blog “Be careful what you wish for: cautionary tales on using single studies to inform policymaking” by Emmanuel Jimenez, posted at http://www.3ieimpact.org.%5D “For a development evaluator, the holy grail is to have evidence from one’s study be taken…
Read MoreCategory: NEWS & EVENTS Tags: 3ie, 3ie replication programme, Dengvaxia, Meta-analysis, Philippines, replication, Systematic reviews
Making Meta-Analyses More Replicable
Posted on by replicationnetwork
[From the article “Automatic extraction of quantitative data from ClinicalTrials.gov to conduct meta-analyses” by Richeek Pradhan et al., published in the Journal of Clinical Epidemiology] “Systematic reviews and meta-analyses are labor-intensive and time-consuming. Automated extraction of quantitative data from primary…
Read MoreCategory: NEWS & EVENTS Tags: Automatic extraction, ClinicalTrials.gov, EXACT, Meta-analysis, Python, replication, Systematic reviews
What Can Meta-Analyses Tell Us About Reproducibility?
Posted on by replicationnetwork
[From the abstract of the article “What Meta-Analyses Reveal About the Replicability of Psychological Research” by T.D. Stanley, Evan Carter, and Hristos Doucouliagos, published in Psychological Bulletin] “Can recent failures to replicate psychological research be explained by typical magnitudes of statistical power,bias…
Read MoreCategory: NEWS & EVENTS Tags: Chris Doucouliagos, Heterogeneity, Meta-analysis, Power, Psychological Bulletin, Psychology, replication, Tom Stanley
Are Meta-Analyses Overrated?
Posted on by replicationnetwork
[From the article, “Meta-analyses were supposed to end scientific debates. Often, they only cause more controversy” by Jop de Vrieze, published at http://www.sciencemag.org%5D “Meta-analyses were thought to be debate enders, but now we know they rarely are,” Ferguson says. “They should…
Read MoreCategory: NEWS & EVENTS Tags: Cochrane, Media violence, Meta-analysis, Science journal, Video games
IN THE NEWS: Wired (November 14, 2017)
Posted on by replicationnetwork
[From the article “The Dismal Science Remains Dismal, Say Scientists” by Adam Rogers at wired.com] “WHEN HRISTOS DOUCOULIAGOS was a young economist in the mid-1990s, he got interested in all the ways economics was wrong about itself—bias, underpowered research, statistical shenanigans. Nobody wanted…
Read MoreCategory: NEWS & EVENTS Tags: COLIN CAMERER, Hristos Doucouliagos, John Ioannidis, Meta-analysis, Reproducibility, Underpowered
FYI: ScienceOpen Has a Collection of Papers on How to Fix the Replicability Crisis
Posted on by replicationnetwork
ScienceOpen has a collection entitled: “Remedies to the Reproducibility Crisis”. The collection is introduced thusly: “Psychology, Medicine, Neuroscience and many other research fields, are facing a serious reproducibility crisis, that is, most of the findings published in peer-review journals, independently…
Read MoreConcurrent Replication
Posted on by replicationnetwork
[From Rolf Zwaan’s blog “Zeitgeist”.] “A form of replication that has received not much attention yet is what I will call concurrent replication. The basic idea is this. A research group formulates a hypothesis that they want to test. At…
Read MoreCategory: NEWS & EVENTS Tags: Concurrent replication, Meta-analysis, Pre-registration, replication, Rolf Zwaan
Another Session on Reproducibility at the 2017 ASSA Meetings
Posted on by replicationnetwork
TRN previously posted about two sessions on replication at the 2017 ASSA meetings. In what may be a sign of changing times, the American Economic Association is sponsoring a third session on this topic. The session, “Meta-Analysis and Reproducibility in…
Read MoreCategory: Uncategorised Tags: BITSS, Chris Doucouliagos, John Ioannidis, Meta-analysis, Reproducibility, SSMART grants, Tom Stanley
Tags
Andrew Gelman BITSS Brian Nosek Data sharing economics Journal policies Meta-analysis null hypothesis significance testing Open Science p-hacking p-values Pre-registration Psychology publication bias Registered Reports replication replication crisis replications Reproducibility TransparencyBlogroll
Recent Posts
- REED: You Can Calculate Power Retrospectively — Just Don’t Use Observed Power
- ROODMAN: Appeal to Me – First Trial of a “Replication Opinion”
- AoI*: “Introducing Synchronous Robustness Reports” by Bartos et al. (2025)
- AoI*: “The Sources of Researcher Variation in Economics” by Huntington-Klein et al. (2025)
- AoI*: “Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology” by Gould et al. (2025)
The Replication Network 
You must be logged in to post a comment.