Archives


AoI*: “The Robustness Reproducibility of the American Economic Review” by Campbell et al. (2024)

[*AoI = “Articles of Interest” is a feature of TRN where we report abstracts of recent research related to replication and research integrity.] ABSTRACT (taken from the article) “We estimate the robustness reproducibility of key results from 17 non-experimental AER…

Read More

AoI*: “Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias? Evidence from 15,992 Test Statistics and Suggestions for Improvement” by Brodeur  et al. (2023)

[*AoI = “Articles of Interest” is a feature of TRN where we report excerpts of recent research related to replication and research integrity.] EXCERPTS (taken from the article) “Pre-registration is regarded as an important contributor to research credibility. We investigate this…

Read More

AoI*: “Conventional Wisdom, Meta-Analysis, and Research Revision in Economics” by Gechert  et al. (2023)

[*AoI = “Articles of Interest” is a feature of TRN where we report excerpts of recent research related to replication and research integrity.] EXCERPT (taken from the article) “The purpose of this study is to compare the findings of influential…

Read More

REED: Meta-Analysis and Univariate Regression Tests for Publication Bias – Seriously?

[This blog first appeared at the MAER-Net Blog under the title “Univariate Regression Tests for Publication Bias: Why Do We Do Them?”, see here] The FAT-PET Framework: A standard meta-analysis article goes something like this (see, for example, Knaisch and…

Read More

Have Registered Reports Uncovered Massive Publication Bias? Evidence from Psychology

[Excerpts taken from the preprint, “An excess of positive results: Comparing the standard Psychology literature with Registered Reports” by Anne Scheel, Mitchell Schijen, and Daniël Lakens, posted at PsyArXiv] “Registered Reports (RRs) are a new publication format…Before collecting data, authors…

Read More

What Do Replications Tell Us About the Reliability of Meta-Analyses? Evidence from Psychology

[Excerpts taken from the article “Comparing meta-analyses and preregistered multiple-laboratory replication projects” by Amanda Kvarven, Eirik Strømland, and Magnus Johannesson, published in Nature Human Behaviour] “In the past 30 years, the number of meta-analyses published across scientific fields has been…

Read More

Bad News and Good News for Meta-Analyses in Economics

[Excerpts taken from the working paper “Practical Significance, Meta-Analysis and the Credibility of Economics” by Tom Stanley and Chris Doucouliagos, posted at SSRN] “…we find that large biases and high rates of false positives will often be found by conventional meta-analysis methods….

Read More

Pre-Registration: It’s a Journey

[Excerpts taken from the preprint “Preregistration Is Hard, And Worthwhile” by Brian Nosek and others, posted at PsyArXiv Preprints] “Preregistration of studies serves at least three aims for improving the credibility and reproducibility of research findings.” “First, preregistration of analysis…

Read More

Meta-Research: The Big Picture

[From the paper “Calibrating the scientific ecosystem through meta-research” by Tom Hardwicke, Stylianos Serghiou, Perrine Janiaud, Valentin Danchev, Sophia Crüwell, Steven Goodman, and John Ioannidis, forthcoming in Annual Review of Statistics and Its Application] “Meta-research has been defined as ‘the…

Read More

GOODMAN & REED: A Friendly Debate about Pre-Registration

Background: Nat Goodman is generally pessimistic about the benefits of pre-registration. Bob Reed is generally optimistic about pre-registration. What follows is a back-and-forth dialogue about what each likes and dislikes about pre-registration. [GOODMAN, Opening Statement] We need to remember that…

Read More