Archives


Pre-Analysis Plans in Economics and Political Science: Are They Working?

[Excerpts taken from the article, “Pre-analysis Plans: A Stocktaking” by George Ofosu and Daniel Posner, posted at George Ofosu’s website at the London School of Economics] “We draw a representative sample of PAPs and analyze their content to determine whether…

Read More

Pre-registration: The Naysayers Strike Back!

[Excerpts taken from the preprint, “Preregistration is redundant, at best” by Aba Szollosi et al., posted at PsyArXiv Preprints] “The key implication argued by proponents of preregistration is that it improves the diagnosticity of statistical tests…In the strong version of…

Read More

Pre-Registration: It’s a Journey

[Excerpts taken from the preprint “Preregistration Is Hard, And Worthwhile” by Brian Nosek and others, posted at PsyArXiv Preprints] “Preregistration of studies serves at least three aims for improving the credibility and reproducibility of research findings.” “First, preregistration of analysis…

Read More

Meta-Research: The Big Picture

[From the paper “Calibrating the scientific ecosystem through meta-research” by Tom Hardwicke, Stylianos Serghiou, Perrine Janiaud, Valentin Danchev, Sophia Crüwell, Steven Goodman, and John Ioannidis, forthcoming in Annual Review of Statistics and Its Application] “Meta-research has been defined as ‘the…

Read More

GOODMAN & REED: A Friendly Debate about Pre-Registration

Background: Nat Goodman is generally pessimistic about the benefits of pre-registration. Bob Reed is generally optimistic about pre-registration. What follows is a back-and-forth dialogue about what each likes and dislikes about pre-registration. [GOODMAN, Opening Statement] We need to remember that…

Read More

Another Economics Journal Pilots Pre-Results Review

[From the article “Pre-results review reaches the (economic) lab: Experimental Economics follows the Journal of Development Economics in piloting pre-results review”, an interview with Irenaeus Wolff, published at http://www.bitss.org. The following are excerpts from that interview.] “In its April 2019…

Read More

Pre-Registration As a Self-Control Mechanism

[From the blog “Registering studies when all you want is a little more credibility” by Berk Ozler, posted at World Bank Blogs] “Some time ago, colleagues from World Bank (WB) operations contacted me with a request to evaluate an upcoming…

Read More

Pre-Registration: The Doctor is Still Out

[From the article “A solution to psychology’s reproducibility problem just failed its first test” by David Adam, published at http://www.sciencemag.org] “Efforts to improve the robustness of research by asking psychologists to state their methods and goals ahead of time, a process…

Read More

A Random Sampling of the State of Transparency and Reproducibility in Social Science Journals

[From the preprint “An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017)” by Tom Hardwicke, Joshua Wallach, Mallory Kidwell, & John Ioannidis posted at MetaArXiv Preprints] “In this study, we evaluated a broad range of…

Read More

Wondering How to Write a Pre-Analysis Plan? Simulate it.

[From the blog “Better pre-analysis plans through design declaration and diagnosis” by Graeme Blair, Jasper Cooper, Alexander Coppock, and Macartan Humphreys, posted at BITSS] “Pre-analysis plans (PAPs) are used by researchers to register a set of analyses before they learn…

Read More