Archives


DON’T: Aim for significance. DO: Aim for precision

[From the recent working paper, “The statistical significance filter leads to overoptimistic expectations of replicability” by Vasishth, Mertzen, Jäger, and Gelman posted at PsyArXiv Preprints] “…when power is low, using significance to decide whether to publish a result leads to a proliferation of exaggerated…

Read More

National Academies of Science-Funded Committee on Research Reproducibility Holds First Meeting

[From the article “National Academies Launches Study of Research Reproducibility and Replicability” by Will Thomas, posted at FYI: Science Policy News from AIP (American Institute of Physics)] “On Dec. 12 and 13, the National Academies convened the first meeting of…

Read More

Another Economics Journal To Publish Replication Studies

[From the article “Replication Studies” by David McMillan, Senior Editor of the journal Cogent Economics & Finance] “Cogent Economics & Finance recognises the importance of replication studies. As an indicator of this importance, we now welcome research papers that focus…

Read More

More Evidence on How Control Variables Affect Statistical Significance

[From the working paper “Achieving Statistical Significance with Covariates and without Transparency” by Gabriel Lenz and Alexander Sanz] “An important yet understudied area of researcher discretion is the use of covariates in statistical models. Researchers choose which covariates to include in…

Read More

REED: The Replication Crisis – A Single Replication Can Make a Big Difference

[This post is based on the paper, “A Primer on the ‘Reproducibility Crisis’ and Ways to Fix It” by the author] In a previous post, I argued that lowering α from 0.05 to 0.005, as advocated by Benjamin et al….

Read More

MURPHY: Quantifying the Role of Research Misconduct in the Failure to Replicate

[NOTE: This blog is based on the article “HARKing: How Badly Can Cherry-Picking and Question Trolling Produce Bias in Published Results?” by Kevin Murphy and Herman Aguinis, recently published in the Journal of Business and Psychology.] The track record for…

Read More

Not Everybody Thinks the Emphasis on Reproducibility is a Good Thing

[From the article “Reproducible research: a minority opinion” by Chris Drummond, published in the Journal of Experimental & Theoretical Artificial Intelligence.] “Reproducible research, a growing movement within many scientific fields, including machine learning, would require the code, used to generate…

Read More

SUTCH: Lesson Learned from Replicating Piketty — How Not To Do Economic History

[NOTE: This post refers to the article “The One Percent across Two Centuries: A Replication of Thomas Piketty’s Data on the Concentration of Wealth in the United States” by Richard Sutch. It appears in the current issue of the journal Social…

Read More

Replicating Piketty’s “Capital in the Twenty-First Century”

 [From the blog post “Is Piketty’s Data Reliable?” by Alex Tabarrok at Marginal Revolution] “When Thomas Piketty’s Capital in the Twenty-First Century first appeared many economists demurred on the theory but heaped praise on the empirical work. “Even if none of Piketty’s theories stands…

Read More

IN THE NEWS: NY Times Magazine (October 18, 2017)

[From the article “When the Revolution Came for Amy Cuddy” by Susan Dominus at nytimes.com]  “As a young social psychologist, she played by the rules and won big: an influential study, a viral TED talk, a prestigious job at Harvard. Then, suddenly,…

Read More