Archives


REED: Replications in Economics are Different from Replications in Psychology, and Other Thoughts

In July 2017, Economics: The Open Access, Open Assessment E-Journal issued a call for papers for a special issue on the practice of replication. The call stated, “This special issue is designed to highlight alternative approaches to doing replications, while…

Read More

How Pre-Registration Can Lead to Higher-Powered Studies and Increase Reproducibility

[From the paper “Preregistration and reproducibility” by Eirik Strømland, forthcoming in the Journal of Economic Psychology] “I present a simple model in which the research community is populated by agents who may report their results unconditionally or conditionally on statistical…

Read More

Resources for Incorporating Pre-registration and Replication in the Classroom

Nicole Janz (University of Nottingham, @PolSciReplicate) has a set of great resources for those looking for an example of a pre-registration assignment for undergraduates (this for a political science course). The slides to accompany the assignment are here. The template…

Read More

How to Win Friends (or at Least Not Maximize Enemies) and Influence People When Doing Replications

[From the paper “Good and Bad Replications in Political Science: How Replicators and Original Authors (Should) Talk to Each Other” by Nicole Janz and Jeremy Freese, prepared for presentation at the MZES Open Social Science Conference 2019] “We propose two main…

Read More

An Interesting Thread on Pre-registration

The Psychonomic Society has been hosting an interesting blog series on pre-registration. Below are the five blogs with ridiculously brief, and probably misleading, summaries: Stephen Lindsay (“Arguments for Preregistering Psychology Research”) SUMMARY: Proper interpretation of the results from hypothesis testing…

Read More

Changes in Open Science Practices Over Time in Economics, Political Science, Psychology, and Sociology

[From a talk by Betsy Levy Paluck entitled “Open Science Practices are on the Rise Across Four Social Science Disciplines”, based on research with co-authors David Birke, Garret Christensen, Rebecca Littman, Ted Miguel, & Zenan Wang, presented at the annual meeting of BITSS]…

Read More

GOODMAN: Hold the Bus!

A recent news piece in Nature reported in glowing terms on the “first analysis of ‘pre-registered’ studies”, stating that “[pre-registration] seems to work as intended: to reduce publication bias for positive results.” There are reasons to be somewhat dubious about…

Read More

IN THE NEWS: New York Times (November 19, 2018)

[From the article, “Essay: The Experiments Are Fascinating. But Nobody Can Repeat Them” by Andrew Gelman, published in The New York Times] “At this point, it is hardly a surprise to learn that even top scientific journals publish a lot…

Read More

Your One Stop Shop for the Pre-registration Debate

Recently the 59th annual meeting of the Psychonomic Society in New Orleans played host to an interesting series of talks on how statistical methods should interact with the practice of science. Some speakers discussed exploratory model building, suggesting that this activity may not…

Read More

When Pre-Registration Isn’t Practical

[From the working paper, “Sound Inference in Complicated Research: A Multi-Strategy Approach” by Sanjay Srivastava, posted at PsyArXiv Preprints] “Preregistration is effective because it creates decision independence: analytic decisions are the same regardless of the specific and potentially spurious features of…

Read More