Just in case you missed it, the latest issue of Behavioral and Brain Sciences includes an article by Rolf Zwaan, Alexander Etz, Richard Lucas, and Brent Donnellan entitled “Making Replications Mainstream”. It is something of a tour-de-force by four prominent scholars in the area of psychology and cognitive science. The article is organized around 6 “concerns”. Each concern addresses a concern or criticism of replication, which the authors then respond to.
The six concerns are:
Concern I: Context is too variable
Concern II: The theoretical value of direct replications is limited
Concern III: Direct replications are not feasible in certain domains
Concern IV: Replications are a distraction
Concern V: Replications affect reputations
Concern VI: There is no standard method to evaluate replication results
Together, the discussion of the respective concerns, along with the authors’ responses, provides a thorough introduction to the current state of play of replications in psychology/cognitive science.
But wait! There’s more!
Following the main article are thirty-six (36!) commentaries by a constellation of academic stars all weighing in on various aspects of the appropriate place of replication in psychological science. Many of the names will be familiar to even the most distant observers of the replication debate: Daniel Kahneman, John Ioannidis, Andrew Gelman, and many, many others. How many others? Eight-one!
Counting the original four authors, this collection of main article and commentaries consists of the combined efforts of 85 scholars. Thus, in both quality and quantity, the discussion testifies to the serious attention that replication is receiving in psychology and cognitive science.
Economists should blush with shame given their meagre efforts in comparison.
To read the article and corresponding commentaries, click here (but note that the article is behind a paywall).