The Replication Network

Economists aspire to adopt practices of the natural sciences where replication is seen as a crucial and necessary activity. In the natural sciences results are seldom given much credibility unless they have been replicated. Replication contributes in a crucial way to quality. So it is rather strange that replication has traditionally not been seen as a rewarding activity among economists.

Nevertheless, replication has been an ongoing concern among economists; manifest perhaps most prominently the American Economic Review’s requirement that datasets are made available to other researchers prior to accepting an article for publication. This practice has been praised by other branches of the social sciences and there have been various movements to encourage replication (see blogroll on this website).

Given these development we felt it is time to revisit the progress that has been made in the area of replication in economics. Our paper reports the findings of a survey which was administered to the editors of 333 economics journals. It also provides an analysis of 162 replication studies that have been published in peer-reviewed economics journals from 1977-2014.

We find that the publication of replication studies has been slow and that few journals publish them, though this is thankfully changing now. It is not unusual to find replication studies that do not confirm the results of the original studies. This might explain why journals are slow to publish these as they are afraid of the potentially contentious nature of exchanges between replicator and replicatee that might follow.

However, in times of transparency, accountability and easy accessibility of data sources, the economics profession cannot afford to continue to neglect the area of replication. Recent cases such as Reinhart and Rogoff or Piketty have received a lot of media attention naming and shaming leading academics as well as putting pressure on the profession as a whole to account for their work.

We hope that our paper can lead to renewed discussions of replication in economics, especially among editors of economics journals.