Economics has become an empirical discipline. Applied econometrics has replaced mathematical economics in all but a few niche journals, and economists are collecting primary data again. But publication practices are lagging behind. Replication of a theoretical paper has never been an issue. You get out your pencil and paper and work through the proof of this or that theorem. Replication of empirical papers requires more consideration.
Led by the American Economic Association, an increasing number of economics journals demand that empirical papers be replicable. Data and code are archived with the paper. Efforts are now underway, by Mendeley and others, to make data and code searchable – papers, of course, have been searchable for a long time.
But replicability is not replication. Economics papers are replicated all the time, but the results are the subject of classroom discussions, online gossip, and whispers at conferences – rather than published formally. The profession rewards original contributions and looks down at derivative research. The former sentiment is fine but the latter is not. Not every paper can break new ground. Not every economist can win a Nobel Prize. The numbers matter, particularly in policy advice, and checking someone else’s results is important for building confidence in the predictions we make about the impact of policy interventions.
Energy economics is a subdiscipline of applied economics. Reliable, affordable and clean energy is fundamental to economic activity, social justice, and environmental quality. The results published in Energy Economics inform and shape energy policy. The results therefore had better be right. In this regard, energy economics is not different from health economics, labour economics, or education economics. Energy Economics takes replication sufficiently seriously to incentivise it.
Inviting replication papers is dangerous. Anyone can download the .dta and .do files for a paper, click a few buttons, and claim success. At Energy Economics, we are still wondering how to report successful replications, how to reward replicators, and how to tell genuine claims of successful replication from trivial or fake ones.
For the special issue of Energy Economics, we therefore opted to stretch the notion of replication. We call for papers that replicate and update important, but outdated papers. We call for papers that encompass and explain contrasting findings in previously published papers. Thus defined, replication is still derivative – there is no escape from that – but it is not intellectually barren. Replicators have to make an effort to get the reward, a publication in the top field journal.
This is an experiment. We hope to attract interesting replications. While there certainly has been a lot of interest, we will have to carefully study whether the submitted papers meet our expectations.
The experiment is not limited to Energy Economics. Elsevier wants to foster a new surge in reproducibility and reproduction. There are some perceived barriers to disseminating replication studies, such as that they are only valuable if the results disagree with the original research, or that editors don’t want to publish these studies. We want to break these myths. Elsevier are now working on a range of initiatives that raise the bar on reproducibility and lower the barriers for researchers to publish replication studies, including a series of Virtual Special Issues, a new article type especially for replication studies, and various calls for papers (the first one in Energy Economics) to encourage submissions. By empowering researchers to share their methods and data, championing rigorous and transparent reporting and creating outlets for replication research, Elsevier is helping to make reproducibility and replication a reality.
Making sure (published) research can be reproduced is a massive step towards making it trustworthy and showing peers, funders and the public that science can be trusted. Publishing replication studies contributes to building this trust, ultimately safeguarding science.
Richard Tol is the Editor-in-Chief of Energy Economics. He teaches at the University of Sussex and the Vrije Universiteit Amsterdam. Donna de Weerd-Wilson is the Executive Publisher of Energy Economics, and manages one of the Economics portfolios at Elsevier.