It is well known that there is a bias towards publication of statistically significant results. In fact, we have known this for at least 25 years since the publication of De Long and Lang (JPE 1992):

“Economics articles are sprinkled with very low t-statistics – marginal significance levels very close to one – on nuisance coefficients. […] Very low t-statistics appear to be systematically absent – and therefore null hypotheses are overwhelmingly false – only when the universe of null hypotheses considered is the central themes of published economics articles. This suggests, to us, a publication bias explanation of our findings.” (pp. 1269-1270)

While statistically insignificant results are less “sexy”, they are often not less important. Failure to reject the null hypothesis can be interesting in itself, is a valuable data point in meta-analyses, or can indicate to future researchers where they are unlikely to find an effect. As McCloskey (2002) famously puts it:

“[…] statistical significance is neither necessary nor sufficient for a result to be scientifically significant.” (p. 54)

This problem is not unique to Economics but several other disciplines have moved faster than us to try and address it. For example, the following disciplines already have journals dedicated to publishing “insignificant” results:

Psychology: Journal of Articles in Support of the Null Hypothesis

Biomedicine: Journal of Negative Results in Biomedicine

Ecology and Evolutionary Biology: Journal of Negative Results

Is it time for Economics to catch up? I suggest it is and I know that I am not alone in this view. In fact, a number of prominent Economists have endorsed this idea (even if they are not ready to pioneer the initiative). So, imagine… a call for papers along the following lines:

Series of Unsurprising Results in Economics (SURE)

Is the topic of your paper interesting, your analysis carefully done, but your results are not “sexy”? If so, please consider submitting your paper to SURE. An e-journal of high-quality research with “unsurprising” findings.

How does it work:

— We accept papers from all fields of Economics…

— Which have been rejected at a journal indexed in EconLit…

— With the ONLY important reason being that their results are statistically insignificant or otherwise “unsurprising”.

To document that your paper meets the above eligibility criteria, please send us all referee reports and letters from the editor from the journal where your paper has been rejected.  Two independent referees will read these reports along with your paper and evaluate whether they indicate that: 1. the paper is of high quality and 2. the only important reason for rejection was the insignificant/unsurprising nature of the results.  Submission implies that you (the authors) give permission to the SURE editor to contact the editor of the rejecting journal regarding your manuscript.

SURE benefits writers by:

— Providing an outlet for interesting, high-quality, but “risky” (in terms of uncertain results) research projects;

— Decreasing incentives to data-mine, change theories and hypotheses ex post, exclusively focus on provocative topics.

SURE benefits readers by:

— Mitigating the publication bias and thus complementing other journals in an effort to provide a complete account of the state of affairs;

— Serving as a repository of potential (and tentative) “dead ends” in Economics research.

Feedback is definitely invited! Please submit your comments here or email me at andrea.menclova@canterbury.ac.nz.

Andrea Menclova is a Senior Lecturer at the University of Canterbury in New Zealand.

REFERENCES:

De Long J. Bradford and Kevin Lang. 1992. “Are all Economic Hypotheses False?” Journal of Political Economy, 100:6, pp.1257-1272

McCloskey, Deirdre. 2002. The Secret Sins of Economics. Prickly Paradigm Press, Chicago.