[*AoI = “Articles of Interest” is a feature of TRN where we report excerpts of recent research related to replication and research integrity.]

EXCERPTS (taken from the article)

“Pre-registration is regarded as an important contributor to research credibility. We investigate this by analyzing the pattern of test statistics from the universe of randomized controlled trials (RCT) studies published in 15 leading economics journals.”

“We draw two conclusions:”

“(a) Pre-registration frequently does not involve a pre-analysis plan (PAP), or sufficient detail to constrain meaningfully the actions and decisions of researchers after data is collected. Consistent with this, we find no evidence that pre-registration in itself reduces p-hacking and publication bias.”

“(b) When pre-registration is accompanied by a PAP we find evidence consistent with both reduced phacking and publication bias.”

“…we proceed with notions of pre-registration and PAPs as practiced in economics, or at least as operationalized by the largest and most influential professional association in the discipline, the AEA.”

“In their discussion relating to psychology, Nosek et al. (2018) contend that: “An effective solution is to define the research questions and analysis plan before observing the research outcomes – a process called preregistration,”which implies that pre-registration and the existence of a PAP are one and the same thing (see also Simmons et al. (2021) for a similar contention). This is far from how things work in economics, as we will show here.”

“We contend that many readers believe, wrongly according to our analysis, that pre-registration in itself implies enhanced research credibility, which would explain the weight apparently attached to whether a study is pre-registered or not in assessing its likely validity. Our finding is that credibility is enhanced only with inclusion of a PAP.”

REFERENCES:

Brodeur, Abel; Cook, Nikolai M.; Hartley, Jonathan S.; Heyes, Anthony (2023) : Do Pre-Registration and Pre-Analysis Plans Reduce p-Hacking and Publication Bias?: Evidence from 15,992 Test Statistics and Suggestions for Improvement, GLO Discussion Paper, No. 1147 [pre.], Global Labor Organization (GLO), Essen

Nosek, B. A., Ebersole, C. R., DeHaven, A. C. and Mellor, D. T.: 2018, The Preregistration Revolution, Proceedings of the National Academy of Sciences 115(11), 2600–2606.

Simmons, P. J., Nelson, L. D. and Simonsohn, U.: 2021, Pre-Registration: Whyand How, Journal of Consumer Psychology 31(1), 151–162.