DID, IV, RCT, and RDD: Which Method Is Most Prone to Selective Publication and p-Hacking?
[From the working paper, “Methods Matter: P-Hacking and Causal Inference in Economics” by Abel Brodeur, Nikolai Cook, and Anthony Heyes]
“…Applying multiple methods to 13,440 hypothesis tests reported in 25 top economics journals in 2015, we show that selective publication and p-hacking is a substantial problem in research employing DID and (in particular) IV methods. RCT and RDD are much less problematic. Almost 25% of claims of marginally signiﬁcant results in IV papers are misleading.”