[From the article, “Essay: The Experiments Are Fascinating. But Nobody Can Repeat Them” by Andrew Gelman, published in The New York Times]

“At this point, it is hardly a surprise to learn that even top scientific journals publish a lot of low-quality work — not just solid experiments that happen, by bad luck, to have yielded conclusions that don’t stand up to replication, but poorly designed studies that had no real chance of succeeding before they were ever conducted.”

“…We see it all the time. Remember the claims that subliminal smiley faces on a computer screen can cause big changes in attitudes toward immigration? That elections are decided by college football games and shark attacks? These studies were published in serious journals or promoted in serious news outlets.”

“Scientists know this is a problem. In a recent paper in the journal Nature Human Behaviour, a team of respected economists and psychologists released the results of 21 replications of high-profile experiments.”

“…Here’s where it gets really weird. The lack of replication was predicted ahead of time by a panel of experts using a ‘prediction market,’ in which experts were allowed to bet on which experiments were more or less likely to — well, be real.”

“… One potential solution is preregistration, in which researchers beginning a study publish their analysis plan before collecting their data. Preregistration can be seen as a sort of time-reversed replication, a firewall against “data dredging,” the inclination to go looking for results when your first idea doesn’t pan out. But it won’t fix the problem on its own.”

To read more, click here.