[From the article “A solution to psychology’s reproducibility problem just failed its first test” by David Adam, published at www.sciencemag.org]

“Efforts to improve the robustness of research by asking psychologists to state their methods and goals ahead of time, a process called preregistration, have stumbled at the first hurdle.”

“‘Preregistration is not as easy as it may seem,’ says Aline Claesen, a psychologist at the Catholic University of Leuven (KU Leuven) in Belgium. She and her colleagues examined 27 preregistration plans filed by psychologists from February 2015 … to November 2017. In every case, … the researchers deviated from their plan—and in every paper but one, they did not fully disclose these deviations.”

“‘I was totally surprised by how many of these [changes] were undisclosed,’ says Wolf Vanpaemel, a psychologist on the KU Leuven team. ‘There’s no good excuse for not transparently indicating all changes you made.'”

“The lack of transparency is troubling, but understandable, Vanpaemel says: Some researchers might fear their paper won’t be published if they admit to not having entirely followed their preregistration.”

“Steve Lindsay of the University of Victoria in Canada who is also editor-in-chief of Psychological Science admits that he has given authors plenty of leeway to write vague preregistrations and not account for all the deviations in a paper. He says policing the system would take time and effort the journal hasn’t budgeted for.”

“…Dan Simons, a psychologist at the University of Illinois in Champaign, describes the identified shortcomings as growing pains. ‘My guess is that most [authors] were well-intentioned and just didn’t know how to do it very well.'”

“Brian Nosek, a psychologist at the University of Virginia in Charlottesville who directs the Center for Open Science, which runs the Open Science Framework, says the KU Leuven team’s findings should help. ‘The key message here,’ he says, ‘is that preregistration is a skill and not a bureaucratic process.'”

To read more, click here.