Lesson Learned from Registered Reports From Somebody Who Was There at the Beginning

[Excerpts taken from “The registered reports revolution: Lessons in cultural reform” by Chris Chambers, published in Significance, a publication of the Royal Statistical Society”]
“On 12 November 2012, as my train sped towards London, I received one of the most important emails of my life. The message cordially informed me that the publisher of Cortex, a scientific journal that I had recently joined as an editor, had approved our request to launch a new type of article called a registered report.”
“Having just one journal offer registered reports was never going to be enough, and so, in the months that followed, we lobbied hard for others to follow suit. In June 2013, we brought together over 80 journal editors and wrote a joint article in The Guardian. ‘[A]s a group of scientists with positions on more than 100 journal editorial boards,’ we wrote, ‘we are calling for all empirical journals in the life sciences – including those journals that we serve – to offer pre-registered articles at the earliest opportunity.'”
“Regardless of the storm it created (or perhaps because of it), the letter appeared to work as intended. Beyond the heat of the debate – and the numerous misconceptions voiced by opponents – other scientists were quietly deciding that registered reports made sense. The idea had turned a corner and the number of adopting journals steadily grew. By the end of 2013 we had three adopting journals. A year later we had seven. Through 2015, 2016, 2017 and 2018, the number of adopters rose to 21, 41, 88 and 154.”
“Today, registered reports are offered by 202 journals and rising, and across a wide range of sciences. Nearly 200 completed articles have been published, with hundreds more in the pipeline.”
“…we are seeing the first evidence of impacts, and the signs are promising: registered reports are more likely to reveal evidence that is inconsistent with the authors’ pre-specified hypotheses (a possible indicator of reduced confirmation bias); they also have more reproducible code and data than regular articles; and they are cited, on average, at or above the impact factors of the journals in which they are published.”
“Reforming science is a hideously difficult task…If someone then comes along and says, “Whoa, there! You should archive your data in a public repository. You should preregister your protocols to control bias. You should replicate that study before submitting it to Nature”, the response of the player isn’t even “No”. The response is silence.”
“The working scientist sweeps past in a blur of desperation, racing towards the next publication, or tenure, or the next grant or fellowship with a tiny success rate. Every so often, the weary explorer looks up and catches a fragment of the reform argument. ‘You’re telling me I should do X, Y, Z. But why would I? Unless someone is going to make everyone else do it too, I’m just going to become less competitive. I’m a scientist, not a sacrificial lamb.’” 
“‘Should’ arguments, like those above, fail because they offer only judgement, not solutions. If should arguments were sufficient to change behaviour then behaviour would have changed decades ago.”
“Breaking this impasse requires aligning incentives so that any reform works for the community and the individual. Registered reports achieve this by turning the pursuit of high quality science into a virtuous transaction: submit your protocol to our journal, receive a positive assessment (most likely after performing some revision based on expert reviews), and we will guarantee to publish your paper regardless of whether or not your results support your hypothesis. Work hard at designing rigorous, careful, important science and we will de-stress your life by making the results a dead currency in quality evaluation.”
“The key lesson of registered reports here is clear: do not tell scientists what they should do. Instead, tell them why what you are proposing is better than the status quo and why the new practice is both in their career interests as well as in the interests of the scientific community. Give them no reason to say, ‘This will harm my career’ or ‘My peers will disapprove’. Give them every reason to say, ‘This works for me and helps me leave a lasting positive legacy on my field’.”
To read more, click here.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: