[From the paper “Good and Bad Replications in Political Science: How Replicators and Original Authors (Should) Talk to Each Other” by Nicole Janz and Jeremy Freese, prepared for presentation at the MZES Open Social Science Conference 2019]

“We propose two main ways to conduct a well-conducted, constructive and publishable replication:”

“First, your study should be carefully and transparently planned. … make very clear if the replication aims to conduct a duplication or replication study, or both, and have a clear list for yourself of what differences between the results you expect when you use new data or methods.”

“Second, a good replication study should also use sensitive and professional wording when referring to the original study to avoid embarrassment for the original author(s). We emphasize that binary judgments such as ‘failed replication’ can do more harm than good.”

“How ‘far’ must your results deviate from the original work before claiming that some effects could not be confirmed? Take into account that different data and measurements (replication) will naturally yield different results …”

“Be transparent and reproducible: Why have you chosen the original study for replication?”

“… your study must have an extensive and clear methods part, and potentially a supplement, to give all details on how the new findings were produced.”

“Ideally, a replication would be pre-registered to eliminate reporting flexibility (Zigerell 2017). … transparency can avoid later accusations of p-hacking or deliberate error hunting. Have someone crosscheck your replication before journal submission.”

​”Always try to talk about the study, not the author, to make it less personal.”

“Gary King advises that a copy of the paper should first be sent to the original author by his students, who can respond to the critique and comment on possible failed replications. In psychology, there have been calls to involve the original authors even earlier, as part of the planning or pre-registration process …”

To read the paper, click here.