[From the paper, “Perceived Crisis and Reforms: Issues, Explanations, and Remedies”, authored by Paul De Boeck and Minjeong Jeon, published in the July issue of Psychological Bulletin]

“…we believe that the OSC [Open Science Collaboration] study allows us to obtain an indication of the true discovery rate (TDR) for the rejections in the original studies and of π0 [the probability of no effect] more in general if the effects investigated in the OSC study are representative.”

“…a cautious conclusion is that a substantial proportion of positive binary inferences are true (35.7% or more) and that another substantial proportion includes either zero or very small effects.”

“…Based on the OSC study the probability of the null hypothesis being true can in fact be estimated if again a number of assumptions are made. Based on the replication studies the TDR for the original studies turned out to be .357 when assuming that the replication study power is .92, while the TDR is .691 when assuming that the replication study power is 0.50. We now can tentatively estimate π0 for the broader set of studies from which the studies selected for the OSC replications are a representative subset.”

“…Starting from a TDR of .357, …, for a power value of .20 [of the original studies], π0 = .878, and for a power value of .50, π0 = .947.

“…Starting from a TDR of .691, …, for a power value of .20 of the original studies, π0 = .641, and for a power value of .50, π0 = .817. For higher power levels even higher estimates of π0 are obtained.”

…

“One cannot expect that significant effects (given that most published effects are significant) replicate well if they relate to unlikely hypotheses, and neither can one expect that they replicate well if they depend on perhaps subtle and not so evident differences in the context and when they depend on a complex interplay of factors.”

To access the full article, click here (but note the article is behind a paywall).