[From the Twitter thread started by @JessieSunPsych]

Jessie Sun (@JessieSunPsych) relayed the following question that was raised at a recent Psychology conference: “At what point can a theory be falsified (e.g., if the effect size is d = .02)? We often just predict the direction of the effect, but do we need to think about the specificity of effect sizes?”

This led to a large number of responses. Daniel Lakens (@lakens ) replied by giving three links to works that he has either authored or co-authored, each addressing a piece of the answer.

1) “Good question! For a paper on falsifiability by @richarddmorey and me, see https://medium.com/@richarddmorey/new-paper-why-most-of-psychology-is-statistically-unfalsifiable-4c3b6126365a”

Among other things, this blog recommends that a researcher should “power your experiment such that you, or someone else, can conduct a similarly-sized experiment and have high power for detecting an interesting difference from your study. We need to stop thinking about studies as if they are one-offs, only to be interpreted once in light of the hypotheses of the original authors. This does not support cumulative science.”

2) “For setting the smallest effect size of interest and statistical tools (equivalence testing) see https://journals.sagepub.com/doi/abs/10.1177/2515245918770963”

Equivalence testing helps one to decide whether failure to reject means null effect.

3) “…and for range predictions, see http://daniellakens.blogspot.com/2018/07/strong-versus-weak-hypothesis-tests.html”

The latter addresses how to test a theory (or the claims of a prior paper) that a given parameter takes a range of values. It also encourages researchers to choose alternative hypotheses that would be unlikely to be true unless the theory was correct, so rejection of the null actually means something.

To read the Twitter thread, click here.