Somebody Asks, How Can A Theory Be Falsified? Twitter Responds

[From the Twitter thread started by @JessieSunPsych]
Jessie Sun (@JessieSunPsychrelayed the following question that was raised at a recent Psychology conference: “At what point can a theory be falsified (e.g., if the effect size is d = .02)? We often just predict the direction of the effect, but do we need to think about the specificity of effect sizes?”
This led to a large number of responses. Daniel Lakens (@lakens ) replied by giving three links to works that he has either authored or co-authored, each addressing a piece of the answer.
1) “Good question! For a paper on falsifiability by @richarddmorey and me, see https://medium.com/@richarddmorey/new-paper-why-most-of-psychology-is-statistically-unfalsifiable-4c3b6126365a
Among other things, this blog recommends that a researcher should “power your experiment such that you, or someone else, can conduct a similarly-sized experiment and have high power for detecting an interesting difference from your study. We need to stop thinking about studies as if they are one-offs, only to be interpreted once in light of the hypotheses of the original authors. This does not support cumulative science.”
2) “For setting the smallest effect size of interest and statistical tools (equivalence testing) see https://journals.sagepub.com/doi/abs/10.1177/2515245918770963
Equivalence testing helps one to decide whether failure to reject means null effect.
3) “…and for range predictions, see http://daniellakens.blogspot.com/2018/07/strong-versus-weak-hypothesis-tests.html
The latter addresses how to test a theory (or the claims of a prior paper) that a given parameter takes a range of values. It also encourages researchers to choose alternative hypotheses that would be unlikely to be true unless the theory was correct, so rejection of the null actually means something.
To read the Twitter thread, click here.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: