[From the article “Second-Guessing Predictions: When to trust scientific predictions—and when to ignore them” by Alexander Danvers]

“One of the key reforms of the Credibility Revolution in psychology research is the use of preregistration: Scientists write down what they predict will happen in an experiment beforehand, so there’s a record which other scientists and the public can look at when evaluating their work.”

“Unfortunately, in psychology, I and many other young researchers were taught a model of manuscript writing based on a famous book chapter by psychologist Daryl Bem called ‘Writing the Empirical Journal Article.'”

“Bem argues that you should write the article around the most interesting results you can find once you have analyzed your data, minimizing or abandoning what your experiment was originally designed to test if needed.”

“More thoughtful and prominent psychologists than me have already written about how this can undermine the scientific process, especially when the researcher then writes up the new result as if it was the main point of their study all along.”

“This is referred to as HARK-ing, or Hypothesizing After the Results are Known (HARK), and it is deceitful. … It’s like shooting at the barn door and then drawing a bullseye around wherever you hit.”

“But something has always nagged me about this argument about Bem’s article. …If a theory can be made compatible with almost any result, how could I have ever used it to make predictions in the first place? … I would argue that you should not care about my prediction. You, therefore, should not care about my preregistration.”

“There are strong arguments in favor of making psychology a descriptive science.”

“As long as researchers are writing their manuscripts in the language of prediction—my theory predicted this result—we will need preregistration to keep ourselves honest.”

“But as long as we need preregistration to prove what a theory really predicted, we won’t be dealing with very deep theories. The next step in the Credibility Revolution might just be a step away from theory, and towards description.”

To read more, click here.