Archives


Bad News and Good News for Meta-Analyses in Economics

[Excerpts taken from the working paper “Practical Significance, Meta-Analysis and the Credibility of Economics” by Tom Stanley and Chris Doucouliagos, posted at SSRN] “…we find that large biases and high rates of false positives will often be found by conventional meta-analysis methods….

Read More

Pre-Registration: It’s a Journey

[Excerpts taken from the preprint “Preregistration Is Hard, And Worthwhile” by Brian Nosek and others, posted at PsyArXiv Preprints] “Preregistration of studies serves at least three aims for improving the credibility and reproducibility of research findings.” “First, preregistration of analysis…

Read More

Meta-Research: The Big Picture

[From the paper “Calibrating the scientific ecosystem through meta-research” by Tom Hardwicke, Stylianos Serghiou, Perrine Janiaud, Valentin Danchev, Sophia Crüwell, Steven Goodman, and John Ioannidis, forthcoming in Annual Review of Statistics and Its Application] “Meta-research has been defined as ‘the…

Read More

GOODMAN & REED: A Friendly Debate about Pre-Registration

Background: Nat Goodman is generally pessimistic about the benefits of pre-registration. Bob Reed is generally optimistic about pre-registration. What follows is a back-and-forth dialogue about what each likes and dislikes about pre-registration. [GOODMAN, Opening Statement] We need to remember that…

Read More

Not Only That, Effect Sizes from Registered Reports Are Also Much Lower

[In a recent post at TRN, we highlighted that positive results were drastically lower in registered reports. In this post, we report findings about effect sizes. Excerpts are taken from “The Meaningfulness of Effect Sizes in Psychological Research: Differences Between…

Read More

Why So Many Insignificant Results in a Meta-analysis?

[From the blog “Where Do Non-Significant Results in Meta-Analysis Come From?” by Ulrich Schimmack, posted at Replicability-Index] “It is well known that focal hypothesis tests in psychology journals nearly always reject the null-hypothesis … However, meta-analyses often contain a fairly…

Read More

Another Economics Journal Pilots Pre-Results Review

[From the article “Pre-results review reaches the (economic) lab: Experimental Economics follows the Journal of Development Economics in piloting pre-results review”, an interview with Irenaeus Wolff, published at http://www.bitss.org. The following are excerpts from that interview.] “In its April 2019…

Read More

IN THE NEWS: Vox (May 17, 2019)

[From the article “This economics journal only publishes results that are no big deal: Here’s how that might save science” by Kelsey Piper, published in Vox] “Most new publications, upon their launch, seek to promote their content as novel, surprising,…

Read More

In Two Decades, Will We Look Back And Wonder At All the Flawed Research?

[From the article, “Rein in the four horsemen of irreproducibility”, by Dorothy Bishop, published in Nature] “More than four decades into my scientific career, I find myself an outlier among academics of similar age and seniority: I strongly identify with…

Read More

Does Psychology Have a Publication Bias Problem? Yes and No

[From the article, “The Meaningfulness of Effect Sizes in Psychological Research: Differences Between Sub-Disciplines and the Impact of Potential Biases” by Thomas Schäfer and Marcus Schwarz, published April 11, 2019 in Frontiers in Psychology] “From past publications without preregistration, 900…

Read More