Archives


Have Registered Reports Uncovered Massive Publication Bias? Evidence from Psychology

[Excerpts taken from the preprint, “An excess of positive results: Comparing the standard Psychology literature with Registered Reports” by Anne Scheel, Mitchell Schijen, and Daniël Lakens, posted at PsyArXiv] “Registered Reports (RRs) are a new publication format…Before collecting data, authors…

Read More

What Do Replications Tell Us About the Reliability of Meta-Analyses? Evidence from Psychology

[Excerpts taken from the article “Comparing meta-analyses and preregistered multiple-laboratory replication projects” by Amanda Kvarven, Eirik Strømland, and Magnus Johannesson, published in Nature Human Behaviour] “In the past 30 years, the number of meta-analyses published across scientific fields has been…

Read More

Bad News and Good News for Meta-Analyses in Economics

[Excerpts taken from the working paper “Practical Significance, Meta-Analysis and the Credibility of Economics” by Tom Stanley and Chris Doucouliagos, posted at SSRN] “…we find that large biases and high rates of false positives will often be found by conventional meta-analysis methods….

Read More

Pre-Registration: It’s a Journey

[Excerpts taken from the preprint “Preregistration Is Hard, And Worthwhile” by Brian Nosek and others, posted at PsyArXiv Preprints] “Preregistration of studies serves at least three aims for improving the credibility and reproducibility of research findings.” “First, preregistration of analysis…

Read More

Meta-Research: The Big Picture

[From the paper “Calibrating the scientific ecosystem through meta-research” by Tom Hardwicke, Stylianos Serghiou, Perrine Janiaud, Valentin Danchev, Sophia Crüwell, Steven Goodman, and John Ioannidis, forthcoming in Annual Review of Statistics and Its Application] “Meta-research has been defined as ‘the…

Read More

GOODMAN & REED: A Friendly Debate about Pre-Registration

Background: Nat Goodman is generally pessimistic about the benefits of pre-registration. Bob Reed is generally optimistic about pre-registration. What follows is a back-and-forth dialogue about what each likes and dislikes about pre-registration. [GOODMAN, Opening Statement] We need to remember that…

Read More

Not Only That, Effect Sizes from Registered Reports Are Also Much Lower

[In a recent post at TRN, we highlighted that positive results were drastically lower in registered reports. In this post, we report findings about effect sizes. Excerpts are taken from “The Meaningfulness of Effect Sizes in Psychological Research: Differences Between…

Read More

Why So Many Insignificant Results in a Meta-analysis?

[From the blog “Where Do Non-Significant Results in Meta-Analysis Come From?” by Ulrich Schimmack, posted at Replicability-Index] “It is well known that focal hypothesis tests in psychology journals nearly always reject the null-hypothesis … However, meta-analyses often contain a fairly…

Read More

Another Economics Journal Pilots Pre-Results Review

[From the article “Pre-results review reaches the (economic) lab: Experimental Economics follows the Journal of Development Economics in piloting pre-results review”, an interview with Irenaeus Wolff, published at http://www.bitss.org. The following are excerpts from that interview.] “In its April 2019…

Read More

IN THE NEWS: Vox (May 17, 2019)

[From the article “This economics journal only publishes results that are no big deal: Here’s how that might save science” by Kelsey Piper, published in Vox] “Most new publications, upon their launch, seek to promote their content as novel, surprising,…

Read More