Archives


GOODMAN: When You’re Selecting Significant Findings, You’re Selecting Inflated Estimates

Replication researchers cite inflated effect sizes as a major cause of replication failure. It turns out this is an inevitable consequence of significance testing. The reason is simple. The p-value you get from a study depends on the observed effect…

Read More

The Problem Isn’t Bad Incentives, It’s the Ritual Behind Them

[From the article, “Statistical Rituals: The Replication Delusion and How We Got There” by Gerd Gigerenzer, published in Advances in Methods and Practices in Psychological Science] “The “replication crisis” has been attributed to misguided external incentives gamed by researchers (the…

Read More

Choosing the Right α: What You Need to Know

[From the article “The quest for an optimal alpha” by Jeff Miller and Rolf Ulrich, published in PLOS One] “The purpose of the present article is to show exactly what is necessary to provide a principled justification for a particular α…

Read More

MILLER: The Statistical Fundamentals of (Non-)Replicability

“Replicability of findings is at the heart of any empirical science” (Asendorpf, Conner, De Fruyt, et al., 2013, p. 108) The idea that scientific results should be reliably demonstrable under controlled circumstances has a special status in science.  In contrast…

Read More

Excellent, Cross-Disciplinary Overview of Scientific Reproducibility in the Stanford Encyclopedia of Philosophy

[From the article “Reproducibility of Scientific Results”, by Fiona Fidler and John Wilcox, published in The Stanford Encyclopedia of Philosophy] “This review consists of four distinct parts. First, we look at the term “reproducibility” and related terms like “repeatability” and…

Read More

Redefining RSS

[From the blog “Justify Your Alpha by Decreasing Alpha Levels as a Function of the Sample Size” by Daniël Lakens, posted at The 20% Statistician] “Testing whether observed data should surprise us, under the assumption that some model of the data is…

Read More

How do Psychologists Interpret Nonsignificant Results? Evidence from the Journals

[From the abstract to the article, “Quantifying Support for the Null Hypothesis in Psychology: An Empirical Investigation” by Aczel et al., recently published in Advances in Methods and Practices in Psychological Science] “In the traditional statistical framework, nonsignificant results leave researchers…

Read More

How to Get Something from Nothing. Or, “Yes, Virginia, You Can Do Ex-post Power Analyses”.

[From the article, “The effect of the conservation reserve program on rural economies: Deriving a statistical verdict from a null finding” by Jason Brown, Dayton Lambert, and Timothy Wojan, recently published in the American Journal of Agricultural Economics] “This article suggests…

Read More

REED: Why Lowering Alpha to 0.005 is Unlikely to Help

[This blog is based on the paper, “A Primer on the ‘Reproducibility Crisis’ and Ways to Fix It” by the author] A standard research scenario is the following: A researcher is interested in knowing whether there is a relationship between…

Read More

PARASURAMA: Why Overlapping Confidence Intervals Mean Nothing About Statistical Significance

[NOTE: This is a repost of a blog that Prasanna Parasurama published at the blogsite Towards Data Science]. “The confidence intervals of the two groups overlap, hence the difference is not statistically significant”  The statement above is wrong. Overlapping confidence…

Read More