[Excerpts taken from the article “Are confidence intervals better termed ‘uncertainty intervals’?” by Andrew Gelman and Sander Greenland, published in the BMJ.]

Are confidence intervals better termed “uncertainty intervals?”

Yes—Andrew Gelman

“Confidence intervals can be a useful summary in model based inference. But the term should be “uncertainty interval,” not “confidence interval”…”

“Officially, all that can be interpreted are the long term average properties of the procedure that’s used to construct the interval, but people tend to interpret each interval implicitly in a bayesian way—that is, by acting as though there’s a 95% probability that any given interval contains the true value.”

“Using confidence intervals to rule out zero (or other parameter values) involves all of the well known problems of significance testing. So, rather than constructing this convoluted thing called a confidence procedure, which is defined to have certain properties on average but can’t generally be interpreted for individual cases, I prefer to aim for an uncertainty interval, using the most appropriate statistical methods to get there.”

“Let’s use the term “uncertainty interval” instead of “confidence interval.” The uncertainty interval tells us how much uncertainty we have.”

No—Sander Greenland

“The label “95% confidence interval” evokes the idea that we should invest the interval with 95/5 (19:1) betting odds that the observed interval contains the true value…”

“…the 95% is overconfident because it takes no account of procedural problems and model uncertainties that should reduce confidence in statistical results. Those possibilities include uncontrolled confounding, selection bias, measurement error, unaccounted-for model selection, and outright data corruption.”

“…no conventional interval adequately accounts for procedural problems that afflict data generation or for uncertainties about the statistical assumptions.”

“Nonetheless, all values in a conventional 95% interval can be described as highly compatible with data under the background statistical assumptions, in the very narrow sense of having P>0.05 under those assumptions.”

“In equivalent terms: given any value in the interval and the background assumptions, the data should not seem very surprising. This leads to the intentionally modest term “compatibility interval” as a replacement for ‘confidence interval.'”

“In summary, both “confidence interval” and “uncertainty interval” are deceptive terms, for they insinuate that we have achieved valid quantification of confidence or uncertainty despite omitting important uncertainty sources.

“Replacing “significance” and “confidence” labels with “compatibility” is a simple step to encourage honest reporting of how little we can confidently conclude from our data.”

To read the full article, click here. (NOTE: Article is behind a paywall.)