[From the article, “The effect of the conservation reserve program on rural economies: Deriving a statistical verdict from a null finding” by Jason Brown, Dayton Lambert, and Timothy Wojan, recently published in the American Journal of Agricultural Economics]

“This article suggests two methods for deriving a statistical verdict from a null finding, allowing economists to more confidently conclude when “not significant” can be interpreted as “no substantive effect.” The example used to demonstrate the methods is the Economic Research Service’s (ERS) 2004 Report to Congress on the economic implications of the Conservation Reserve Program (CRP).”

“…The ERS study … supported a cautious conclusion of “no evidence of negative employment impacts from CRP.” However, the report correctly noted that the “absence of evidence is not evidence of absence.” Rather, the statistical power of the test was unknown.”

“… we develop approaches to generate ex post statistical power estimates to supplement the interpretation of nonsignificant findings. The first approach uses a bootstrap resampling-with-replacement procedure. The second approach is Bayesian, estimating power based on posterior marginal distributions of posited effect sizes.”

“…In many circumstances, economists do not have the opportunity to conduct ex ante power analysis before research starts. The approaches we suggest can be used to determine ex post power for univariate analyses or multivariate regressions if the data-generating process can be replicated and if the effect size of economic significance or policy relevance is stated.”

To read the full article, click here (but note that it is behind a paywall).