Meta-Research: The Big Picture

[From the paper “Calibrating the scientific ecosystem through meta-research” by Tom Hardwicke, Stylianos Serghiou, Perrine Janiaud, Valentin Danchev, Sophia Crüwell, Steven Goodman, and John Ioannidis, forthcoming in Annual Review of Statistics and Its Application]
“Meta-research has been defined as ‘the study of research itself: its methods, reporting, reproducibility, evaluation, and incentives.'”
“We will map the endeavor of meta-research using the translational framework depicted in Figure 1. This framework is not necessarily comprehensive and aims to be descriptive rather than prescriptive. The goal is to highlight how individual meta-research projects form part of a broader effort to continuously calibrate the scientific ecosystem towards higher standards of efficiency, credibility, and quality.”
1.1 Incentives and norms
1.2 Lack of transparency
1.3 Statistical schools of thought and statistical misuse
1.4 Reproducibility
2.1 Incentive structures
2.2 Publication bias and selective reporting
2.3 Transparency of research resources
2.4 Suboptimal research design
2.5 Statistical misuse
2.6 Reproducibility
3.1 Journal, funder, society and university policies
3.2 Pre-registration and registered reports
3.3 Reporting guidelines
3.4 Peer review
3.5 Collaboration
3.6 Statistical reform
3.7 Evidence synthesis
4.1 Journal policy
4.2 Reporting guidelines
4.3 Pre-registration and registered reports
To read the article, click here.

One Comment on “Meta-Research: The Big Picture

  1. This study cites a previous one twice that was also discussed here
    and that as I already pointed out is based on an sample of studies that is alarmingly small and completely ill-suited for the chosen subject of “social sciences” as a very large part of the studies investigated are from journals that do not cover social sciences at all. I recommend a correction.
    Hardwicke TE, Wallach JD, Kidwell MC, Ioannidis, JPA (2019). An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017). MetaArXiv Preprints. Retrieved from:


Leave a Reply to Jan H. Höffler Cancel reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: