Meta-Research: The Big Picture
[From the paper “Calibrating the scientific ecosystem through meta-research” by Tom Hardwicke, Stylianos Serghiou, Perrine Janiaud, Valentin Danchev, Sophia Crüwell, Steven Goodman, and John Ioannidis, forthcoming in Annual Review of Statistics and Its Application]
“Meta-research has been defined as ‘the study of research itself: its methods, reporting, reproducibility, evaluation, and incentives.'”
“We will map the endeavor of meta-research using the translational framework depicted in Figure 1. This framework is not necessarily comprehensive and aims to be descriptive rather than prescriptive. The goal is to highlight how individual meta-research projects form part of a broader effort to continuously calibrate the scientific ecosystem towards higher standards of efficiency, credibility, and quality.”
Outline
1. IDENTIFYING PROBLEMS
1.1 Incentives and norms
1.2 Lack of transparency
1.3 Statistical schools of thought and statistical misuse
1.4 Reproducibility
2. INVESTIGATING PROBLEMS
2.1 Incentive structures
2.2 Publication bias and selective reporting
2.3 Transparency of research resources
2.4 Suboptimal research design
2.5 Statistical misuse
2.6 Reproducibility
3. DEVELOPING SOLUTIONS
3.1 Journal, funder, society and university policies
3.2 Pre-registration and registered reports
3.3 Reporting guidelines
3.4 Peer review
3.5 Collaboration
3.6 Statistical reform
3.7 Evidence synthesis
4. EVALUATION SOLUTIONS
4.1 Journal policy
4.2 Reporting guidelines
4.3 Pre-registration and registered reports
To read the article, click here.
Like this:
Like Loading...
The Replication Network 
This study cites a previous one twice that was also discussed here https://replicationnetwork.com/2019/05/07/a-random-sampling-of-the-state-of-transparency-and-reproducibility-in-social-science-journals/
and that as I already pointed out is based on an sample of studies that is alarmingly small and completely ill-suited for the chosen subject of “social sciences” as a very large part of the studies investigated are from journals that do not cover social sciences at all. I recommend a correction.
Hardwicke TE, Wallach JD, Kidwell MC, Ioannidis, JPA (2019). An empirical assessment of transparency and reproducibility-related research practices in the social sciences (2014-2017). MetaArXiv Preprints. Retrieved from: https://doi.org/10.31222/osf.io/6uh
LikeLike