[Excerpts taken from the RFP “Data Enhancement of the DARPA SCORE Claims Dataset” posted at the Center for Open Science website]

“The DARPA SCORE Dataset contains claims from about 3,000 empirical papers published between 2009 and 2018 in approximately 60 journals in the social and behavioral sciences.”

“We seek proposals from research teams to enhance the Dataset with information about the papers that may be relevant to assessing the credibility of the coded claims. Such enhancements could include information such as:”

– “Extraction of statistical variables or reporting errors in the original papers.”

– “Identification of the public availability of data, materials, code, or preregistrations associated with the studies reported in the papers.”

– “Citations or other altmetrics associated with the papers.”

– “Identification of replications or meta-analytic results of findings associated with the paper or claims.”

– “Indicators of credibility, productivity, or other features of the authors of the papers.”

– “Extraction of design features, reporting styles, quality indicators, or language use from the original papers.”

“Possible data enhancements for the Database are not limited to these examples.”

“The key criteria for proposed data enhancements are:”

– “Relevance of the proposed variable additions to assessment of credibility of the papers and claims.”

– “Potential applicability of credibility/validity variable additions to other/broader targets or levels of abstraction besides claims or papers. For example, averaging or aggregation of claim or paper-level scores to generate “average”/expected credibility of authors, journals, or sub-disciplines.”

– “The proportion of papers that are likely to benefit from the data enhancement (i.e., minimization of missing data).”

– “The extent to which the data enhancement is automated.”

– “Evidence that extraction or identification of new data is valid, reliable, and easy to integrate with the Dataset.”

– “Description of data quality control practices that will be employed in proposed data-generation activity (e.g., interrater agreement tests, data sample auditing/revision processes).”

– “Cost for conducting the work.”

– “Completing the proposed work by June 30, 2020.”

“A total of $100,000 is available for data enhancement awards and we expect to make 4 to 15 total awards. Proposals should be no more than 2 pages and address the selection criteria above.”

To learn more, click here.