[From the article “Certify reproducibility with confidential data” by Christophe Pérignon, Kamel Gadouche, Christophe Hurlin, Roxane Silberman, and Eric Debonnel, published in Science]

“Many government data, such as sensitive information on individuals’ taxes, income, employment, or health, are available only to accredited users within a secure computing environment…However, researchers using confidential data are inexorably challenged with regard to research reproducibility.”

“We describe an approach that allows researchers who analyze confidential data to signal the reproducibility of their research. It relies on a certification process conducted by a specialized agency accredited by the confidential-data producers and which can guarantee that the code and the data used by a researcher indeed produce the results reported in a scientific paper.”

“In France, the Centre d’Accès Sécurisé aux Données (CASD) is a public research infrastructure that allows users to access and work with government confidential data under secured conditions. This center currently provides access to data from the French Statistical Institute and the French Ministries for Finance, Justice, Education, Labor, and Agriculture, as well as Social Security contributions and health data.”

“The Certification Agency for Scientific Code and Data (cascad, http://www.cascad.tech) is a not-for-profit certification agency created by academics…cascad was granted a permanent accreditation by the French Statistical Secrecy Committee to all 280 datasets available on CASD…the whole certification process remains within the CASD environment…no data can ever be downloaded.”

“When an author requests a cascad certification for a paper, he or she needs to provide the paper, the computer code used in the analysis, and any additional information (software version, readme files, etc.) required to reproduce the results. Then, a reproducibility reviewer, who is a full-time cascad employee specialized in the software used by the author, accesses a CASD virtual machine that is a clone of the one used by the author.”

“The reviewer executes the code, compares compares the output with the results displayed in the tables and figures of the paper, and lists any potential discrepancies in an execution report…a reproducibility certificate is sent to the author and is stored in the cascad database…The author can transfer the reproducibility certificate to an academic journal when submitting a new manuscript.”

To read the full article, click here. (NOTE: The article is behind a paywall.)