This is an invitation to join a project that aims to improve the quality of research in applied microeconomics by examining the choices that researchers make. We are hoping to recruit up to 200 participants. Full participants in this project will receive coauthorship on the final paper, as well as a $2,000 payment.

Participation is open broadly to anyone with a published or forthcoming paper in the applied microeconomics literature, or who holds a PhD and works in a job where they write non-academic reports using tools to estimate causal effects from applied microeconomics. In addition to academic faculty, this includes researchers in the non-academic private or public sectors, as well as graduate students who have a published or forthcoming paper. Researchers are encouraged to participate from any country, of any gender, race, ethnicity, or sexual or gender identity, and at any stage in their careers. 

This project is a follow-up to Huntington-Klein et al. (2021), where multiple researchers each replicated the same studies (a “many-analyst study”). The study examined differences in the analytic and data-preparation choices that researchers made, and how those choices affected their results. This was the Economic Inquiry Paper of the Year in 2021.

In this new project, a larger number of researchers, up to 200, will independently complete the same research task. Then, there will be several rounds where you will revise your original results. These will follow either peer review, or a change in the research task that standardizes some of the choices made, for example providing a data set in which key variables are pre-prepared instead of having researchers create their own. Without revealing the specific hypotheses of the project, these multiple rounds of revision will allow us to better understand the parts of research that are least standardized across researchers, whether standardization of research methods is desirable, and what tools might be most effective in standardizing research decisions, if that is desirable.

Full participation in the project means completing all rounds of revision. The payment of $2,000, and coauthorship on the paper describing the results of the many-analyst study, are contingent on full participation. You will complete the research task several times, and may occasionally be asked to provide peer review. Your first replication task should take you about as long as you’d expect to spend on creating the results section for a short Letters-style publication. After that, the project is expected to take a several hours of your time every few months, concluding in early 2024.

If you are interested in signing up, please visit the Sign-Up Form, or see the project webpage.

This project is generously supported by the Alfred P. Sloan Foundation, and approved by the Seattle University IRB.