About the repliCATS project

Based at the University of Melbourne, the repliCATS project team are part of a wider program called SCORE funded by DARPA. We are excited about reimagining peer review as a structured deliberation process. We’re testing this by crowdsourcing expert judgements about the credibility of published research in eight social science disciplines – criminology, education, economics, marketing, management, political science, psychology, public administration, and sociology.

repliCATS workshops – get paid to do post-publication peer review!

In workshops you will be asked to evaluate the credibility of two to three published papers within your domain expertise. We use a structured group deliberation approach called the IDEA protocol on a custom-built web-based platform which means you can participate from anywhere in the world! Working in small groups, you will be asked to evaluate papers across a number of credibility signals – from its comprehensibility, to its replicability, robustness and transparency.

Participants will first make private judgements about the paper, and then get to review and discuss their group’s responses before submitting their final judgements. For these workshops, we’ll group you by compatible time zones, and each group will have its own facilitator who will guide you through a short virtual discussion.

All participants are eligible for a US$200 assessment grant.

Your time commitment

Workshops will run over a period of 7 days, but you get to make your judgements when it suits you over that period. Here is how it works:

1) Consent & demographics, and create repliCATS platform account before the workshop: 20-25 minutes.

2) Workshop introduction: 45 mins, a number of live webinars run by the repliCATS team. Also recorded and can be watched on demand.

3) Assessment period: 45-70 minutes/paper at your own pace over the 7-day assessment period.

4) Group discussion: 30-45 minutes during the 7-day assessment period, dedicated facilitator will guide discussion.

5) Assessment grant administration: 5-10 minutes.

Sign-ups now open for July, August & September workshops

20-27 July – economics, marketing & management (35-40 participants)

24-31 August – Criminology, Political Science & Sociology (35-40 participants)

21-28 September – Criminology, Political Science, Public administration (government and law) & Sociology (35-40 participants)

Sign-up here:  https://replicats.research.unimelb.edu.au/2021/06/02/express-interest-now-open-jul-sept-bushel-workshops/

Have questions? E-mail us at repliCATS-project@unimelb.edu.au