For Evaluators — The Unjournal

Why Evaluate for The Unjournal?

Unlike traditional journals where peer review is unpaid, anonymous, and invisible, The Unjournal compensates evaluators, publishes their work openly, and gives them proper academic credit.

Paid Work

We compensate evaluators for their expertise—averaging $450 per evaluation including base pay and incentives.

Published & Citable

Your evaluation receives a DOI and is published in The Unjournal (ISSN 3071-2173) on PubPub. Real scholarly credit for review work.

Prize Opportunities

Exceptional evaluations are eligible for our annual prizes—$6,500 awarded in 2024-25.

Signed or Anonymous

Choose whether to sign your evaluation publicly or remain anonymous. Your choice, every time.

Flexible Timeline

We work with your schedule. Most evaluations take 3-6 weeks, but we accommodate busy periods.

Impactful Research

Evaluate research that matters—work informing global health, development, policy, and effective interventions.

Compensation

~$450 average compensation per evaluation Base pay + incentives + potential prizes

Our compensation structure includes:

Base payment: $300+ per evaluation (varies by scope and complexity)
Incentive bonuses: Additional pay for thoroughness, timeliness, and quality
Annual prizes: $6,500 in prizes for exceptional evaluations (2024-25)

Prize recognition: Each year we recognize outstanding evaluations with monetary prizes and public acknowledgment. See our 2024-25 prize winners.

How It Works

Join Our Pool

Complete a simple form with your expertise areas and background. We'll add you to our evaluator database.

Receive Invitations

When we have research matching your expertise, we'll reach out with details and timeline. You can accept or decline.

Conduct Your Evaluation

Review the research using our structured guidelines. Provide written feedback and quantitative ratings across multiple dimensions.

Publication & Payment

Your evaluation is published on PubPub with a DOI. Payment is processed promptly after submission.

      What we ask of evaluators: Provide substantive, constructive feedback that helps authors improve their work. Rate research across dimensions like methodological rigor, real-world relevance, and contribution to the field. Be thorough but fair.
    

Evaluation Forms

Our evaluation process uses structured forms to ensure consistency and thoroughness:

Current form: Academic evaluation form (Coda) — the form evaluators currently use to submit evaluations
Preview draft: Interactive evaluation form prototype — a draft of our planned interface improvements (not yet in use)

AI Tools in Evaluation

The Unjournal supports evaluators using AI tools thoughtfully as a complement to, not a substitute for, expert human judgment. We are developing infrastructure to provide AI-generated reviews and consistency checks to evaluators after they submit, so they can compare perspectives and flag any important points they may have missed.

      Working policy (April 2026): Evaluators may use AI tools selectively — for example, as a lookup tool to clarify paper content — but should not use AI for overall evaluations or for ratings and predictions. All content must be independently verified and stood behind as the evaluator's own judgment. We ask evaluators to report how AI tools were used and to spend at least 8 hours on human work per evaluation.
    

Recent evidence: A large-scale survey of AAAI-26 authors and program committee members found that participants not only found AI reviews useful, but actually preferred them to human reviews on key dimensions such as technical accuracy and research suggestions (Biswas et al., 2025). This offers some promise for AI-assisted peer review, and informs how we are developing our own approach.

Full details: our AI/LLM use policy for evaluators.

Who We're Looking For

We seek evaluators with expertise in quantitative research methods and relevant subject areas. Ideal candidates have:

✓ PhD or equivalent research experience

✓ Expertise in economics, psychology, policy, or related quantitative fields

✓ Track record of published research

✓ Commitment to constructive, substantive feedback

✓ Interest in global priorities research

✓ Ability to assess methodological rigor

Focus Areas

We particularly need evaluators with expertise in:

Development economics and global health
Environmental economics and climate policy
Behavioral science and psychology
AI governance and technology policy
Animal welfare research
Meta-science and research methods
Catastrophic and existential risk

Join Our Evaluator Pool

Complete a short form to be added to our database. We'll reach out when we have research matching your expertise.

Apply to Evaluate Evaluator Guidelines

Learn more: Read our complete information for prospective evaluators and evaluation policies.