Physics Reasoning Evaluator — Expert AI Response Reviewer

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We connect specialists with projects that shape how state-of-the-art AI systems learn from human expertise.

This role is posted through OpenTrain’s platform — create a free account to apply, manage your profile, and see paid onboarding tasks.

Why Work in AI Training

AI training (data labeling and human-feedback work) is the human foundation of modern AI. Contributors provide the examples, ratings, and corrections that guide model behavior.

Work is typically fully remote, flexible, and accessible: many projects require only discipline-specific knowledge and attention to detail. As a specialist you’ll directly influence how physics reasoning is represented in AI systems.

• 100% remote: do this work from anywhere with a computer and internet.
• Flexible scheduling: work part-time and fit tasks around your life.
• Cutting edge: help shape how AI solves and explains physics problems.

The Role

We’re hiring physics specialists to evaluate AI-generated physics responses and author clear, rigorous exemplar solutions. This is a contractor, part-time role focused on high-quality assessment and explanation.

You will judge correctness, reasoning depth, clarity, and methodological rigor; identify subtle conceptual and computational errors; and compose step-by-step model answers that the team can use as training references.

• Position type: Contractor / Part-time.
• Work format: Remote; review and rating tasks delivered through labeling software.
• Label type: Evaluation/Rating of model responses (EVALUATION_RATING).
• Data type: Video (some tasks may include video-based model outputs).

What You’ll Do Day-to-Day

Evaluate and compare multiple AI-generated solutions using detailed rubrics, flagging errors in assumptions, derivations, units, or methodology.

Author exemplar solutions and explanations that are rigorous and pedagogically clear. Fact-check physics claims using reputable public sources and include precise references when needed.

• Apply evaluation rubrics consistently to rate correctness, depth, clarity, and reproducibility.
• Spot subtle conceptual, methodological, and computational mistakes in derivations and numerical work.
• Write step-by-step solutions in clear C1+ English; include correct notation and, when appropriate, LaTeX-formatted math.
• Provide concise, evidence-backed fact-checks and references for contentious claims.

Requirements (Must-Have)

This role is for physics specialists — not generalists. All must-have qualifications below are required to be considered.

Candidates who cannot demonstrate the listed items should not apply.

• BS, MS, or PhD in Physics (or closely related physics discipline) from a top‑100 university (completed or in-progress).
• Mastery across core physics areas: classical mechanics, electromagnetism, thermodynamics, quantum mechanics, and statistical physics; familiarity with relativity is a plus.
• Strong quantitative reasoning: dimensional analysis, unit consistency, and awareness of uncertainty and approximation.
• Excellent scientific writing in clear C1+ English; ability to produce step-wise explanations with correct notation; LaTeX proficiency preferred.
• High attention to detail and ability to apply detailed evaluation rubrics consistently.

Preferred & Bonus Qualifications

These are not strictly required but will make you more competitive for the role and for higher-volume projects.

• Research experience or history of analytical scientific writing or debate.
• Programming literacy (Python, Matlab, or similar) for checking calculations or producing reproducible examples.
• Prior experience with data labeling, RLHF, or model evaluation is a bonus.

Time, Pay & Onboarding

Compensation is $80 USD per hour (pay-per-hour contract). Typical commitment is under 20 hours/week, but applicants must meet the role’s minimum availability requirement.

Minimum availability: roughly 17–20 hours per week; preferred cadence is about 8 hours per day during active sprints.

• Paid onboarding includes a 1–2 hour qualification exam and a paid 1–2 hour project exam required before full access to tasks.
• Employment types: contractor, part-time. Worldwide applicants accepted.

How To Apply

Apply through your OpenTrain account. Your application should demonstrate your degree (or in-progress status) at a top‑100 university and highlight relevant physics coursework, research, or publications.

Be prepared to complete the paid qualification and project exams as part of onboarding.

• Include examples of prior technical writing or solutions if available (papers, graded solutions, or teaching materials).
• List any programming experience and prior model-evaluation or labeling work.