Physics LLM Evaluation Expert

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We help people start and grow careers teaching AI: discover projects, build a lasting portfolio, and apply in minutes. Creating an OpenTrain account is free.

We run and manage the contractor relationships for the projects we post. For this role you’ll work directly with OpenTrain as a remote contractor on physics-focused model evaluation tasks.

About AI training work

AI training (data labeling / annotation / human feedback) is the human side of building modern AI systems. Contributors prepare and review examples that teach models how to reason, explain, and solve problems.

This work is highly flexible, accessible to many backgrounds, and directly shapes how state-of-the-art models behave. Many contributors work remotely and on part-time schedules that fit around other commitments.

The role

You will design challenging physics problems and produce clear, step-by-step solutions to evaluate how large language models perform on abstraction, multi-step reasoning, and symbolic manipulation. Problems will span undergraduate through PhD-level topics.

This is a contractor, part-time assignment expected to require 20+ hours per week. Work is fully remote and conducted in English; contributors collaborate with OpenTrain researchers to align problems with evaluation goals.

What you'll do

• Design physics problems that target abstraction, multi-step reasoning, and symbolic manipulation in LLMs across classical mechanics, E&M & optics, and thermodynamics/statistical physics.
• Write high-quality, step-by-step solutions that show all reasoning and make complex concepts clear and teachable.
• Create evaluation items and rubrics that map to specific learning objectives and model-behavior tests.
• Collaborate with LLM researchers to align problems with evaluation goals and iterate on item difficulty and scoring.
• Help define benchmarks and organize items by topic, difficulty, and expected solution structure.
• Produce text-based evaluation artifacts (prompts, solutions, rating guides) and provide constructive feedback on model outputs.

Requirements

• Advanced physics problem-solving ability across classical mechanics, electromagnetism & optics, or thermodynamics and statistical physics.
• Graduate-level physics training preferred (MSc, PhD, or postdoctoral experience is ideal).
• Proven ability to write clear, structured step-by-step explanations for complex solutions in English.
• Experience probing model limitations with multi-step reasoning and symbolic manipulation tasks (designing or evaluating such items).
• Strong analytical and research skills, excellent written English comprehension, and structured communication.
• Self-motivated remote work habits; able to collaborate independently and meet deadlines.
• Availability for roughly 20+ hours per week in a contractor, part-time arrangement.

Helpful background

• Experience solving advanced physics problems at the graduate or contest level (e.g., engineering entrance, Olympiad-style, or graduate coursework).
• Familiarity with designing evaluation benchmarks or test items for academic or technical assessments.
• Comfort creating simple visuals or describing diagrams in text to clarify solution steps.
• Previous remote contractor or academic collaboration experience is a plus.

How it works

OpenTrain manages contracting and onboarding. You’ll apply with your OpenTrain profile (free to create) and any supporting examples of problems/solutions. Selection focuses on demonstrated physics reasoning and clear written explanations.

This role works with text data and primarily produces evaluation ratings and text-generation artifacts. Compensation and exact scheduling are set per contract; this listing expects a 20+ hour weekly commitment.

• Data type: text; label types: evaluation rating and text generation.
• Employment types: contractor, part-time; remote work from anywhere with reliable internet.
• Language required: English.