Skip to content
OpenTrain AIFor AI Companies

STEM Rubric Design Specialist

Apply now

Job details

Pay

$100 USD/hr

Engagement

Contract, part-time

Location

Remote

United States only

Posted

Sep 30, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting for a STEM Rubric Design Specialist to create expert-level technical questions, evaluate AI-generated solutions, and design detailed evaluation rubrics for frontier model training. In this hourly, remote contractor position, you will apply your deep technical expertise to probe the reasoning boundaries of advanced AI systems and improve their accuracy and rigor. This is a role for experienced scientists, engineers, and mathematicians who enjoy translating complex technical judgment into measurable criteria.

What you will do
- Create expert-level technical questions that test genuine multi-step reasoning and expose weaknesses in advanced AI systems
- Design and write structured evaluation rubrics defining required reasoning, correct methodology, acceptable alternatives, partial-credit criteria, and critical errors
- Evaluate AI-generated solutions for correctness, reasoning quality, completeness, methodology, and technical precision
- Identify hidden reasoning errors, including cases where an answer reaches the correct conclusion via invalid assumptions or flawed logic
- Critique peer rubrics for technical accuracy, clarity, completeness, and scoring consistency
- Develop reference content including expert answers, explanations, and gold-standard evaluations for RL and SFT workflows

Required skills
- Bachelor’s degree or higher in Mathematics, Physics, Chemistry, Engineering, Computer Science, Statistics, Applied Science, or another technical discipline
- Strong professional proficiency in English, minimum C1, with the ability to write precise technical explanations and evaluation criteria
- 3+ years of professional, academic, research, or industry experience in your stated technical domain
- Demonstrated advanced expertise in a clearly defined technical field or specialization
- Strong ability to design detailed evaluation rubrics defining required reasoning, correct methodology, acceptable alternatives, partial-credit criteria, and critical errors
- Ability to distinguish between a correct final answer and an answer supported by valid versus flawed reasoning
- Comfortable reviewing and critiquing other experts’ rubrics for ambiguity, missing criteria, redundancy, or technical inaccuracies

Helpful background
- Experience with exam writing, academic grading, peer review, research review, technical QA, standards development, or assessment design is strongly preferred
- Prior experience with AI evaluation, RLHF, SFT, benchmarking, data annotation, prompt design, or LLM evaluation is preferred

Requirements

  • Languages: English

Open to applicants in

  • United States

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmuns5yhc00060agmiw6iwvf9

Keep exploring

Similar open roles

View all AI training jobs

High School Teacher Expert

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Text
Remote · United Arab Emirates, Antigua and Barbuda, Albania +92 more
English
Part-time · Flexible
Hourly · $55/hr

Posted Nov 8, 2026

Learning and Assessment Specialist

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

instructional designerseducators
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $90–$130/hr

Posted Oct 15, 2026

Legal AI Evaluation and Rubric Design Expert

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

AI evaluationComplianceContractsCorporate lawEnglishExpert review
Text
Remote · United States
English
Part-time · Flexible
Hourly · $120/hr

Posted Sep 30, 2026

Healthcare AI Rubric Evaluation Specialist

About OpenTrain. OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

AI evaluationClinical pharmacologyClinical reasoningDentistryEmergency medicineEnglish
Text
Remote · United States
English
Part-time · Flexible
Hourly · $100/hr

Posted Sep 30, 2026

Retail Merchandising Benchmark Evaluation Specialist

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Customer Interaction
Text
Remote · Worldwide
English
Part-time · Flexible

Posted Oct 27, 2026