Skip to content
OpenTrain AIFor AI Companies

Member of Technical Staff, Frontier AI Evaluation Lead

Apply now

Job details

Pay

$115.38–$168.27 USD/hr

Engagement

Contract

Location

Remote

52 countries

Posted

Nov 19, 2026

OpenTrain on Trustpilot

The work

  • Own research and evaluation initiatives from problem framing through data design, quality calibration, and signal validation.
  • Design ML-oriented data systems with task definitions, annotation schemas, rubrics, incentives, and pipelines tied to model performance.
  • Analyze model and system failures to find root causes, edge cases, and opportunities to improve performance.
  • Turn unclear system behavior into evaluation frameworks and new data categories.
  • Work with researchers and domain experts to calibrate quality and strengthen evaluation signals.
  • Iterate on evaluations, datasets, and feedback loops to improve model or agent performance.
  • Make evidence-based quality decisions when data integrity or signal strength is not sufficient.
  • Find gaps in evaluation coverage and recommend where to invest, iterate, or stop.
  • Collaborate with researchers, domain experts, operators, and cross-functional teams to improve evaluation quality and deployed AI systems.

Requirements

  • Time: 20+ hours per week.
  • Language: Professional fluency in English.
  • Experience designing ML-oriented datasets, evaluation frameworks, and quality assurance processes; strong judgment about research signal quality and evidence readiness; the ability to structure messy real-world system behavior into research and evaluation opportunities, explain tradeoffs and limitations clearly, work directly with experts during calibration and iteration, and take a systems-level view of end-to-end model or agent performance.
  • Skills: Research Signal Judgment, ML-Oriented Data Design, Ops-to-Research Translation, RL Environments

Open to applicants in

  • Algeria
  • Argentina
  • Austria
  • Bahrain
  • Bangladesh
  • Belgium
  • Brazil
  • Canada
  • Chile
  • Colombia
  • Denmark
  • Egypt
  • France
  • Germany
  • Ghana
  • Greece
  • India
  • Indonesia
  • Italy
  • Japan
  • Jordan
  • Kazakhstan
  • Kenya
  • Kuwait
  • Kyrgyzstan
  • Lebanon
  • Malaysia
  • Mexico
  • Morocco
  • Netherlands
  • Nigeria
  • Oman
  • Pakistan
  • Palestine
  • Peru
  • Philippines
  • Portugal
  • Qatar
  • Saudi Arabia
  • Singapore
  • South Africa
  • Spain
  • Sri Lanka
  • Switzerland
  • Taiwan
  • Thailand
  • Tunisia
  • United Arab Emirates
  • United Kingdom
  • United States
  • Uzbekistan
  • Vietnam

Preferred

  • Helpful background includes reinforcement learning environments, simulators, feedback-driven training systems, agentic systems, applied research, production systems, complex-task evaluation, or expert incentive design.

How to apply

  1. Press Apply now. You create a free OpenTrain account and add your resume.
  2. Finish your application on the hiring company's site. The company decides who it hires, so it runs its own form. It asks for your resume again, your LinkedIn profile, your phone number, your location and answers to 4 screening questions.
  3. Take a short AI interview.

OpenTrain on Trustpilot

Job ID: cmwo0bt7z000s07lcwbutofha

Keep exploring

Similar open roles

View all AI training jobs

AI Model Response Evaluator

Review AI-generated text, images, and other outputs, rate response quality, and suggest improvements. This contractor role pays $3 to $6 per hour and is open in listed countries.

Generative AI & RLHF
Text
Remote · Afghanistan, Albania, Algeria +220 more
Flexible hours
Intermediate level
Hourly · $3–$6/hr

Posted Nov 20, 2026

Music Teacher - AI Training

Help improve AI-generated music by reviewing musical content, creating examples, and explaining music theory. This remote contractor role offers flexible scheduling and pays $15 to $45 per hour.

Chord ProgressionKey ProgressionsMusic TeachingComposingInstrument Proficiency
Generative AI & RLHF
Audio
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $15–$45/hr

Posted Nov 20, 2026

Airline Customer Service Specialist (English)

Use airline customer service expertise to write realistic support scenarios, review AI responses, and explain procedures through written and voice work. Pay is $18 to $28 per hour, with applications through OpenTrain.

airline customer serviceflight reservation managementbaggage issue resolutioncustomer complaint resolutionverbal and written communicationempathy
Generative AI & RLHF
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $18–$28/hr

Posted Nov 20, 2026

Telecommunications Customer Support Specialist (English)

Use your telecommunications support experience to write answers, review AI dialogue, and improve troubleshooting guidance remotely for $18-$28 per hour. Apply through OpenTrain, then complete the hiring site application.

telecommunications customer supporttroubleshootingwritten communicationverbal communicationhelp documentation creationanalytical thinking
Generative AI & RLHF
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $18–$28/hr

Posted Nov 20, 2026

Retail Customer Support Specialist (English)

Evaluate AI-generated retail support replies, create realistic order and returns scenarios, and support voice data work. This remote contractor role pays $18 to $28 per hour and requires professional retail support experience plus fluent English.

retail customer supportorder and return process knowledgecomplaint resolutiondata annotationdocumentationwritten communication
Generative AI & RLHF
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $18–$28/hr

Posted Nov 20, 2026