Skip to content
OpenTrain AIFor AI Companies

Member of Technical Staff, Frontier AI Evaluation Lead

Apply now

Job details

Pay

$115.38–$168.27 USD/hr

Engagement

Contract

Location

Remote

52 countries

Posted

Nov 19, 2026

OpenTrain on Trustpilot

The work

  • Own research and evaluation initiatives from problem framing through data design, quality calibration, and signal validation.
  • Design ML-oriented data systems with task definitions, annotation schemas, rubrics, incentives, and pipelines tied to model performance.
  • Analyze model and system failures to find root causes, edge cases, and opportunities to improve performance.
  • Turn unclear system behavior into evaluation frameworks and new data categories.
  • Work with researchers and domain experts to calibrate quality and strengthen evaluation signals.
  • Iterate on evaluations, datasets, and feedback loops to improve model or agent performance.
  • Make evidence-based quality decisions when data integrity or signal strength is not sufficient.
  • Find gaps in evaluation coverage and recommend where to invest, iterate, or stop.
  • Collaborate with researchers, domain experts, operators, and cross-functional teams to improve evaluation quality and deployed AI systems.

Requirements

  • Time: 20+ hours per week.
  • Language: Professional fluency in English.
  • Experience designing ML-oriented datasets, evaluation frameworks, and quality assurance processes; strong judgment about research signal quality and evidence readiness; the ability to structure messy real-world system behavior into research and evaluation opportunities, explain tradeoffs and limitations clearly, work directly with experts during calibration and iteration, and take a systems-level view of end-to-end model or agent performance.
  • Skills: Research Signal Judgment, ML-Oriented Data Design, Ops-to-Research Translation, RL Environments

Open to applicants in

  • Algeria
  • Argentina
  • Austria
  • Bahrain
  • Bangladesh
  • Belgium
  • Brazil
  • Canada
  • Chile
  • Colombia
  • Denmark
  • Egypt
  • France
  • Germany
  • Ghana
  • Greece
  • India
  • Indonesia
  • Italy
  • Japan
  • Jordan
  • Kazakhstan
  • Kenya
  • Kuwait
  • Kyrgyzstan
  • Lebanon
  • Malaysia
  • Mexico
  • Morocco
  • Netherlands
  • Nigeria
  • Oman
  • Pakistan
  • Palestine
  • Peru
  • Philippines
  • Portugal
  • Qatar
  • Saudi Arabia
  • Singapore
  • South Africa
  • Spain
  • Sri Lanka
  • Switzerland
  • Taiwan
  • Thailand
  • Tunisia
  • United Arab Emirates
  • United Kingdom
  • United States
  • Uzbekistan
  • Vietnam

Preferred

  • Helpful background includes reinforcement learning environments, simulators, feedback-driven training systems, agentic systems, applied research, production systems, complex-task evaluation, or expert incentive design.

How to apply

  1. Press Apply now. You create a free OpenTrain account and add your resume.
  2. Finish your application on the hiring company's site. The company decides who it hires, so it runs its own form. It asks for your resume again, your LinkedIn profile, your phone number, your location and answers to 4 screening questions.
  3. Take a short AI interview.

OpenTrain on Trustpilot

Job ID: cmwo0bt7z000s07lcwbutofha

Keep exploring

Similar open roles

View all AI training jobs

Biology and Life Sciences AI Evaluation Expert

Create and assess difficult biology content used to test and improve large language models. This remote, project-based role is open in the US and requires 20+ hours weekly, advanced biology knowledge, and clear scientific writing.

Generative AI & RLHF
Text
Remote · United States
English
Part-time · Flexible

Posted Nov 21, 2026

Generalist (Japan) for E-Commerce AI Testing

Test AI-powered shopping features through real transactions on Japanese e-commerce platforms. Place, track, and cancel orders, review AI outputs, and give written and verbal feedback for $30 to $49 per hour.

GeneralistTesterEnglish communication
Generative AI & RLHF
Text
Remote · Japan
Japanese, English
Part-time · Flexible
Hourly · $30–$49/hr

Posted Nov 21, 2026

Generalist (France) AI E-Commerce Testing

Test e-commerce AI in France by placing, tracking, and canceling orders, reviewing model outputs, and writing clear feedback. The role pays $30 to $49 per hour and requires fluent French, strong English, and local purchase-task equipment.

GeneralistTesterEnglish communication
Generative AI & RLHF
Text
Remote · France
French, English
Part-time · Flexible
Hourly · $30–$49/hr

Posted Nov 21, 2026

E-commerce AI Testing Generalist (German, Germany)

Test German e-commerce transactions and review AI outputs for accuracy and cultural relevance. This part-time contractor role pays $30 to $49 per hour and requires fluent German, strong English, and residence in Germany.

GeneralistTesterEnglish communication
Generative AI & RLHF
Text
Remote · Germany
German, English
Part-time · Flexible
Hourly · $30–$49/hr

Posted Nov 21, 2026

Generalist (Brazil) E-Commerce AI Testing Specialist

Test Brazilian e-commerce workflows, review AI outputs, and document clear feedback for $30 to $49 per hour. This part-time contractor role is open to candidates in Brazil and requires fluent Portuguese, strong English, and a computer.

GeneralistTesterEnglish communication
Generative AI & RLHF
Text
Remote · Brazil
Portuguese, English
Part-time · Flexible
Hourly · $30–$49/hr

Posted Nov 21, 2026