Skip to content
OpenTrain AIFor AI Companies

Applied AI Agent Evaluation Research Engineer

Apply now

Job details

Pay

$96.15–$125 USD/hr

Engagement

Contract

Location

Remote

52 countries

Posted

Nov 17, 2026

OpenTrain on Trustpilot

The work

About OpenTrain. OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and develop a durable freelance career. We help candidates discover high-quality AI training, evaluation, and data work without scattering their track record across disconnected sites.

About this role. OpenTrain is recruiting for an Applied AI Agent Evaluation Research Engineer to support AI training work focused on LLM agent evaluation and benchmarking. This role uses agent benchmarks, evaluation datasets, and task specifications to help improve AI systems through structured human expertise, careful review, and clear technical feedback. Prior AI training experience is not required unless stated elsewhere on this job page; strong real-world knowledge of the subject area is most important.

What you will do. You will design and implement evaluation frameworks, benchmarks, and datasets for LLM-powered agents and complex AI workflows. You will analyze model, agent, and workflow failures, identify root causes, run experiments, and measure improvements in reliability and effectiveness. You will develop training and evaluation data, task specifications, metrics, and feedback mechanisms, while building Python-based evaluation pipelines and research tools. You will collaborate with researchers, engineers, domain experts, and customer stakeholders to define evaluation strategies and technical improvement plans.

Required skills. Candidates should have a strong background in applied AI research, research engineering, machine learning, or LLM-based systems. The role requires hands-on experience evaluating AI agents, including tool use, multi-step workflows, and task completion, as well as strong Python programming skills. Candidates should understand benchmarks, datasets, evaluation metrics, LLM behavior, agentic architectures, reliability, robustness, and generalization. They should be able to translate ambiguous problems into structured experiments and communicate effectively across research, engineering, and project teams.

Helpful background. Experience evaluating or optimizing production or enterprise AI agents, building research infrastructure or automated experimentation frameworks, contributing to research publications or open-source benchmarks, or working in an applied AI research environment is valuable.

Why work with OpenTrain. OpenTrain gives freelancers one place to find AI training work, build a unified AI training portfolio, and use that work history to qualify for more opportunities over time. Specialized technical judgment can become a flexible and repeatable source of income.

You apply here in about two minutes and finish on the hiring platform. The next step is a short AI interview (technical roles include a coding challenge). Passing it is how you qualify for paid tasks, so take it soon after you are invited.

The hiring platform's application for this job asks for your resume, your LinkedIn profile, your phone number, your location and answers to 4 screening questions.

Requirements

  • Languages: English
  • Skills: AI Agent Evaluation & Benchmark Design, Python & Research Engineering, Agent Failure Analysis & Experimentation, LLM & Agentic Systems

Open to applicants in

  • Algeria
  • Argentina
  • Austria
  • Bahrain
  • Bangladesh
  • Belgium
  • Brazil
  • Canada
  • Chile
  • Colombia
  • Denmark
  • Egypt
  • France
  • Germany
  • Ghana
  • Greece
  • India
  • Indonesia
  • Italy
  • Japan
  • Jordan
  • Kazakhstan
  • Kenya
  • Kuwait
  • Kyrgyzstan
  • Lebanon
  • Malaysia
  • Mexico
  • Morocco
  • Netherlands
  • Nigeria
  • Oman
  • Pakistan
  • Palestine
  • Peru
  • Philippines
  • Portugal
  • Qatar
  • Saudi Arabia
  • Singapore
  • South Africa
  • Spain
  • Sri Lanka
  • Switzerland
  • Taiwan
  • Thailand
  • Tunisia
  • United Arab Emirates
  • United Kingdom
  • United States
  • Uzbekistan
  • Vietnam

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmwkasv3n00ea08l6v73yulm6

Keep exploring

Similar open roles

View all AI training jobs

Member of Technical Staff, Frontier AI Evaluation Lead

Lead model evaluation, failure analysis, and ML data systems remotely for 20+ hours per week, earning $115.38 to $168.27 per hour. Apply through OpenTrain to begin.

Research Signal JudgmentML-Oriented Data DesignOps-to-Research TranslationRL Environments
Generative AI & RLHF
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $115.38–$168.27/hr

Posted Nov 19, 2026

Coding Agent Evaluation Researcher

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

LLMsCoding EvaluationAI EvaluationML Systems
Code
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $86.54–$125/hr

Posted Nov 18, 2026

Coding Agent Evaluation Researcher

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

LLMsCoding EvaluationAI EvaluationML Systems
Code
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $96.15–$125/hr

Posted Nov 18, 2026

AI Agent Evaluator

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

ChatGPTMuseAI agentsAttention to DetailFollowing InstructionsScreen Recording
Video
Remote · United States
English
Part-time · Flexible
Hourly · $25–$45/hr

Posted Nov 16, 2026

AI Agent Tester for Real-World Task Evaluation

About OpenTrain. OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

ChatGPTMuseAI agentsAttention to DetailFollowing InstructionsScreen Recording
Video
Remote · United States
English
Part-time · Flexible
Hourly · $25–$45/hr

Posted Oct 24, 2026