Skip to content
OpenTrain AIFor AI Companies

Coding Agent Evaluation Researcher

Apply now

Job details

Pay

$86.54–$125 USD/hr

Engagement

Contract, part-time

Location

Remote

52 countries

Posted

Nov 18, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting for a Coding Agent Evaluation Researcher to design and advance evaluation systems for frontier coding agents. This role combines software engineering, AI research, model evaluation, benchmark development, and dataset creation to measure coding-system performance and identify ways to improve it.

What you will do
Design evaluation frameworks with benchmark specifications, scoring methodologies, rubrics, and quality standards. Lead research initiatives that measure coding-model performance across software engineering tasks. Develop datasets, golden examples, and evaluation protocols for reliable assessment. Investigate model behavior and failure modes, translating findings into improvements for training and evaluation. Build tooling and infrastructure for experimentation, data generation, review workflows, and evaluation pipelines. Establish rigorous, reproducible practices for coding-agent assessment. Design experiments with research and engineering collaborators and communicate results through technical reports and benchmark studies.

Required skills
Strong software engineering experience with Python, C++, or comparable programming languages. At least three years of experience in software engineering, machine learning, AI research, evaluation, or a related technical discipline. Experience designing, reviewing, or validating technical assessments, benchmarks, coding tasks, or evaluation methodologies. Familiarity with large language models, coding agents, reinforcement learning, model evaluation, or related AI systems. Ability to build tooling, automate workflows, investigate model behavior, and derive insights from complex technical systems. Strong written and verbal communication skills, including the ability to explain technical findings to diverse audiences.

Helpful background
Experience with frontier AI systems, coding-agent research, or large-scale benchmark and dataset design is helpful. Familiarity with agentic workflows, tool use, reinforcement learning, or post-training methods is also valuable, as are publications, open-source contributions, or demonstrated technical leadership in AI, machine learning, or software engineering.

Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.

You apply here in about two minutes and finish on the hiring platform. The next step is a short AI interview (technical roles include a coding challenge). Passing it is how you qualify for paid tasks, so take it soon after you are invited.

The hiring platform's application for this job asks for your resume, your LinkedIn profile, your phone number, your location and answers to 2 screening questions.

Requirements

  • Languages: English
  • Skills: LLMs, Coding Evaluation, AI Evaluation, ML Systems

Open to applicants in

  • Algeria
  • Argentina
  • Austria
  • Bahrain
  • Bangladesh
  • Belgium
  • Brazil
  • Canada
  • Chile
  • Colombia
  • Denmark
  • Egypt
  • France
  • Germany
  • Ghana
  • Greece
  • India
  • Indonesia
  • Italy
  • Japan
  • Jordan
  • Kazakhstan
  • Kenya
  • Kuwait
  • Kyrgyzstan
  • Lebanon
  • Malaysia
  • Mexico
  • Morocco
  • Netherlands
  • Nigeria
  • Oman
  • Pakistan
  • Palestine
  • Peru
  • Philippines
  • Portugal
  • Qatar
  • Saudi Arabia
  • Singapore
  • South Africa
  • Spain
  • Sri Lanka
  • Switzerland
  • Taiwan
  • Thailand
  • Tunisia
  • United Arab Emirates
  • United Kingdom
  • United States
  • Uzbekistan
  • Vietnam

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmwm7578m00a108jmhckhttan

Keep exploring

Similar open roles

View all AI training jobs

Coding Agent Evaluation Researcher

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

LLMsCoding EvaluationAI EvaluationML Systems
Code
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $96.15–$125/hr

Posted Nov 18, 2026

Applied AI Agent Evaluation Research Engineer

About OpenTrain. OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and develop a durable freelance career. We help ca…

AI Agent Evaluation & Benchmark DesignPython & Research EngineeringAgent Failure Analysis & ExperimentationLLM & Agentic Systems
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $96.15–$125/hr

Posted Nov 17, 2026

Member of Technical Staff, Coding Agent Research and Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Coding Agent Evaluation & Benchmark DesignSoftware Engineering & PythonLLM & Agentic SystemsModel Failure Analysis & Experimentation
Code
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $96.15–$125/hr

Posted Nov 17, 2026

Clinical Researcher AI Evaluation Specialist

Review AI-generated clinical research and biomedical content, rate model responses, and write expert explanations. This remote contractor project offers up to $40 to $70 per hour, requires 20+ hours weekly, and needs 5+ years of clinical research experience.

BiasBias & confoundingBiostatisticsClinical trial designConfoundingData interpretation
Generative AI & RLHF
Text
Remote · Bangladesh, Bhutan, Brazil +14 more
English
Part-time · Flexible
Intermediate level
Hourly · $40–$70/hr

Posted Jul 8, 2026

Python Coding-Agent Benchmark Expert

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Python
Code
Remote · Worldwide
English
Part-time · Flexible

Posted Nov 18, 2026