Skip to content
OpenTrain AIFor AI Companies

AI Code Evaluation and Benchmarking Engineer

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

United States only

Posted

Jul 17, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting for an AI Code Evaluation and Benchmarking Engineer to help assess and improve AI-generated software. You will review code for correctness, efficiency, maintainability, and compliance with requirements, then validate proposed solutions against real-world engineering tasks. The work supports human feedback and evaluation data for systems that generate and reason about code. You will also examine model-generated explanations and implementation approaches, investigate technical failure modes, and help maintain reliable coding benchmarks and grading standards.

What you will do
Review AI-generated code and determine whether it satisfies functional and technical requirements. Analyze software engineering tasks and validate whether proposed solutions meet expected outcomes. Debug code, reproduce issues, and verify fixes across different programming environments. Assess model-generated explanations, reasoning, and implementation approaches for technical accuracy. Create, refine, and maintain evaluation datasets, benchmarks, and grading rubrics for coding tasks. Identify edge cases and failure modes where AI systems struggle with software engineering problems. Document technical findings and provide structured feedback that supports consistent evaluations. Collaborate with project teams to establish quality standards and evaluation methodologies.

Required skills
A bachelor's or master's degree in computer science, software engineering, or a related technical field is required, along with at least three years of professional software engineering experience. You should be proficient in one or more of Python, Java, C/C++, Go, Swift, Objective-C, PHP, or SQL, with strong knowledge of data structures, algorithms, software design principles, and debugging methodologies. Experience conducting code reviews and evaluating quality in production or large-scale codebases is important. Familiarity with Git or comparable version control systems and modern software development workflows is expected. Strong written communication, careful attention to detail, and the ability to assess complex technical solutions with minimal supervision are also required.

Helpful background
Experience with AI or machine learning data annotation, natural language processing, prompt engineering, model evaluation, large language model projects, AI-generated code evaluation, benchmark creation, or software quality assessment is valuable.

Schedule and engagement
This is a remote contractor assignment for freelancers located in the United States. The contract is expected to last one month and requires at least 20 hours per week, including at least four hours per day and four hours of overlap with Pacific Time. Compensation details are not disclosed.

Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.

Requirements

  • Languages: English

Open to applicants in

  • United States

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmroshmk5000809kk93lh8phu

Keep exploring

Similar open roles

View all AI training jobs

Coding Agent Evaluation Researcher

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

LLMsCoding EvaluationAI EvaluationML Systems
Code
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $86.54–$125/hr

Posted Nov 18, 2026

Coding Agent Evaluation Researcher

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

LLMsCoding EvaluationAI EvaluationML Systems
Code
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $96.15–$125/hr

Posted Nov 18, 2026

Member of Technical Staff, Coding Agent Research and Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Coding Agent Evaluation & Benchmark DesignSoftware Engineering & PythonLLM & Agentic SystemsModel Failure Analysis & Experimentation
Code
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $96.15–$125/hr

Posted Nov 17, 2026

Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Backend Engineer for AI Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026