Skip to content
OpenTrain AIFor AI Companies

Senior Software Engineer AI Evaluation Coding Agents

Apply now

Job details

Pay

$100–$150 USD/hr

Engagement

Contract, part-time

Location

Remote

2 countries

Posted

Sep 24, 2026

OpenTrain on Trustpilot

The work

You will evaluate AI-generated code and coding-agent behavior using real-world software repositories. Your reviews will help AI systems improve through structured human judgments and clear technical feedback.

  • Review model outputs and source materials against project guidelines.
  • Apply evaluation rubrics and make evidence-based judgments.
  • Explain why an implementation is correct or incorrect.
  • Document technical decisions in clear written feedback.
  • Help maintain consistent quality across repeated review tasks.
  • Support evaluation processes for AI coding agents.

Requirements

The work requires strong real-world software engineering knowledge, but prior AI training experience is not required.

  • Time: 20 or more hours per week.
  • Language: Professional fluency in English.
  • Experience: At least 5 years of hands-on software engineering experience.
  • Programming: Strong proficiency in Python, TypeScript or JavaScript, Go, or another major production language.
  • Codebases: Experience working in substantial real-world codebases.
  • Core skills: Strong code review, technical judgment, and written communication.
  • Technical knowledge: Familiarity with modern large language models or AI coding tools.
  • Skills: Python, Typescript, JavaScript, Go, Java, GitHub, CI/CD, Open Source, LLM, Code Reviews, Code Analysis

Open to applicants in

  • Canada
  • United States

Preferred

  • Helpful experience: LLM evaluation, coding agents, RLHF, preference data, rubric design, or post-training.

How to apply

  1. Press Apply now. You create a free OpenTrain account and add your resume.
  2. Finish your application on the hiring company's site. The company decides who it hires, so it runs its own form.

Job ID: cmufdsdgd000o0agmt0v7bcxs

Keep exploring

Similar open roles

View all AI training jobs

Senior Software Engineer DevOps

Build AWS automation, CI/CD pipelines, and security controls for LLM evaluation work. This US-based remote contract offers 20+ hours per week and pays $25 per label.

DevOps
Coding & Software
Computer Code Programming
Remote · United States
English
Part-time · Flexible
Per task · $25/label

Posted Nov 14, 2026

Junior Machine Learning Engineer

Complete coding-heavy AI training tasks involving machine learning models, Python, evaluation systems, and performance optimization. Work remotely for about 15 hours per week, with $60 to $120 per hour equivalent, paid per accepted task.

Machine Learning
Coding & Software
Computer Code Programming
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Per task · $60–$120/hr equivalent

Posted Nov 12, 2026

Software Engineer, AI Coding Task Creator

Create realistic bugs, features, refactoring, and performance tasks that test AI coding ability. This part-time contractor role pays $50 to $100 per hour equivalent, paid per task that meets project specifications.

Python3JAVARustAlgorithmsBasics C++Typescript
Coding & Software
Computer Code Programming
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Per task · $50–$100/hr equivalent

Posted Nov 11, 2026

Game Developer (Godot Expert)

Work remotely on AI training projects involving Godot game development, gameplay systems, and technical reviews. Earn $70 to $130 per hour with flexible weekly hours over 1 to 3 months.

GodotGame Development
Coding & Software
Computer Code Programming
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $70–$130/hr

Posted Nov 11, 2026

Backend Software Engineer for AI Coding Tasks

Evaluate AI coding tasks using real repository code, technical rubrics, and clear written feedback. The role pays $50 to $100 per hour equivalent, paid per task that meets the project specifications, for about 15 hours per week.

Python3JAVARustAlgorithmsBasics C++Typescript
Coding & Software
Computer Code Programming
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Per task · $50–$100/hr equivalent

Posted Nov 11, 2026