Skip to content
OpenTrain AIFor AI Companies

AI Engineer for Software Engineering Evaluation

Help build reinforcement learning environments that evaluate AI agents on complex software engineering tasks using MCP tools. This flexible remote contract offers approximately 15 hours per week at $60–$120/hour.

OpenTrain AI

Coding & Software

Remote Hourly · $60–$120/hr

$60–$120/hr

Compensation

16 countries

Eligibility

Intermediate

Experience

Aug 25, 2026

Posted

Open to applicants in

United Kingdom United States United Arab Emirates Ireland India South Korea Japan Finland Mexico Brazil Australia Austria Canada Belgium Egypt France

Interested in this role?

Create a free OpenTrain account and apply in minutes.

About OpenTrain

OpenTrain AI is the hiring and contracting organization for this role and the #1 platform for finding and building careers in AI training and data labeling. OpenTrain helps people discover cutting-edge projects, build their AI work profile, and apply in minutes. Creating an OpenTrain account is free.

About AI Training and Evaluation

AI training is the human side of building modern artificial intelligence. Engineers, researchers, and other contributors create examples, test model behavior, and provide feedback that helps AI systems reason, use tools, and produce more reliable results.

This project focuses on evaluating AI agents through realistic software engineering tasks. Your technical judgment will help measure how well models discover information, solve problems, use development tools, and implement dependable software.

The AI Engineer Role

OpenTrain AI is seeking experienced AI Engineers and software engineers for next-generation AI training and evaluation projects. You will help create reinforcement learning environments that test AI models on complex software engineering tasks using Model Context Protocol tools.

This is a remote contractor opportunity for approximately 15 hours per week, with a flexible schedule that may include weekends. The expected project duration is 1 to 3 months, and urgent hiring is underway.

  • Compensation: $60–$120 per hour
  • Contract, part-time engagement
  • Workload: approximately 15 hours per week and under 20 hours weekly
  • Remote work with flexible days and hours
  • Output-based payment for completed tasks meeting project specifications and quality requirements
  • Minimum weekly submission requirements may apply

What You'll Do

You will contribute practical software engineering expertise to development environments and evaluation systems designed for AI agents. Individual assignments may vary according to project needs, experience, and workflow.

  • Fix bugs and debug complex technical issues
  • Implement new features and develop maintainable software solutions
  • Refactor existing codebases
  • Optimize software performance and scalability
  • Design reproducible development environments
  • Create deterministic verification systems
  • Develop golden reference solutions for AI evaluation
  • Evaluate software engineering ability and effective MCP tool usage

Required Skills and Qualifications

Strong software engineering expertise and practical programming experience are the primary requirements. No prior professional AI experience is required, but you should be comfortable working from technical specifications and producing reliable, maintainable code.

The role requires fluent English and experience with computer code programming labeling workflows, AI software engineering training, and evaluation subject matter. Candidates should be available to collaborate effectively in a remote environment.

  • Strong proficiency in C++, Python, Java, Go, TypeScript, or Rust
  • Python 3, Java, Rust, C++ fundamentals, and TypeScript experience
  • Deep understanding of algorithms, data structures, and performance tuning
  • Demonstrated experience debugging complex software issues
  • Strong experience with feature development and codebase refactoring
  • Proven ability to optimize software for performance and scalability
  • Excellent written and verbal communication skills
  • Strong attention to detail and ability to follow technical specifications
  • Intermediate experience level
  • Fluent English proficiency

Preferred Experience

The following experience is helpful but not required. Candidates may be considered based on the strength of their software engineering background and practical programming work.

  • Experience with large-scale or distributed codebases
  • Familiarity with AI or machine learning systems
  • Experience with modern developer tools and APIs
  • Participation in rigorous code reviews
  • Knowledge of software engineering best practices
  • Experience designing automated testing or verification systems
  • Familiarity with Model Context Protocol or similar tool-use frameworks

Location, Selection, and Start Timeline

Candidates must be based in one of the following countries: the United Kingdom, United States, United Arab Emirates, Ireland, India, South Korea, Japan, Finland, Mexico, Brazil, Australia, Austria, Canada, Belgium, Egypt, or France. The working language is English.

The selection process is designed to assess technical capability and project fit. Selected candidates should be prepared to begin their first tasks within 24–48 hours after completing onboarding.

  • Submit an application with your updated resume and availability for the interview
  • Complete screening questions
  • Complete an approximately 30-minute AI interview
  • Complete a technical assessment if required
  • Proceed through hiring manager review
  • Complete onboarding and receive a project assignment
  • Start as soon as possible if selected

Ready to apply?

Create a free OpenTrain account and apply for this role in minutes.

Keep exploring

Similar jobs

View all AI training jobs

MCP AI Software Evaluation Engineer

Evaluate AI agents on realistic software engineering tasks and build reliable MCP-based reinforcement learning environments. This flexible remote contractor role offers an ideal rate range of $60 to $120 per hour.

Coding & Software
Computer Code Programming
Remote · Worldwide
English
Part-time · Flexible
Entry level
Hourly · $60–$120/hr

Posted Aug 25, 2026

Software Engineering AI Code Evaluator

Use your software engineering expertise to create datasets, test AI-generated code, and assess solutions across the development lifecycle. This flexible contractor role offers 10 to 40 hours per week for candidates in six eligible countries.

Coding & Software
Computer Code Programming
Remote · United States, Canada, Austria +3 more
English
Part-time · Flexible
Entry level

Posted Aug 6, 2026

Software Engineering AI Evaluation Expert

Use your software engineering judgment to evaluate AI-generated technical content, refine prompts, fact-check claims, and create high-quality engineering artifacts. Work remotely as a flexible contractor for 20+ hours per week.

Coding & Software
Document
Remote · Worldwide
English
Part-time · Flexible
Entry level
Hourly · $100–$200/hr

Posted Aug 4, 2026