Skip to content
OpenTrain AIFor AI Companies

Software Engineer AI Code Evaluation Benchmarking

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

United States only

Posted

Jul 17, 2026

OpenTrain on Trustpilot

The work

  • Support human feedback and evaluation data for systems that generate and reason about code.
  • Review AI-generated code for correctness, efficiency, maintainability, and compliance with requirements.
  • Analyze software engineering tasks and validate whether proposed solutions meet expected outcomes.
  • Debug code, reproduce issues, and verify fixes across different programming environments.
  • Assess model-generated explanations, reasoning, and implementation approaches for technical accuracy.
  • Create, refine, and maintain evaluation datasets, coding benchmarks, and grading rubrics.
  • Identify edge cases and technical failure modes where AI systems struggle with software engineering problems.
  • Document technical findings and provide structured feedback for consistent evaluations.
  • Collaborate with project teams to establish quality standards and evaluation methods.

Requirements

  • The contract is expected to last 1 month.
  • Work at least 20 hours per week, including at least 4 hours per day and 4 hours of overlap with Pacific Time.
  • Compensation details are not disclosed.
  • Bachelor's or master's degree in computer science, software engineering, or a related technical field.
  • At least 3 years of professional software engineering experience.
  • Proficiency in 1 or more of Python, Java, C/C++, Go, Swift, Objective-C, PHP, or SQL.
  • Strong knowledge of data structures, algorithms, software design principles, and debugging methods.
  • Experience reviewing code and evaluating quality in production or large-scale codebases.
  • Familiarity with Git or a comparable version control system and modern software development workflows.
  • Strong written communication, careful attention to detail, and the ability to assess complex technical solutions with minimal supervision.
  • Languages: English

Open to applicants in

  • United States

Preferred

  • Experience with AI or machine learning data annotation, natural language processing, prompt engineering, model evaluation, large language model projects, benchmark creation, AI-generated code evaluation, or software quality assessment is helpful.

How to apply

  1. Press Apply now. You create a free OpenTrain account and add your resume.
  2. Finish your application on the hiring company's site. The company decides who it hires, so it runs its own form.

OpenTrain on Trustpilot

Job ID: cmroshmk5000809kk93lh8phu

Keep exploring

Similar open roles

View all AI training jobs

Talent Automation Lead

Build and evolve an automation agent for talent operations through workflow automation, large language model integration, and fullstack development. This remote contractor role pays $57.69 to $76.92 per hour and requires 20+ hours weekly.

Workflow automationProcess improvementLLMsFullstack development
Coding & Software
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $57.69–$76.92/hr

Posted Nov 21, 2026

Forward Deployed Engineer

Build production AI systems, data infrastructure, agent workflows, and evaluation tools as a Forward Deployed Engineer. Pay is $86.54 to $120.19 per hour, with 20+ hours per week.

PythonLLM SystemsML InfrastructureRAG/AI Automation
Coding & Software
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $86.54–$120.19/hr

Posted Nov 20, 2026

Director of Infrastructure Engineering

Lead cloud infrastructure, reliability, security, and platform engineering for advanced AI products at $110.58 to $125 per hour. The role requires 20+ hours per week and strong AWS, GCP, Terraform, Kubernetes, and team leadership experience.

AWSGCPIaCCI/CDSecurity-first MindsetTeam Leadership
Coding & Software
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $110.58–$125/hr

Posted Nov 20, 2026

Software Engineer, Financial Platform (English)

Build and maintain financial platform software across backend services, REST APIs, AWS infrastructure, and responsive interfaces. This contractor role pays $67.31 to $76.92 per hour and requires 20+ hours per week.

ReactNode.jsPostgreSQLAWS
Coding & Software
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $67.31–$76.92/hr

Posted Nov 19, 2026

Software Engineer, Internal Platforms

Build internal platforms, backend services, APIs, and frontend tools as a contractor. This entry-level role pays $57.69 to $74.52 per hour and requires 20+ hours weekly.

ReactNode.jsPostgreSQLAWS
Coding & Software
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $57.69–$74.52/hr

Posted Nov 19, 2026