Skip to content
OpenTrain AIFor AI Companies

Software Engineer, LLM Evaluation

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

United States only

Posted

Nov 6, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting for a Software Engineer, LLM Evaluation role focused on creating and evaluating coding datasets used to train and benchmark large language models. You will work with code examples and AI-generated implementations, applying software engineering judgment to assess correctness, efficiency, scalability, reliability, and quality across practical development scenarios.

What you will do
Curate code examples, build precise solutions, and correct software primarily in Python. Evaluate AI-generated code across backend services, data pipelines, ML infrastructure, and full-stack development. Build Python agents and automated verification tools that identify error patterns and assess code quality. Design mechanisms that automatically verify solutions to software engineering tasks. Evaluate model capabilities across prototyping, architecture, API design, production implementation, monitoring, and operational maintenance. Collaborate with cross-functional teams to improve AI-driven coding solutions against industry performance benchmarks.

Required skills
At least three years of software engineering experience. Strong Python expertise, including production frameworks, tooling, and best practices. Experience building full-stack applications and deploying scalable software. Deep knowledge of software architecture, development, debugging, and code review. Clear oral and written communication skills for structured evaluation rationales. Working knowledge of JavaScript, including ReactJS, C or C++, Java, Rust, or Go is relevant to the coding scope.

Helpful background
Experience working on advanced AI systems or at organizations developing intelligent software can be valuable. Strong computer science foundations and demonstrated engineering ability are welcomed, while practical skill and experience are the primary focus.

Engagement details
This is a contractor engagement for candidates based in the United States. The schedule is flexible, with a commitment of 10 to 40 hours per week, and the initial duration is one month with potential extensions based on performance and fit.

Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.

Requirements

  • Languages: English

Open to applicants in

  • United States

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmw4vczdm000w08l5qfv5o3cv

Keep exploring

Similar open roles

View all AI training jobs

Senior Software Engineer, LLM Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Software Engineer - LLM Evaluation

About OpenTrain OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and turn specialized knowledge into a durable freel…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Software Engineer, LLM Evaluation

OpenTrain is recruiting senior software engineers to evaluate AI-generated code and create training and benchmarking datasets for large language models. This contractor role combines hands-on full-stack engineering with…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Software Engineer C++ LLM Evaluation

Evaluate C++ code, repositories, and LLM performance on real software tasks. This remote contract offers flexible 20, 30, or 40-hour options across 9 countries.

C++
Coding & Software
Code
Remote · India, Pakistan, Nigeria +6 more
English
Part-time · Flexible

Posted Jul 17, 2026