Skip to content
OpenTrain AIFor AI Companies

LLM Code Evaluation Engineer

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

Worldwide

Posted

Jul 16, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting a software engineering contractor to evaluate large language models against realistic coding problems and validate repositories used to create AI training datasets. You will work with public open-source codebases, investigate bug-fixing scenarios, and help produce verifiable software-engineering tasks through hands-on technical review. The work combines practical development, repository quality assessment, and structured evaluation of model performance.

What you will do
Analyze and triage issues across widely used open-source libraries. Configure repositories and development environments, including Docker-based setups. Review unit-test coverage and assess test quality. Modify and run codebases locally to evaluate LLM performance on bug-fixing tasks. Collaborate with researchers to identify repositories and issues that present meaningful challenges for language models. You may also help coordinate junior engineers on project work.

Required skills
You should have strong programming experience in at least one of Python, JavaScript, Java, Go, Rust, C, C++, C#, or Ruby. You should be proficient with Git, Docker, and basic software pipeline setup, and able to understand and navigate complex codebases. Comfort running, modifying, and testing real-world projects locally is required. Experience contributing to or evaluating open-source projects is useful, particularly with well-maintained public repositories.

Helpful background
Previous work in LLM research or evaluation, or experience building and testing developer tools or automation agents, is relevant. Familiarity with widely used public repositories, including repositories with 500 or more stars, is also helpful.

Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.

Requirements

  • Languages: English

Open worldwide

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmrnidzfy001j04l5mxkpgan7

Keep exploring

Similar open roles

View all AI training jobs

Java LLM Code Evaluation Engineer

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Java
Code
Remote · Argentina, Belize, Bolivia +20 more
English
Part-time · Flexible
Per task · $25/label

Posted Nov 10, 2026

Senior Python Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and turn specialized knowledge into a durable freel…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Java LLM Code Evaluation Engineer

About OpenTrain OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and turn specialized knowledge into a durable freel…

Java
Code
Remote · Argentina, Bangladesh, Belize +28 more
English
Part-time · Flexible
Per task · $25/label

Posted Nov 3, 2026