Skip to content
OpenTrain AIFor AI Companies

Senior LLM Code Evaluation Engineer

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

9 countries

Posted

Jul 17, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting for a Senior LLM Code Evaluation Engineer to build LLM evaluation and training datasets from public GitHub repositories. This role blends practical software engineering with AI research to assess how LLMs interact with real code, directly improving AI-assisted software development. Using a human-in-the-loop approach, you will help create verifiable software engineering tasks that push the boundaries of LLM capabilities.

What you will do
- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories including Dockerization and environment setup.
- Evaluate unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to identify repositories and issues that challenge LLMs.
- Lead a team of junior engineers on related projects.

Required skills
- Minimum 3 years of overall software engineering experience.
- Strong experience with the Go programming language.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfortable running, modifying, and testing real-world projects locally.

Helpful background
- Previous participation in LLM research or evaluation projects.
- Experience building or testing developer tools or automation agents.

Requirements

  • Languages: English

Open to applicants in

  • Bangladesh
  • Egypt
  • Ghana
  • India
  • Kenya
  • Mexico
  • Nigeria
  • Pakistan
  • Türkiye

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmroc4vj4000z04l2zmduacmh

Keep exploring

Similar open roles

View all AI training jobs

LLM Evaluation Software Engineer (Ruby)

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Ruby
Text
Remote · India, Pakistan, Nigeria +6 more
English
Part-time · Flexible

Posted Jul 20, 2026

Senior Python Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and turn specialized knowledge into a durable freel…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026