Skip to content
OpenTrain AIFor AI Companies

Senior Software Engineer, Python (LLM Evaluation and Repository Validation)

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

9 countries

Posted

Jul 16, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting for a senior software engineering role focused on LLM evaluation and training datasets for realistic software engineering problems. The work includes analyzing and triaging GitHub issues, configuring repositories and development environments, evaluating unit test coverage and quality, and running code locally to assess LLM bug-fixing performance. It also involves helping researchers identify challenging repositories and issues and may include leading junior engineers on project work.

What you will do
- Analyze and triage GitHub issues across open-source libraries.
- Set up repositories with Docker and development environment automation.
- Evaluate unit test coverage and quality.
- Run and modify codebases locally to assess model performance on bug-fixing tasks.
- Collaborate with researchers on repository and issue selection.
- Support junior engineers on collaborative project work when needed.

Required skills
- 3+ years of overall engineering experience.
- Strong Python experience.
- Practical skill with Git, Docker, and software pipeline setup.
- Comfort navigating complex codebases and testing real-world projects locally.
- Experience contributing to or evaluating open-source projects is a plus.
- Prior LLM research or evaluation experience is helpful.

Helpful background
- Experience building or testing developer tools or automation agents.
- Familiarity with high-quality public GitHub repositories.

Requirements

  • Languages: English

Open to applicants in

  • Bangladesh
  • Egypt
  • Ghana
  • India
  • Kenya
  • Mexico
  • Nigeria
  • Pakistan
  • Türkiye

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmrnidzzm001n04l5sa5c1hx4

Keep exploring

Similar open roles

View all AI training jobs

LLM Evaluation Software Engineer (Ruby)

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Ruby
Text
Remote · India, Pakistan, Nigeria +6 more
English
Part-time · Flexible

Posted Jul 20, 2026

Senior Python Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and turn specialized knowledge into a durable freel…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Python Engineer, LLM Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Python Engineer, LLM Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Python Engineer, LLM Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026