Skip to content
OpenTrain AIFor AI Companies

C# Software Engineer, LLM Code Evaluation

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

9 countries

Posted

Jul 20, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting for a C# Software Engineer, LLM Code Evaluation role focused on building and validating training datasets for realistic software engineering problems. You will work with public GitHub repositories and repository histories to help create verifiable software engineering tasks using human review, while assessing how language models interact with real code. The role combines hands-on software engineering, repository analysis, test-quality judgment, and practical evaluation of LLM bug-fixing performance.

What you will do
- Analyze and triage issues across trending open-source libraries.
- Configure repositories and development environments, including Dockerization and basic pipeline setup.
- Evaluate unit-test coverage and test quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Help identify repositories and issues that present meaningful challenges for language models.
- Collaborate with researchers on task and dataset design.
- Lead or coordinate junior engineers when project needs call for additional support.

Required skills
- At least three years of professional software engineering experience.
- Strong experience with C#.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to navigate complex codebases and work comfortably with real-world projects locally.
- Sound judgment when analyzing GitHub issues, repository quality, test coverage, and expected software behavior.

Helpful background
Experience contributing to or evaluating open-source projects is useful, as is previous participation in LLM research or evaluation work. Experience building or testing developer tools or automation agents can also help you assess software engineering tasks from both implementation and model-performance perspectives.

Working arrangement
This is an individual contractor assignment with a fully remote arrangement. The expected commitment is at least 20 hours per week, with options for 20, 30, or 40 hours and four hours of daily overlap with PST.

Requirements

  • Languages: English

Open to applicants in

  • Bangladesh
  • Egypt
  • Ghana
  • India
  • Kenya
  • Mexico
  • Nigeria
  • Pakistan
  • Türkiye

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmrt54qyb000504jrsqqw46zn

Keep exploring

Similar open roles

View all AI training jobs

Senior LLM Code Evaluation Engineer

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Go
Code
Remote · India, Pakistan, Nigeria +6 more
English
Part-time · Flexible

Posted Jul 17, 2026

Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Software Engineer - LLM Evaluation

About OpenTrain OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and turn specialized knowledge into a durable freel…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Software Engineer, LLM Evaluation

OpenTrain is recruiting senior software engineers to evaluate AI-generated code and create training and benchmarking datasets for large language models. This contractor role combines hands-on full-stack engineering with…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Software Engineer, LLM Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026