About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting for a C# Software Engineer, LLM Code Evaluation role focused on building and validating training datasets for realistic software engineering problems. You will work with public GitHub repositories and repository histories to help create verifiable software engineering tasks using human review, while assessing how language models interact with real code. The role combines hands-on software engineering, repository analysis, test-quality judgment, and practical evaluation of LLM bug-fixing performance.
What you will do
- Analyze and triage issues across trending open-source libraries.
- Configure repositories and development environments, including Dockerization and basic pipeline setup.
- Evaluate unit-test coverage and test quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Help identify repositories and issues that present meaningful challenges for language models.
- Collaborate with researchers on task and dataset design.
- Lead or coordinate junior engineers when project needs call for additional support.
Required skills
- At least three years of professional software engineering experience.
- Strong experience with C#.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to navigate complex codebases and work comfortably with real-world projects locally.
- Sound judgment when analyzing GitHub issues, repository quality, test coverage, and expected software behavior.
Helpful background
Experience contributing to or evaluating open-source projects is useful, as is previous participation in LLM research or evaluation work. Experience building or testing developer tools or automation agents can also help you assess software engineering tasks from both implementation and model-performance perspectives.
Working arrangement
This is an individual contractor assignment with a fully remote arrangement. The expected commitment is at least 20 hours per week, with options for 20, 30, or 40 hours and four hours of daily overlap with PST.