About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting for a Senior LLM Code Evaluation Engineer to build LLM evaluation and training datasets from public GitHub repositories. This role blends practical software engineering with AI research to assess how LLMs interact with real code, directly improving AI-assisted software development. Using a human-in-the-loop approach, you will help create verifiable software engineering tasks that push the boundaries of LLM capabilities.
What you will do
- Analyze and triage GitHub issues across trending open-source libraries.
- Set up and configure code repositories including Dockerization and environment setup.
- Evaluate unit test coverage and quality.
- Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
- Collaborate with researchers to identify repositories and issues that challenge LLMs.
- Lead a team of junior engineers on related projects.
Required skills
- Minimum 3 years of overall software engineering experience.
- Strong experience with the Go programming language.
- Proficiency with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfortable running, modifying, and testing real-world projects locally.
Helpful background
- Previous participation in LLM research or evaluation projects.
- Experience building or testing developer tools or automation agents.