About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting for a Rust LLM evaluation and repository validation engineer to support the creation of training and evaluation datasets for realistic software engineering problems. The work centers on analyzing GitHub issues, setting up and configuring repositories, evaluating test coverage and quality, and running code locally to assess how LLMs handle bug-fixing scenarios.
What you will do
- Analyze and triage GitHub issues across open-source libraries.
- Set up repositories, including Dockerization and environment configuration.
- Evaluate unit test coverage and quality for software tasks.
- Modify and run codebases locally to assess LLM performance in bug-fixing workflows.
- Collaborate with researchers to identify repositories and issues that are challenging for LLMs.
- Contribute technical judgment at a senior software engineering level, with potential opportunity to lead junior engineers on project work.
Required skills
- Strong Rust experience.
- Experience with Git, Docker, and basic software pipeline setup.
- Ability to understand and navigate complex codebases.
- Comfort running, modifying, and testing real-world projects locally.
- Experience contributing to or evaluating open-source projects is a plus.
- Prior LLM research or evaluation experience is helpful.
Helpful background
- Experience with developer tools or automation agents.
- Senior-level software engineering experience, especially in repository analysis and testing.