Evaluate real software repositories and help measure how well AI coding systems fix bugs. This remote, three-month contractor assignment offers 20 hours per week for engineers skilled in codebases, testing, Git, and Docker.
Coding & Software
Worldwide
Eligibility
Entry
Experience
Jul 16, 2026
Posted
Open worldwide
OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. It helps contributors discover projects, build a professional profile, and apply for opportunities that match their skills.
OpenTrain AI is hiring and contracting for this remote software engineering evaluation assignment.
AI training is the human side of building artificial intelligence. For coding systems, experienced software professionals help evaluate generated code, test whether bugs are fixed correctly, and create reliable benchmarks from real-world projects.
This work contributes to the development of advanced AI systems while offering flexible remote opportunities for people with software engineering experience.
As an LLM Evaluation & Repository Validation Engineer, you will help build verifiable software engineering tasks from public repository histories. You will analyze GitHub issues, configure code repositories, establish development environments, review test coverage and code quality, and run projects locally to assess bug-fixing performance.
You will also help identify repositories and issues that are especially challenging for large language models, contributing technical judgment from a software engineering perspective.
You will work with open-source software repositories and collaborate with researchers on repository selection and evaluation task design. The work requires hands-on technical investigation rather than purely theoretical review.
This role is suited to someone who can confidently navigate complex codebases, debug software locally, and assess whether tests and fixes are reliable. Strong experience with public GitHub repositories is important.
The following experience is helpful for contributing effectively to repository validation and AI evaluation work, but is not presented as a mandatory requirement.
AI training and data-labeling work spans code evaluation, model feedback, language, images, audio, and other forms of human-reviewed data. Contributors use their professional knowledge to help shape how modern AI systems perform.
Through OpenTrain, you can build a profile that showcases relevant experience, discover projects aligned with your skills, and develop a lasting portfolio in this rapidly growing field.
Keep exploring
Evaluate real Rust repositories and GitHub issues to help measure how LLMs solve software bugs. This part-time contractor role offers 20+ hours per week for engineers experienced with Rust, Git, Docker, and local testing.
Posted Jul 16, 2026
Join OpenTrain as a remote contractor to evaluate LLM performance on real open-source codebases using Ruby, Git, and Docker. This part-time role requires at least 20 hours/week, a 4-hour PST overlap, and candidates based in specified countries.
Posted Jul 20, 2026
Join OpenTrain to build LLM evaluation and training datasets by validating real open-source codebases with C#. This remote contractor role requires 3+ years of software engineering, 20+ hours/week (options to 30–40) and is open to candidates in specified countries.
Posted Jul 20, 2026
Explore related categories
