You will combine hands-on software engineering with AI evaluation. The work uses public GitHub repositories and real-world codebases to test software quality and measure how well language models complete coding and bug-fixing tasks.
- Analyze and triage issues across open-source repositories.
- Configure development environments and Dockerize repositories for reliable local workflows.
- Run, modify, debug, and test software projects locally.
- Evaluate unit test coverage, reliability, and overall test quality.
- Assess model performance on software engineering and bug-fixing tasks.
- Find challenging repositories and issues for AI evaluation benchmarks.
- Help technical researchers and engineers design strong evaluation scenarios.
- Review code, provide technical guidance, and coordinate small teams when needed.
What it pays and takes
This is a part-time contractor role with a minimum time commitment of 20 hours per week. The listing does not specify a pay rate.
- Work arrangement: Contractor and part time.
- Time requirement: 20 or more hours per week.
- Location: Worldwide.
- Language: English.
- Experience: The listing is marked entry level, but it requires at least five years of professional software engineering experience.
- Technical skills: Strong Python and JavaScript, plus practical Git and Docker experience.
- Knowledge: Software development pipelines and continuous integration workflows.
- Ability to understand, navigate, modify, run, and test complex real-world codebases locally.
- Strengths: Debugging, code review, problem-solving, and technical communication.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI training work
AI training work uses human judgment to prepare examples, review model outputs, and test whether AI systems solve real tasks correctly. Experienced software engineers are paid for applying their technical judgment to code review, debugging, and reliable evaluation scenarios.
