You will build code evaluation and training datasets from public GitHub repositories. The work combines practical software engineering with reviewing how large language models handle real code and software development tasks.
You will also help create software engineering tasks that can be checked with clear results, and guide junior engineers working on related projects.
- Analyze and sort GitHub issues from popular open-source libraries.
- Set up code repositories, including Docker environments and basic project configuration.
- Review unit test coverage and assess whether tests are effective.
- Run and modify codebases locally to evaluate LLM performance on bug-fixing tasks.
- Work with researchers to find repositories and issues that are challenging for LLMs.
- Lead junior engineers on related projects.
What it pays and takes
This is a part-time contractor role requiring at least 20 hours each week. The role is open to applicants in the listed countries and requires English.
- Pay: The source listing does not specify a rate.
- Workload: 20+ hours per week.
- Contract type: Part-time contractor.
- Location: India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, or Mexico.
- Experience: At least 3 years of overall software engineering experience.
- Programming: Strong experience with Go.
- Tools: Proficiency with Git, Docker, and basic software pipeline setup.
- Technical ability: Navigate complex codebases and run, modify, and test real-world projects locally.
- Helpful background: LLM research or evaluation work, or experience building and testing developer tools or automation agents.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI training work
AI training work uses human examples and reviews to improve how artificial intelligence systems understand and produce useful results. In this role, software engineers help test and evaluate how LLMs work with real code.
