AI/ML Model Evaluator, Coding Solutions

You will review three AI coding agents that handled the same software engineering request. You will inspect each agent’s commands, changed files, validation checks, and final message, then judge how well the solution works.

Your ratings will support improvements to AI coding agents and their software development skills.

• Compare three AI-generated software solutions for the same developer request.
• Review commands, modified files, checks, and final communications.
• Assess instruction following, functionality, efficiency, safety, honesty, and communication across ten quality areas.
• Give each run an overall score from 1 to 5.
• Rank the three solutions and explain the ranking with technical evidence.

What it pays and takes

This is a remote contractor assignment lasting one month. The work requires careful technical judgment when comparing code correctness, efficiency, safety, and execution quality.

• Pay: $25 per hour equivalent, paid per task that meets project specifications.
• Time: 20+ hours per week. The assignment requires at least 8 hours per day and 40 hours per week, including 4 hours of overlap with PST.
• Location: Open to candidates in India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico, and Laos.
• Experience: At least three years of overall software engineering experience.
• Programming: Strong experience with Python, JavaScript or TypeScript, Java, Go, PHP, or a similar language.
• Tools: Proficiency with Shell commands, Git, Docker, and basic software pipeline setup.
• Project work: Ability to run, modify, and test real-world software projects locally.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind systems that learn from examples, including reviewing code and rating AI-generated responses. Experienced software engineers are needed to judge whether these systems produce correct, safe, efficient, and useful results.