SWE Bench Engineer Golang

You will help evaluate how well AI systems handle real software engineering tasks. Your work will use open-source repositories, GitHub issues, source code, and unit-test suites to produce clear, consistent judgments.

You will review model outputs against project guidelines and explain the evidence behind your decisions. The feedback should help improve model behavior across repeated evaluation tasks.

• Evaluate LLM software engineering outputs using project rubrics.
• Review source materials, code, and model outputs against project guidelines.
• Run, modify, and test real-world projects locally.
• Document clear, evidence-based judgments.
• Maintain consistency and quality across repeated reviews.

What it pays and takes

This is a contractor, part-time role with a minimum commitment of 20 hours per week. The schedule requires four hours of overlap with Pacific Time.

• Pay: $25 per label.
• Minimum availability: 20+ hours per week.
• Schedule: Four hours of overlap with Pacific Time.
• Location: Open to candidates in Argentina, Belize, Bolivia, Brazil, Chile, Colombia, Costa Rica, the Dominican Republic, Ecuador, El Salvador, Guatemala, Guyana, Haiti, Honduras, Mexico, Nicaragua, Panama, Paraguay, Peru, Puerto Rico, Suriname, Uruguay, and Venezuela.
• Language: Professional fluency in English.
• Experience: At least three years of overall experience.
• Technical skills: Strong experience with Golang, plus proficiency with Git, Docker, and basic software pipeline setup.
• You should be able to understand and navigate complex codebases.
• Helpful background includes open-source contributions or evaluation, LLM research or evaluation work, and building or testing developer tools or automation agents.

How it works

Apply on OpenTrain with your resume and then complete the application on the hiring site.

About AI training work

AI training is the human work behind systems that learn from examples, including code reviews, model ratings, and written feedback. People with strong software engineering knowledge help evaluate difficult cases and show AI systems what accurate, useful work looks like.