SWE Bench Engineer (Java)

This role evaluates Java code produced or reviewed for large language models. You will work with public Java repositories, GitHub issues, and unit-test codebases, then provide clear feedback that helps improve AI systems.

The duties and qualifications below describe Java LLM code evaluation, so review them before applying.

• Evaluate model outputs using project rubrics and record evidence-based judgments.
• Review source materials and model outputs against project guidelines.
• Explain your decisions clearly so feedback can improve model behavior.
• Run, modify, and test real-world Java projects locally.
• Maintain consistent, high-quality decisions across repeated review tasks.

This is a fully remote contractor assignment. The project is open to candidates in the countries listed below and requires English fluency.

• Time: At least 20 hours per week.
• Schedule: At least four hours per day and four hours of overlap with Pacific Standard Time.
• Language: English fluency.
• Experience: At least three years of overall experience; previous AI training experience is not required.
• Technical skills: Strong Java experience, Git, Docker, basic software pipeline setup, and the ability to navigate complex codebases.