Senior Software Engineer, LLM Code Evaluation

You will help create training and benchmark datasets that assess how well AI models handle software engineering. The work involves reviewing and refining code, writing precise solutions, and explaining your judgments clearly.

• Create or correct code examples in Python, JavaScript, ReactJS, C/C++, Java, Rust, and Go.
• Review AI-generated code for efficiency, scalability, reliability, and performance.
• Build verification agents and other checks that find recurring errors and validate solutions automatically.
• Assess model abilities across prototyping, architecture and API design, production implementation, launch, experimentation, monitoring, and ongoing maintenance.
• Work with cross-functional teams to improve AI coding solutions.

What you need

• Strong full-stack application development skills and production deployment experience.
• Deep knowledge of software architecture, design, development, debugging, and code review.
• Working ability in multiple programming languages named above.
• Fluent in English, with strong spoken and written communication skills for clear, structured evaluation explanations.
• Experience designing automated verification methods and reasoning across the software development lifecycle is helpful.

Schedule and location

• Contractor engagement with flexible scheduling and a requirement of 20 or more hours per week.
• Open to candidates based in the United States.
• The initial engagement is one month, with possible extensions based on performance and fit.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work involves people preparing and reviewing examples that help AI models learn. In this role, software engineering expertise helps assess code and explain whether AI-generated solutions work well.