Senior Software Engineer, LLM Code Evaluation

Create and assess software examples used to train and benchmark large language models. You will apply software engineering judgment to improve AI-generated code and evaluate how well solutions work.

• Curate code examples and write precise solutions for training and benchmarking datasets.
• Correct and refine code in Python, C/C++, Rust, Go, Java, and JavaScript, including ReactJS.
• Evaluate systems, infrastructure, and backend code for correctness, performance, scalability, and reliability.
• Assess model abilities across prototyping, architecture and API design, production implementation, monitoring, and maintenance.
• Build agents that check code quality, find error patterns, and test solutions to software engineering tasks.
• Work with technical teams to improve AI coding solutions against industry performance benchmarks.

Hours, location, and requirements

• Hours: 20+ per week, with flexible participation.
• Location: You must be based in the United States.
• Engagement: Part-time contractor role, starting with a one-month term. Extensions may be possible based on performance and fit.
• Fluent in English.
• Strong skills in systems programming, infrastructure, or backend development, with familiarity in Python, C/C++, Rust, or Go.
• Experience building and deploying scalable, production-ready software, plus a strong understanding of software architecture, design, development, debugging, and code review.
• Ability to write clear, structured explanations for technical evaluations.
• Experience shipping products in fast-moving environments may be helpful. Computer science education is welcome, but demonstrated skills and experience take priority.

How to apply

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work uses examples prepared and reviewed by people to help AI models learn. This role uses software engineering knowledge to create code examples and assess model-generated solutions.