Senior Software Engineer, LLM Evaluation

Create and improve code examples, develop solutions, and review AI-generated code. You will use software engineering judgment to assess correctness, efficiency, scalability, reliability, and overall quality.

• Work primarily in Python across backend services, data pipelines, machine learning infrastructure, and full-stack development.
• Build Python agents and automated tools to find error patterns, check solutions, and assess code quality.
• Evaluate model skills in prototyping, architecture, API design, production implementation, monitoring, and ongoing maintenance.
• Work with cross-functional teams to improve coding systems against industry benchmarks.

Requirements and engagement

• Location: Open to candidates based in the United States.
• Schedule: Flexible, with a commitment of at least 20 hours per week.
• Engagement: Part-time contractor role; initial duration is one month, with possible extensions based on performance and fit.
• Experience: At least three years in software engineering.
• Strong Python skills, including production frameworks, tools, and development practices.
• Experience building full-stack applications and deploying scalable software.
• Strong knowledge of software architecture, development, debugging, and code review.
• Clear spoken and written communication for explaining evaluation decisions.
• Working knowledge of JavaScript, including ReactJS, and one or more of C, C++, Java, Rust, or Go.
• Experience with advanced AI systems or organizations developing intelligent software is helpful. Strong computer science foundations are welcome.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work is the human work behind AI systems: people prepare examples and review model outputs so the systems can learn. Software engineers are needed to judge code quality and build reliable ways to test AI-generated solutions.