Senior Python Engineer, LLM Evaluation (US-Based)

You will evaluate software engineering work used to train and test large language models. The work calls for sound engineering judgment about code quality, performance, and reliability.

• Curate code examples and write precise programming solutions for training datasets.
• Review and improve AI-generated code for correctness, efficiency, scalability, and reliability.
• Build Python agents and automated checks to find error patterns and validate software solutions.
• Assess model capabilities across prototyping, architecture and API design, implementation, launch, experimentation, monitoring, and maintenance.
• Design ways to verify software engineering tasks and compare coding solutions against performance benchmarks.
• Contribute code in Python, JavaScript, including ReactJS, C/C++, Java, Rust, and Go.

What you need

• Location: Open to candidates in the United States.
• Availability: At least 20 hours per week.
• Work arrangement: Part-time contract work.
• Language: English fluency.
• At least three years of software engineering experience.
• Strong Python skills, including production frameworks, tools, and best practices.
• Experience building full-stack applications and deploying scalable software.
• Strong knowledge of software architecture, development, debugging, and code quality review.
• Ability to write clear, structured explanations for technical evaluations.
• Familiarity with backend services, data pipelines, or machine learning infrastructure.
• Experience building production systems in a high-scale engineering environment and a strong computer science foundation are helpful. Practical engineering skill is valued alongside formal education.

How to apply

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind building AI systems: people prepare examples and review model responses, including code. Software engineering experience helps contributors judge whether solutions are correct, reliable, and efficient.