Senior Python Engineer, LLM Evaluation (US-Based)

Create and improve software engineering examples and solutions used to train and assess large language models. You will focus on Python code for backend services, data pipelines, and machine learning infrastructure, with work across other programming languages too.

• Prepare code examples and write precise solutions to software engineering tasks.
• Review and improve AI-generated code for correctness, efficiency, scalability, and reliability.
• Build Python agents and automated tools to find error patterns and assess code quality.
• Design ways to automatically check solutions to programming problems.
• Assess model performance across prototyping, architecture and API design, implementation, launch, experimentation, monitoring, and maintenance.
• Work with other teams to improve AI coding systems against performance benchmarks.

What you need and engagement details

• At least three years of software engineering experience.
• Deep Python knowledge, including production frameworks, tools, and best practices.
• Experience building full-stack applications and deploying software that can scale.
• Strong skills in software architecture, design, development, debugging, and code quality review.
• Ability to write and explain clear, structured evaluation rationales.
• Working experience with at least one of JavaScript, ReactJS, C/C++, Java, Rust, or Go.
• Open to candidates based in the United States.
• Plan to contribute at least 20 hours per week.
• Contractor engagement with an initial one-month duration; extensions may be available based on performance and fit.

How to apply

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work is the human work of preparing examples and reviewing model responses so AI systems can learn and be evaluated. This role draws on software engineering experience to create code examples and judge whether generated programs work well.