Senior Python Engineer, LLM Evaluation (US-Based)

Create and review coding examples used to train and evaluate large language models. You will assess AI-generated code across software development tasks, from early designs to production and ongoing operations.

• Create code examples, write solutions, and correct code for Python backend and machine learning workflows.
• Develop and evaluate JavaScript solutions across React, Node.js, frontend, and API work.
• Review additional solutions in C/C++, Java, Rust, and Go.
• Judge code for architecture, debugging, efficiency, scalability, reliability, and production quality.
• Build agents that check code quality, find recurring errors, and test software engineering solutions.
• Assess model capabilities in prototyping, architecture and API design, production implementation, monitoring, and operational maintenance.

What it pays and takes

• Hours: 20 or more per week.
• Location: Remote, open to candidates in the United States.
• Engagement: Part-time contractor role, initially for one month, with possible extensions based on performance and fit.
• At least three years of software engineering experience.
• Strong full-stack development skills in Python and JavaScript, including React and Node.js.
• Experience deploying scalable, production-grade software using modern languages and tools.
• Strong knowledge of software architecture, design, development, debugging, and code review.
• Ability to explain evaluation decisions clearly in writing and conversation.
• Experience with intelligent systems or advanced AI software is helpful. Familiarity with C/C++, Java, Rust, Go, automated verification, agents, or model benchmarking is also useful.
• Fluent in English.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind teaching AI systems, including creating examples and reviewing model responses. Software engineers bring the judgment needed to assess code and help improve how models handle real development tasks.