Senior Python Engineer, LLM Evaluation (US-Based)

Create and review coding examples used to train and evaluate large language models. You will assess AI-generated code across software development tasks, from early designs to production and ongoing operations.

• Create code examples, write solutions, and correct code for Python backend and machine learning workflows.
• Develop and evaluate JavaScript solutions across React, Node.js, frontend, and API work.
• Review additional solutions in C/C++, Java, Rust, and Go.
• Judge code for architecture, debugging, efficiency, scalability, reliability, and production quality.
• Build agents that check code quality, find recurring errors, and test software engineering solutions.
• Assess model capabilities in prototyping, architecture and API design, production implementation, monitoring, and operational maintenance.

• Hours: 20 or more per week.
• Engagement: Part-time contractor role, initially for one month, with possible extensions based on performance and fit.
• At least three years of software engineering experience.
• Strong full-stack development skills in Python and JavaScript, including React and Node.js.
• Experience deploying scalable, production-grade software using modern languages and tools.
• Strong knowledge of software architecture, design, development, debugging, and code review.
• Ability to explain evaluation decisions clearly in writing and conversation.
• Experience with intelligent systems or advanced AI software is helpful. Familiarity with C/C++, Java, Rust, Go, automated verification, agents, or model benchmarking is also useful.
• Fluent in English.