Senior Python Engineer, LLM Evaluation

Create training and benchmarking datasets for large language models by preparing code examples, writing solutions, and evaluating AI-generated software. The work covers backend, frontend, and API development, with attention to efficiency, scalability, reliability, and performance.

The role focuses on software engineering evaluation across full-stack work, so review the duties and qualifications before applying.

• Curate code examples and provide precise solutions for model training.
• Review and refine AI-generated code in Python, JavaScript, React, Node.js, C/C++, Java, Rust, and Go.
• Assess software architecture, API design, production implementation, experiments, monitoring, and ongoing maintenance.
• Build agents that check code quality, identify recurring errors in full-stack applications, and automatically verify solutions to software engineering tasks.
• Work with cross-functional teams to improve AI coding systems against industry performance benchmarks.

• Schedule: Flexible, with a commitment of 20 or more hours per week.
• Engagement: Part-time contractor work, initially for one month, with potential extensions based on performance and fit.
• Skills: Strong full-stack development skills in Python and JavaScript, including React and Node.js, and experience deploying scalable, production-grade software.
• Knowledge: Software architecture, design, development, debugging, and code review, plus the ability to explain technical judgments clearly in writing and speech. English communication is required.