Senior Software Engineer, LLM Code Evaluation

You will help create training and benchmarking datasets for large language models. The work focuses on reviewing AI-generated code, writing accurate solutions and corrections, and explaining why one solution is stronger than another.

You will assess code quality, reliability, efficiency, scalability, and common error patterns. You will also evaluate AI performance across software work such as prototyping, architecture, API design, production implementation, launch, experimentation, monitoring, and ongoing operations.

• Create, curate, or correct code examples in Python, JavaScript, ReactJS, C/C++, Java, Rust, and Go.
• Review AI-generated code against software performance and quality standards.
• Build verification agents and other checks that find errors and validate coding solutions automatically.
• Assess software engineering work across the full development lifecycle.
• Work with other teams to improve AI coding solutions.

This is a flexible, part-time contractor engagement. The initial project duration is one month, with possible extensions based on performance and fit.

• Time: At least 20 hours per week.
• Language: Fluent English for clear written and spoken evaluation work.
• Experience: At least three years of software engineering experience, including full-stack development and production deployment.
• Skills: Strong knowledge of software architecture, design, development, debugging, and code review.
• Communication: Able to write clear, structured reasons for evaluation decisions.