Senior Software Engineer, LLM Evaluation

You will improve AI coding systems by creating accurate software examples, correcting generated code, and judging whether solutions are correct, efficient, scalable, reliable, and maintainable. Your work will support training and benchmarking datasets for large language models.

The role combines hands-on full-stack engineering with structured model evaluation. You will assess software across prototyping, architecture, API design, production implementation, experimentation, monitoring, and operational maintenance.

• Curate code examples and create solutions for backend, frontend, API, and machine learning workflows.
• Review and refine AI-generated code for correctness, maintainability, scalability, and reliability.
• Evaluate coding systems against software development and production performance benchmarks.
• Build agents and automated checks that verify code quality and identify recurring errors in full-stack applications.
• Work with cross-functional teams on AI coding solutions and evaluation benchmarks.

This is a contractor, part-time role open to candidates in the United States. The work requires at least 20 hours per week and professional fluency in English.

• Schedule: 20 or more hours per week.
• Experience: At least three years of software engineering experience.
• Technical skills: Strong full-stack development in Python and JavaScript, including React and Node.js.
• Additional systems experience: C, C++, Java, Rust, or Go.
• Knowledge: Software architecture, design, development, debugging, scalable production software, and code-quality assessment.
• Communication: Clear spoken and written explanations for structured evaluation decisions.