Senior Software Engineer, LLM Evaluation

You will create and evaluate coding datasets used to train and test large language models. The work calls for software engineering judgment across correctness, efficiency, scalability, reliability, and code quality.

You will work mainly in Python and review AI-generated implementations across backend services, data pipelines, machine learning infrastructure, and full-stack applications.

• Curate code examples, write precise solutions, and correct software.
• Evaluate AI-generated code across practical software development scenarios.
• Build Python agents and automated verification tools that find error patterns and assess code quality.
• Design systems that automatically verify solutions to software engineering tasks.
• Assess model performance in prototyping, architecture, API design, production implementation, monitoring, and operational maintenance.
• Work with cross-functional teams to improve AI coding solutions against industry benchmarks.

The listing does not provide a pay rate. This is a contractor engagement with an initial duration of one month and possible extensions based on performance and fit.

• Schedule: Flexible, with a commitment of 20 or more hours per week.
• Experience: At least three years of software engineering experience.
• Required: Strong Python skills, including production frameworks, tooling, and best practices.
• Required: Experience building full-stack applications and deploying scalable software.
• Required: Strong knowledge of software architecture, development, debugging, and code review.
• Required: Clear spoken and written communication for structured evaluation explanations.
• Relevant coding skills: JavaScript, including ReactJS, plus C or C++, Java, Rust, or Go.