Senior Software Engineer LLM Evaluation (US-Based)

You will help create and refine code datasets used to train and evaluate large language models. The work focuses on reviewing AI-generated software, correcting errors, writing strong solutions, and judging whether implementations are reliable and ready for production.

You will work across backend services, data pipelines, machine learning infrastructure, and broader software engineering workflows. Clear, structured explanations are an important part of each evaluation.

• Curate code examples and build solutions for AI training datasets.
• Review AI-generated code for correctness, efficiency, scalability, and reliability.
• Create Python agents and automated checks that find error patterns and validate solutions.
• Evaluate model performance across architecture, API design, production implementation, monitoring, and maintenance.
• Use software engineering benchmarks to assess AI coding solutions.

The role is a contractor engagement planned for one month, with possible extensions based on performance and fit.

• Time: 20 or more hours per week, with a flexible commitment.
• Experience: At least three years of software engineering experience.
• Skills: Deep Python expertise, including production frameworks, tools, and best practices.
• Skills: Experience building full-stack applications and deploying scalable software.
• Skills: Strong knowledge of software architecture, design, development, debugging, and code review.
• Languages: Professional fluency in English.
• Relevant technologies: JavaScript and ReactJS, C or C++, Java, Rust, or Go.