Senior Python Engineer, LLM Evaluation

You will create and assess software engineering examples used to train and test large language models. The work combines hands-on coding with structured technical reviews and automated checks.

You will help measure how well AI systems handle software design, implementation, monitoring, and maintenance across production-focused engineering tasks.

• Curate code examples and write precise solutions for software engineering tasks.
• Correct and improve implementations in Python, C/C++, Rust, Go, Java, JavaScript, and ReactJS.
• Review AI-generated code for correctness, efficiency, scalability, reliability, and systems-level behavior.
• Build automated agents and verification methods for infrastructure and systems code.
• Find recurring errors and assess model performance across prototyping, architecture, API design, production implementation, monitoring, and operations.
• Work with cross-functional teams to improve AI coding results against performance benchmarks.

This is a contractor engagement for candidates based in the United States. The initial term is one month, with possible extensions based on performance and fit.

• Schedule: 20 or more hours per week with flexible scheduling.
• Language: Professional fluency in English.
• Experience: At least three years of software engineering experience.
• Technical background: Strong systems programming, infrastructure, or backend development experience, including scalable, production-grade software.
• Core skills: Software architecture, design, development, debugging, code-quality assessment, and clear written and verbal technical reasoning.