Senior Python Engineer, LLM Evaluation

You will help evaluate software engineering code used to train and test large language models. The work includes creating code examples, writing or correcting solutions, and reviewing AI-generated code for correctness, efficiency, scalability, reliability, and overall quality.

You will focus on systems-level, backend, infrastructure, and performance-sensitive software. You will also help test model abilities across software design, implementation, deployment, monitoring, and ongoing operations.

• Create and curate code examples in Python, C/C++, Rust, Go, Java, and JavaScript, including ReactJS.
• Review AI-generated code for correctness, performance, scalability, reliability, and code quality.
• Build verification agents that identify error patterns in systems and infrastructure code.
• Design automated ways to verify solutions to software engineering tasks.
• Evaluate architecture, API design, production implementation, experimentation, launch, monitoring, and operational maintenance.
• Work with technical teams to improve AI coding solutions against performance benchmarks.

The listing does not provide a pay rate. This is a contractor engagement with a flexible part-time commitment and a starting duration of one month, with possible extensions based on performance and fit.

• Commitment: 20+ hours per week.
• Duration: One month initially, with possible extensions.
• Experience: At least three years of software engineering experience.
• Technical background: Strong systems programming, infrastructure, or backend development skills, especially in Python, C/C++, Rust, or Go.
• Engineering skills: Experience building and deploying scalable, production-grade software, plus strong architecture, design, debugging, and code review knowledge.
• Communication: Fluent in English and able to write clear, structured explanations for evaluation decisions.