Senior Software Engineer, LLM Evaluation

You will create and assess software datasets used to train and benchmark large language models. Your work will focus on making AI-generated code correct, efficient, scalable, reliable, and ready for production.

You will work across systems programming, backend development, infrastructure, and software architecture. You will also help test model performance across the full software lifecycle, from prototypes and architecture to APIs, deployment, monitoring, and maintenance.

• Curate code examples and write precise solutions for model-training and benchmarking datasets.
• Correct and improve AI-generated code in Python, C/C++, Rust, Go, Java, and JavaScript, including ReactJS.
• Review systems, infrastructure, and backend code for correctness, performance, scalability, and reliability.
• Build agents that check code quality, find error patterns, and test solutions to software engineering tasks.
• Write clear, structured reasons for technical evaluations and help improve coding systems against industry benchmarks.

The listing does not provide a pay rate. This is a flexible contractor engagement for candidates based in the United States.

• Time: 20 or more hours per week.
• Term: One month initially, with potential extensions based on performance and fit.
• Language: Fluent English.
• Experience: At least three years of professional software engineering experience.
• Technical background: Strong systems programming, infrastructure, or backend development skills, especially with Python, C/C++, Rust, or Go.
• Required ability: Experience building and deploying scalable, production-grade software, plus strong skills in architecture, design, development, debugging, and code review.