Software Engineering AI Evaluation Specialist

You will create reproducible environments for software engineering problems and develop golden reference solutions. Your work will help evaluate AI models that fix code, build features, refactor systems, and improve performance.

You will contribute code samples, debugging strategies, technical explanations, and reviews that can be used as AI training data.

• Build reproducible environments for complex software engineering problems.
• Create and document reference solutions for coding tasks.
• Analyze defects and performance bottlenecks in varied codebases.
• Implement maintainable features and improve legacy code for clarity, efficiency, and reliability.
• Explain technical reasoning, solution methods, and engineering decisions.
• Review peer-contributed code and technical submissions for accuracy and clarity.

What It Pays And Takes

This is a remote, part-time contractor role. Previous AI experience is not required, but you need strong hands-on software engineering experience and the ability to assess complex coding work.

• $50 to $100 per hour equivalent, paid per task that meets the project specifications.
• Approximately 15 hours per week, with less than 20 hours expected.
• Open to candidates in the countries listed for this role, including the United States, Canada, the United Kingdom, India, Japan, Singapore, and other supported locations.
• English fluency is required.
• Significant hands-on expertise in at least one of Python 3, Java, Rust, Go, C++, or TypeScript.
• Strong knowledge of algorithms, data structures, and software engineering principles.
• Experience debugging complex systems, resolving defects, and improving performance.
• Experience refactoring large codebases or modernizing legacy systems.
• Ability to explain technical concepts and document engineering decisions clearly.
• Experience delivering features from conception through completion in cross-functional projects is helpful.

How It Works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI Training Work

AI training work uses human expertise to create examples and evaluations that help artificial intelligence systems improve. OpenTrain hires and contracts contributors for this work, and software engineers are needed to test how well models handle real coding problems.