AI Coding Agent Evaluation Expert

About OpenTrain

OpenTrain AI is the hiring and contracting organization for this role and the #1 platform for finding and building careers in AI training and data labeling. We help people discover opportunities, build a professional profile, and grow lasting careers in a fast-moving industry.

About AI Training Work

AI training is the human side of building modern artificial intelligence. Experts review outputs, provide structured feedback, and contribute real-world examples that help AI systems become more capable, reliable, and useful.

• Work remotely with flexible, part-time contractor arrangements
• Apply your professional expertise to cutting-edge AI systems
• Help shape how AI coding tools perform in real software development

The Role

OpenTrain AI is recruiting an AI Coding Agent Evaluation Expert to support AI training focused on coding-agent evaluation. You will use AI coding workflows, technical walkthroughs, and structured feedback to help assess how AI systems handle sophisticated programming challenges.

Prior AI training experience is not required. The most important qualification is strong real-world knowledge of software development, STEM, or applied AI coding-agent use.

• Contractor role with part-time engagement
• 20+ hours per week
• Asynchronous remote work
• Compensation of $80–$100 per hour
• English-language work

What You'll Do

You will contribute detailed technical insights and real-world examples of using AI coding agents such as Copilot in complex software development workflows. Your work will combine hands-on technical reasoning, documentation, testing, and clear evaluation of agent performance.

• Document AI coding-agent use cases involving large codebase navigation
• Explain challenging debugging workflows and architectural changes
• Describe end-to-end feature development supported by AI coding agents
• Explain your decision-making process and the tools used
• Assess how AI agents affect productivity and code quality
• Submit structured feedback on agent effectiveness, limitations, and potential improvements
• Review, test, and validate AI-generated technical work
• Communicate clearly and professionally in written and verbal formats
• Work asynchronously and independently with distributed teams

Required Skills And Experience

This role is listed at the entry level, but it requires current, practical experience applying AI coding agents to sophisticated software development work. You should be able to reason through ambiguous technical problems and explain complex decisions in a structured, understandable way.

• Relevant professional or academic expertise in software development, STEM, or AI coding-agent use
• Current experience applying AI coding agents to real software development work
• Ability to navigate large codebases
• Ability to reason through difficult debugging and architectural refactoring problems
• Familiarity with multiple AI coding tools and their strengths and limitations
• Experience documenting technical workflows or providing collaborative feedback
• Ability to document end-to-end workflows and ambiguous problem-solving decisions
• Ability to review, test, validate, and clearly assess AI-generated technical work
• Strong written and verbal communication
• Careful attention to detail
• Comfort working independently and asynchronously with distributed teams

Who Should Apply

Apply if you have current experience using AI coding agents in demanding software development settings and can translate your technical reasoning into precise, useful evaluations. This opportunity is especially suited to people who understand both the practical realities of software engineering and the strengths and limitations of multiple AI coding tools.

• Software developers working with AI coding assistants
• STEM professionals with substantial programming experience
• Technical contributors experienced in debugging, refactoring, or large codebases
• Professionals who can provide careful written and verbal technical feedback

How It Works

As an OpenTrain AI contractor, you will contribute asynchronously through written and verbal technical work. OpenTrain helps contributors build a profile that showcases credible AI training experience, discover relevant opportunities, and develop a durable portfolio in the growing AI-training industry.

• Create or update your OpenTrain profile
• Apply for the opportunity in minutes
• Complete technical contributions remotely on an asynchronous schedule
• Build experience evaluating AI systems and documenting expert workflows