Software Engineering AI Evaluator

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We help people discover AI training projects, build a unified portfolio of experience, and grow a durable freelance career teaching AI systems through hands-on evaluation and annotation work.

• Work remotely and build experience that companies recognize across the AI training ecosystem.
• OpenTrain focuses on practical, project-based roles that let you demonstrate and expand technical expertise.

About AI training work in engineering

AI training (data labeling / human feedback) is the human side of building modern AI: people review outputs, rate technical solutions, and create examples that models learn from. For software engineering tasks, contributors read code, judge implementations, point out bugs, and explain tradeoffs—work that directly shapes how engineering-focused AI behaves.

• Typical tasks include code review, debugging exercises, architecture evaluation, and written technical explanations.
• Most projects are remote and flexible—ideal for part-time contributions alongside other work.

The role

OpenTrain is recruiting experienced software engineers to act as AI evaluators for engineering-focused tasks. You will review, rate, and explain technical solutions spanning backend, full‑stack, systems, and infrastructure problems. Work is contractor, part‑time, and designed for engineers with strong practical experience and the ability to communicate technical reasoning clearly.

• Labeling focus: computer code / programming evaluation and question-answering tasks.
• Employment type: contractor, part‑time; worldwide applicants accepted; English required.

What you'll do

You will work on assessment tasks that require reading unfamiliar code, diagnosing problems, and judging implementation quality. Tasks emphasize clear, objective written explanations of why a solution works, what issues exist, and what alternatives or tradeoffs were considered.

• Review, debug, and rate code across different languages and stacks (backend, full‑stack, services).
• Evaluate APIs, databases, integrations, deployment workflows, and testing strategies.
• Explain technical tradeoffs around scalability, maintainability, performance, reliability, security, and developer experience.
• Collaborate with project teams on technical reviews and problem‑solving exercises.

Requirements

All requirements below come from the role description and are required for effective participation.

• 3+ years of hands‑on software engineering experience.
• Experience in at least one backend or full‑stack environment such as Python, JavaScript/TypeScript/Node.js, Java, C++, Go, or Ruby.
• Experience building, maintaining, or reviewing production applications, APIs, services, databases, or integrations.
• Strong understanding of debugging, testing, code quality, architecture, and technical tradeoffs.
• Comfort reading and reasoning through unfamiliar codebases and technical requirements.
• Ability to explain complex engineering decisions clearly and objectively.

Helpful background (not required)

These skills make you especially well suited for evaluation tasks but are optional.

• Cybersecurity or SecOps exposure.
• Cloud experience with AWS, GCP, or Azure and familiarity with CI/CD, containers, and production operations.
• Experience with frontend frameworks (React, Next.js, Angular, Vue, React Native).
• Open‑source contributions, public GitHub work, technical writing, or prior mentoring/code review experience.

Compensation, hours, and logistics

This contract role is paid hourly and intended to be less than 20 hours per week. The listing indicates pay up to $75/hour (hourly range shown: $20–$75/hr). Tasks involve evaluating computer code and answering questions or providing ratings via the project's labeling tools.

• Time commitment: less than 20 hours/week, flexible scheduling.
• Pay: hourly, up to $75/hr (hourly range $20–$75/hr).
• Labeling types: evaluation/rating and question‑answering on programming/code tasks.

Who should apply and next steps

Experienced software engineers who enjoy clear technical writing, code review, and practical problem solving should apply. Cybersecurity, cloud, or DevOps experience is a strong plus but not required. If you meet the requirements and want flexible, remote contract work that helps shape engineering AI, create or update your OpenTrain profile and apply to this project.

• This role is open worldwide; English is required.
• Prepare examples of relevant engineering work (GitHub links, public writing, or descriptions) to highlight your experience.