SRE Evaluator — Reliability & Incident Assessment

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We help people start and grow freelance careers teaching AI: discover projects, build a unified portfolio, and apply in minutes — all from a single trusted place.

Why AI training matters

AI training is the human work that shapes how modern models behave: people prepare, review, and rate examples so models learn useful, accurate, and reliable outputs. This work is highly flexible, remote, and accessible — and contributors directly influence how cutting-edge systems perform in real-world tasks.

The role

As an SRE Evaluator you'll assess AI-generated documents, spreadsheets, and slide decks for accuracy, rigor, and domain quality. You will apply incident-management and reliability judgment to rate outputs against defined rubrics, identify factual or presentation issues, and deliver clear, structured written feedback that helps improve model outputs.

• Contract, part-time role: 20+ hours per week (remote, worldwide).
• Work type: document evaluations and evaluation ratings (EVALUATION_RATING) for DOCUMENT data.
• Employment relationship: contractor; schedule flexibility typical for project-based evaluation work.

What you'll do

• Evaluate AI-generated work products (documents, spreadsheets, slide decks) against domain-specific quality rubrics.
• Identify factual errors, presentation and aesthetic issues, clarity problems, and gaps in logical rigor.
• Provide concise, structured written feedback and justification tied to rubric criteria.
• Apply subject-matter judgment in incident management, reliability, and SRE contexts to assess correctness and operational suitability.

Requirements

• 5+ years of professional experience in incident management, site reliability, or SRE.
• Native or professional fluency in English (written and verbal).
• Proven ability to judge factual, presentation, and quality issues using rubrics.
• Strong Microsoft Office and Google Workspace skills, especially Slides/PowerPoint and spreadsheet tools.
• Experience evaluating AI-generated documents, spreadsheets, or slide decks.

Helpful background & who should apply

Candidates with deep SRE or reliability engineering experience who enjoy reviewing technical deliverables and giving actionable feedback will be a strong fit. An advanced degree from a reputable institution is a plus but not required if your professional experience demonstrates the needed judgment.

• Expert-level incident-management, reliability, or SRE practitioners seeking flexible, remote evaluation work.
• People who can translate technical review into clear written guidance tied to rubric criteria.

Compensation, schedule & how to apply

Pay is hourly: $80–$120 per hour (posted top rate up to $120/hr). This is a contractor, part-time engagement with a minimum expectation of 20+ hours per week and flexible scheduling.

OpenTrain handles hiring and contracting for this role. Apply through your OpenTrain profile and complete onboarding tasks, training, and a short qualification assessment to begin assigning evaluations.

• Languages required: English (en).
• Worldwide applicants accepted; work is fully remote.
• You'll complete a qualification task and review sample rubrics before starting live evaluation work.