SuperAnnotate | Data Annotator/QA Expert (AI Evaluation Specialist focus)
Evaluated AI-generated text responses for quality, coherence, factual accuracy, and adherence to safety guidelines using structured rubrics across high-volume task cycles. Identified and escalated harmful, biased, or policy-violating content, maintaining consistent judgment standards even for ambiguous edge cases. Produced detailed written rationale for preference decisions using side-by-side comparisons of AI response pairs to support model improvement. • Quality and safety rubric-based evaluation • Harmful content and policy violation identification/escalation • Side-by-side response preference comparison with written rationale • Iterative feedback incorporation to align with evolving platform guidelines