AI Safety Annotation Reviewer

You will audit evaluations and written assessments created for AI training. The work focuses on policy alignment, reasoning quality, scoring accuracy, required formats, and complete annotations across sensitive review tasks.

You will identify individual and recurring errors, give clear feedback, recommend corrections, and document patterns that can improve calibration, quality, and reviewer training. You will need to maintain accuracy while handling sensitive content and time-sensitive priorities.

• Review trainer-submitted evaluations for policy errors, incomplete reasoning, scoring drift, format errors, and inconsistent annotations.
• Apply detailed AI safety policies consistently across review tasks.
• Check scoring accuracy, policy alignment, required schemas, and annotation completeness.
• Provide direct, constructive feedback and recommend corrective actions.
• Document recurring error patterns and support calibration, quality improvement, and reviewer training.

This is a remote contract, part-time role with flexible working hours and a substantial weekly commitment.

• Time commitment: 20+ hours per week.
• Language: Professional fluency in English.
• Experience level: Intermediate.
• Strong written communication, attention to detail, and analytical and logical reasoning.
• Ability to interpret and apply detailed policy guidelines consistently.
• Ability to identify systematic errors in annotations and evaluations.
• Clear, constructive communication and independent work under time pressure.
• Understanding of US cultural context and communication norms.
• Useful background includes AI annotation review, trust and safety quality assurance, test engineering, content moderation, editorial fact-checking, schema validation, policy compliance review, or quality auditing.
• Experience with sensitive or high-stakes content is relevant.