AI Safety Model Evaluator

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We help people discover specialized projects, build a professional AI training profile, and apply in minutes. Creating an OpenTrain account is free.

• Remote contract work with an expert-level AI training focus
• A profile that helps you document relevant experience and grow your AI training career
• Access to opportunities shaping how advanced AI systems behave

About AI Safety Model Evaluation

AI training is the human side of building artificial intelligence. Evaluators review model responses and provide structured judgments that help AI systems become more accurate, useful, safe, and aligned. This work is part of a fast-growing technology field spanning RLHF, supervised fine-tuning, benchmarking, and safety evaluation.

• Assess examples produced by modern AI systems
• Identify unsafe, inaccurate, misleading, or policy-violating behavior
• Help improve model performance through consistent, evidence-based feedback

The Role

OpenTrain is recruiting an AI Safety Model Evaluator to assess the safety, quality, factual accuracy, and alignment of frontier AI model responses. You will apply structured standards to complex, policy-sensitive, and ambiguous topics, helping improve model behavior and safety performance.

The work spans high-risk content areas including misinformation, political persuasion, self-harm, violence, cyber, and biosecurity. Success requires excellent written English, strong critical thinking, analytical reasoning, and consistent judgment.

• Role: AI Safety Model Evaluator
• Work type: Remote contractor and part-time opportunity
• Experience level: Expert
• Pay: $60-$70 per hour
• Time requirement: 20+ hours per week, with a default commitment of 40 hours per week
• Language: English
• Eligible locations: United States and selected countries across Europe

What You'll Do

You will evaluate AI-generated content against safety, factual accuracy, policy compliance, and overall quality standards. Your reviews will support model alignment, safety benchmarking, and the development of stronger evaluation practices.

• Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and quality
• Review nuanced content across sensitive and high-impact domains
• Apply and refine evaluation rubrics for RLHF, supervised fine-tuning, and AI safety benchmarking
• Identify unsafe outputs, hallucinations, reasoning failures, and policy violations
• Provide structured feedback that supports stronger model alignment and safety performance
• Collaborate with AI researchers and safety teams on evaluation initiatives

Required Qualifications

Applicants must bring substantial professional experience and the ability to make careful, consistent judgments in nuanced and policy-sensitive scenarios. A bachelor's degree or higher is required in journalism, communications, psychology, sociology, public policy, law, biology, chemistry, computer science, or a related discipline.

• Bachelor's degree or higher in a listed or related discipline
• At least five years of professional experience in AI safety, trust and safety, journalism, public policy, scientific research, security, or a related field
• Excellent written English
• Strong critical-thinking and analytical-reasoning skills
• Ability to evaluate misinformation, self-harm, violence, cyber, biosecurity, and other policy-sensitive content
• Ability to identify unsafe outputs, hallucinations, reasoning failures, and policy violations
• Ability to apply evaluation standards consistently

Helpful Background

The following experience is helpful but is presented as preferred background rather than a required qualification. Familiarity with AI safety and evaluation methods can help you contribute effectively to complex review initiatives.

• Experience with AI safety, RLHF, supervised fine-tuning, trust and safety, or AI evaluation
• Familiarity with safety policies or content moderation
• Experience developing or applying evaluation rubrics
• Experience reviewing complex, high-risk, or ambiguous content

Schedule, Location, and Pay

This is remote contract work available to contractors residing in the United States, Denmark, Estonia, Finland, Iceland, Ireland, Latvia, Lithuania, Norway, Sweden, Austria, Belgium, France, Germany, Liechtenstein, Luxembourg, Monaco, the Netherlands, Switzerland, the United Kingdom, Albania, Bosnia and Herzegovina, Croatia, Greece, Italy, Kosovo, Malta, North Macedonia, Portugal, San Marino, Serbia, Slovenia, Spain, Bulgaria, the Czech Republic, Hungary, Moldova, Poland, Romania, or Slovakia.

• Hourly rate: $60-$70 USD
• Minimum expected availability: 20+ hours per week
• Default commitment: 40 hours per week
• Contractor engagement
• Remote work from an eligible country

Build Your AI Training Career With OpenTrain

AI safety evaluation lets experienced professionals directly influence how advanced models respond to difficult and consequential situations. Through OpenTrain, you can build a durable portfolio of AI training experience, show relevant expertise, and find projects aligned with your skills.

• Create a free OpenTrain account
• Build a profile highlighting your professional background
• Apply to eligible AI training opportunities in minutes
• Grow a portfolio in a rapidly developing AI industry