Bilingual Arabic–English LLM Safety Evaluator

About OpenTrain AI

OpenTrain AI is the #1 platform for finding and building careers in AI training and data labeling. We hire contractors directly to do the human work that teaches and polishes modern AI systems.

Why AI Training Matters

AI training (also called data labeling or human feedback work) is the human side of building safe, useful models. Contributors annotate outputs, evaluate model behavior, and provide the examples and judgments that shape how state-of-the-art systems respond.

This work is mostly remote, flexible, and accessible: it’s a way to do cutting-edge, impactful tech work without traditional software-engineering roles.

The Role

You will review AI-generated responses in Arabic and English and create safety-focused evaluation content. Your work helps prevent models from producing unintentional, adversarial, toxic, or unsafe responses by assessing reasoning, annotating outputs for safety issues, and documenting behaviors for engineering teams.

This is a contractor, part-time role (20+ hours/week) and is fully remote and open worldwide.

• Labeling types: evaluation ratings and text generation review on textual model outputs.
• Data type: text; work frequently involves sensitive or explicit material.

What You'll Do

Perform detailed safety evaluations of LLM responses in both Arabic and English. Apply written safety policies to label content consistently, explain edge-case decisions, and provide clear examples and feedback.

Identify adversarial prompts, document unsafe model behaviors, and write concise, actionable notes for engineers and policy teams.

• Assess reasoning quality and factual accuracy of model outputs.
• Annotate for safety categories: hate/harassment, sexual content, suicide/self-harm, violence, bias, illegal goods/services, malicious activities and code, and misinformation.
• Generate evaluation ratings and text edits or safer-response suggestions when requested.
• Record examples of adversarial attacks and describe failure modes in clear, reproducible terms.

Minimum Requirements

You must meet all listed language, experience, and skill requirements to be considered for this role.

• Near-native or native proficiency in Arabic (reading and writing).
• Minimum C1 proficiency in English (reading and writing).
• Bachelor’s degree or higher in Communications, Linguistics, Psychology, Law/Policy, Security Studies, or equivalent professional experience.
• Proven experience in Trust & Safety, content moderation, policy enforcement, risk operations, investigations, or safety evaluation.
• Hands-on LLM red teaming experience, including identifying adversarial prompts and documenting unsafe model behaviors.
• Strong domain knowledge across safety topics listed above and ability to apply written safety policies consistently.
• Comfort with reviewing explicit, toxic, violent, sexual, or psychologically disturbing content as part of daily work.

Preferred Experience & Tools

Prior experience with AI data training, annotation, or evaluation workflows is preferred but not strictly required if you have strong Trust & Safety and red teaming experience.

• Practical experience using LLMs or tools such as Perplexity, Gemini, ChatGPT, or similar systems.
• Experience producing reproducible red-team reports and structured safety annotations.

Compensation, Schedule, and Hiring

This is a contractor, part-time position. Expect to work 20+ hours per week and manage your schedule within that commitment.

Compensation is hourly, paid in USD with a posted range of $15–$40 per hour (hourlyRate field lists $25). OpenTrain AI hires and contracts directly for this role.

• Employment type: Contractor, Part-time.
• Worldwide applicants accepted; you must meet language and experience requirements.

How to Apply & What to Expect

If this role fits your skills, apply through OpenTrain AI with examples of relevant Trust & Safety, moderation, or red-team work and a summary of your Arabic and English proficiency.

During onboarding you will review safety guidelines and complete calibration tasks to demonstrate consistent application of policies. You will be asked to handle sensitive content; the role requires resilience and professional judgment.

• Be prepared to complete sample annotation or red-team exercises during evaluation.
• Successful applicants must follow strict documentation and confidentiality standards while working with sensitive material.