AI Quality Analyst (Turkish Personalization)

About OpenTrain

OpenTrain is the centralized platform where people start and grow careers teaching AI through data-labeling and human-feedback work. We help contributors find projects, build a unified AI training portfolio, and grow a durable freelance career in a fast-growing industry.

About AI training work

AI training (data labeling / annotation) is the human side of building intelligent systems: people create, evaluate, and correct examples the models learn from. This work is remote, flexible, often entry-accessible, and gives you direct influence over how state-of-the-art AI behaves.

• 100% remote, flexible scheduling to fit around other commitments
• Accessible entry points — language fluency and attention to detail are often enough
• Work on cutting-edge personalization features that shape model behavior

The role

OpenTrain is recruiting an AI Quality Analyst to evaluate a new personalization feature in a generative AI system. This is an immediate, 3-month contract focused on Turkish-language evaluation and requires hands-on quality review of conversational personalization.

• Role: AI Quality Analyst — personalization evaluation (Turkish)
• Contract: 3 months, contractor, part-time options
• Language: Turkish (high reading and writing proficiency required)

What you'll do

You will create and evaluate conversations that test how well the model personalizes responses using a contributor's personal context. Work is centered on detailed side-by-side comparison and clear written justification for judgments.

• Design and execute multi-turn conversational prompts (1–5 turns) that require the AI to use your personal information
• Evaluate model responses for personalization quality across Grounding, Integration, and Helpfulness
• Analyze grounding issues to ensure claims are supported and not hallucinated or based on flawed inferences
• Assess Integration to confirm personal data is woven naturally into responses
• Compare and stack-rank two model responses side-by-side (SxS) and write structured rationales referencing specific turns
• Maintain data hygiene by deleting evaluation conversations after review

Requirements

Candidates must meet the core skills and background needed to evaluate nuanced personalization behavior in Turkish. Preserve attention to detail and strong written reasoning in every judgment.

• High proficiency reading and writing in Turkish (required)
• Exceptional analytical thinking for ambiguous or subtle AI behaviors
• Experience designing creative, multi-turn prompts based on personal context
• Understanding of personalization concepts: Grounding, Integration, Helpfulness, and common failure modes
• Attention to detail for SxS review and spotting subtle differences in naturalness or overnarration
• Strong written communication to produce concise, evidence-based rationales
• BS/BA degree or equivalent experience in a relevant field (Policy, Law, Ethics, Linguistics, Journalism, Computer Science, etc.)
• Experience in data annotation, AI quality evaluation, or content moderation strongly preferred
• Desktop or laptop and a reliable internet connection

Helpful background

The following experience is not required but will help you succeed quickly in this role.

• Prior experience evaluating model responses or working on personalization features
• Familiarity with side-by-side (SxS) evaluation methodologies
• Comfort working in a 24-hour global operations environment

Compensation & schedule

This is a contractor position paid hourly. Scheduling requires reliable overlap with a West Coast time zone window to enable coordination with the ops team.

• Pay: $15.00 USD per hour
• Minimum commitment: 30 hours per week (options of 30 or 40 hrs/week)
• Schedule: 4 hours overlap required with Pacific Standard Time (PST)
• Contract length: 3 months
• Work location: Fully remote

How to apply and next steps

Create an OpenTrain account, complete your profile, and apply for this project. Qualified applicants will be asked to complete a short qualification or sample evaluation and onboarding to learn the specific rating rubrics and SxS procedures.

• This is contractor work managed by OpenTrain AI — you'll receive instructions, rubrics, and evaluation tasks after onboarding
• Expect to demonstrate Turkish writing ability and prompt-design skills in the qualification