Social Scenario AI Evaluation Generalist

The work

You will help evaluate how well a language model handles realistic social situations. You will create scenarios, interact with the model under assigned conditions, and judge whether its responses fit the context and the person’s needs.

• Create realistic, ongoing social scenarios involving personal, professional, or interpersonal situations.
• Interact with a language model under assigned evaluation conditions.
• Rate responses for relevance, usefulness, personalization, clarity, and fit with the provided context.
• Explain the reason for each rating through clear, detailed written feedback.
• Follow project guidelines and complete assigned tasks within the required timeline.

What it pays and takes

This is an intermediate contractor role for a short-term evaluation project. Three or more years of experience in data annotation, human-AI interaction, or content evaluation is preferred but not required.

• Pay: Pay per task in USD; no fixed rate is provided in the listing.
• Project length: 2-3 weeks.
• Time: 20+ hours per week; the project is described as a full-time engagement.
• Location: Remote and open worldwide.
• Language: Strong written English proficiency is required.
• Background: Education or professional experience in business analysis, English, literature, journalism, creative writing, scriptwriting, content writing, film and video captioning, or a related field.
• Skills: Clear communication, creative and analytical thinking, and the ability to understand nuanced interpersonal situations and rate AI responses objectively.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind systems that generate and understand language. Contributors create examples and review model responses so AI can become more useful, clear, and appropriate in different situations.