AI Conversation Context Evaluation Specialist (English)

You will evaluate conversations between people and AI models about interpersonal and everyday decision-making scenarios. Your feedback will help assess whether the model gathered enough context before making a recommendation.

The work covers relationships, work, education, family, planning, wellbeing, and resource-related situations.

• Review conversations using structured evaluation guidelines.
• Identify missing context and suggest useful follow-up questions.
• Assess whether more information could have changed an AI recommendation.
• Separate important, decision-changing context from descriptive or non-essential details.
• Write structured qualitative feedback and support consistent assessments.
• Consider biological, psychological, social, spiritual or existential, financial, and practical factors in complex scenarios.

What it pays and takes

This is a part-time contractor role. The role details do not specify a pay rate.

• Time requirement: 20 or more hours per week.
• Location: Open to candidates in the eligible countries listed for this role.
• Language: Fluent in English.
• Experience level: Entry level.
• Relevant experience: 3 to 5 years in psychology, social work, counseling, therapy, qualitative research, HCI, communication, conflict resolution, coaching, mediation, student services, organizational consulting, or a related advisory field.
• Qualifications may include an MSW, applicable social work license, counseling or therapy credential, PhD, PsyD, or advanced expertise in HCI or qualitative research.
• Required experience includes client assessment, interviewing, case review, conversation analysis, qualitative coding, or comparable client-facing work.
• You must be able to draw out important context from incomplete information and explain how specific details affect a recommendation.
• Helpful experience includes reviewing interviews, consultations, coaching sessions, or client interactions, along with thematic analysis, structured case review, and identifying missed information-gathering opportunities.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind AI systems, including reviewing model responses and giving feedback that helps models improve. People with experience in fields such as psychology, counseling, research, and communication can apply their judgment to make AI responses more useful and responsible.