Personalized AI Response Evaluation Analyst

You will test an AI personalization feature by creating prompts based on personal experiences and reviewing how well the system uses personal context. The work combines conversational prompt design, response comparison, and clear written evaluation.

You will use multi-turn conversations, usually one to five turns, and judge whether responses are relevant, helpful, well-grounded, natural, and easy to use. You will explain your decisions with specific references to the conversation.

• Design and run prompts that test the use of personal context.
• Compare and rank AI responses side by side.
• Find unsupported claims, poor inferences, forced connections, incorrect personalization, and unnatural explanations.
• Write clear rationales that describe response strengths and quality problems.
• Check debug information, summaries, and data sources for appropriate use.
• Maintain clean evaluation histories and provide detailed annotations and constructive feedback.

What it pays and takes

This is a remote, part-time contractor role open worldwide. The listing does not provide a pay rate.

• Time requirement: 20 or more hours per week.
• Language: Excellent English reading and writing skills.
• Experience level: Entry level.
• Work setup: A desktop or laptop and a reliable internet connection.
• Required strengths: Analytical judgment, careful attention to detail, creative prompt design, and comfort evaluating ambiguous AI responses.
• Personal data: You must use a primary personal account and relevant personal data sources for genuine evaluation.
• Helpful background: A bachelor's degree or equivalent experience in policy, law, ethics, linguistics, journalism, computer science, or another analytical field.
• Helpful experience: Data annotation, AI quality evaluation, content moderation, or related work.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work is the human side of building artificial intelligence. People review and rate model responses, write examples, and give feedback so AI systems become more useful, accurate, and natural.