Research Evaluation Specialist

You will create rigorous question-and-answer materials that test whether advanced AI systems can reason well, use sound methods, combine evidence, and give accurate answers. The work is independent and requires research expertise in your discipline.

You will use primary literature and authoritative references, test questions against AI systems, and improve items that can be answered through shortcuts or oversimplification.

• Create original, high-difficulty questions and defensible answers in your area of expertise.
• Research primary literature and authoritative references, documenting citations, reasoning, and supporting evidence.
• Design questions that require advanced reasoning, methodological judgment, or synthesis across multiple sources.
• Test questions against AI systems and strengthen items that allow shortcuts or overly simple answers.
• Write precise, clear, and unambiguous question-and-answer pairs for reliable evaluation.
• Apply reviewer feedback and follow project guidelines and quality standards.

What it pays and takes

This is part-time independent contract work for researchers and other specialists contributing expert knowledge to AI training and evaluation. Prior AI experience is helpful but not required.

• $30 to $60 per hour equivalent, paid per task that meets the project specifications.
• Less than 20 hours per week.
• Open to candidates in selected countries across North America, South America, Europe, Asia, Africa, and the Middle East.
• English language work requiring excellent written English and precise explanations.
• Completed PhD, active doctoral candidacy, or equivalent research experience as a specialist, researcher, or professor.
• Deep subject-matter expertise, scholarly research experience, and familiarity with primary literature.
• Strong analytical thinking, close attention to detail, and experience sourcing and comparing authoritative information.
• Ability to write challenging, original, methodologically sound questions.
• Self-direction and reliability when delivering expert-level work independently in a remote setting.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind modern AI systems, including writing questions, reviewing model answers, and judging whether responses are accurate and well reasoned. People with deep subject knowledge help create reliable examples and evaluations that show AI systems where they need to improve.