Psychiatric AI Evaluation Expert

About OpenTrain

OpenTrain AI is the #1 platform for finding and building careers in AI training and data labeling. We help contributors discover specialized projects, build a professional profile, and apply to opportunities that shape how modern AI systems work.

About AI Training Work

AI training is the human side of building artificial intelligence. Expert contributors evaluate model responses, identify reasoning failures, and create high-quality examples that help AI systems become more accurate, reliable, and useful.

This role applies specialist psychiatric knowledge to advanced language-model evaluation. Your clinical judgment will help assess how models handle complex cases, nuanced differentials, and safety-sensitive decisions.

The Role

OpenTrain AI is recruiting a Psychiatric AI Evaluation Expert to evaluate how large language models reason through advanced psychiatric cases. You will design expert-level diagnostic and reasoning challenges that current models fail, then provide authoritative, gold-standard answers.

The work focuses on edge cases, nuanced differentials, and risk assessment. You will also collaborate directly with research teams at a leading AI lab and provide feedback on common failure modes in model reasoning.

• Contractor position
• Part-time engagement of 20+ hours per week
• Pay: $150 USD per hour
• Remote work available in the United States, United Kingdom, or Australia
• English-language work

What You'll Do

• Develop gold-standard answers with clear clinical reasoning.
• Create advanced psychiatric diagnostic and reasoning challenges.
• Emphasize edge cases, nuanced differentials, and risk assessment.
• Evaluate large language model reasoning on complex psychiatric cases.
• Collaborate directly with research teams at a leading AI lab.
• Identify and explain common failure modes in model reasoning.

Required Qualifications

This is an expert-level evaluation role requiring completed psychiatric training, current professional standing, and substantial independent clinical experience. International medical degrees and national equivalents are accepted where specified.

• Completed psychiatry residency through ACGME, AOA, or a national equivalent.
• Medical degree: MD or DO; international degrees qualify.
• Active board certification in psychiatry through ABPN, FRCPC, MRCPsych, FRANZCP, or an equivalent body.
• Active, unrestricted medical license with no suspension or open disciplinary action.
• At least three years of independent psychiatric practice after residency.
• Current practice in an accredited or government-licensed setting.
• Experience developing clinical reasoning challenges or serving as an OSCE examiner.
• Based in the US, UK, or Australia.

Helpful Background

• At least 10 direct patient-contact hours per week in current clinical care.
• Subspecialty depth in Addiction, Child and Adolescent, Forensic, Geriatric, or Consultation-Liaison Psychiatry.
• Experience developing OSCEs or serving as an OSCE examiner.
• A faculty appointment in a department of psychiatry.

Why This Work Matters

Every major AI system depends on carefully prepared and reviewed human feedback. By applying your psychiatric expertise to model evaluation, you will contribute to the development of AI systems that reason more carefully about complex clinical scenarios and safety-sensitive questions.

OpenTrain makes it possible to build experience in this fast-growing field while taking on flexible, remote project work that fits alongside your professional commitments.