Medical AI Clinical Reasoning Evaluator

You will assess how AI systems perform on medical problems and clinical decision-making tasks. The work uses your practical medical judgment and evidence-based medicine to identify gaps in AI knowledge and reasoning.

You will also work with AI researchers to clarify where models perform well or fall short. Experience turning complex clinical judgment into clear evaluation criteria, scenarios, or structured assessments is useful.

• Design systematic evaluation frameworks for medical AI systems.
• Create clinical scenarios that test AI reasoning and decision-making.
• Build assessment methods grounded in the details of clinical practice.
• Identify gaps in AI medical knowledge and clinical reasoning.
• Apply clinical judgment and evidence-based medicine to evaluate AI behavior.
• Collaborate with AI researchers to improve model performance.

What it pays and takes

This is a remote, flexible engagement for a practicing physician. Scheduling can be arranged around ongoing clinical commitments, and the engagement may be extended based on performance and fit.

• Pay: Not specified in the listing.
• Hours: 20 or more hours per week, up to 30 hours per week.
• Duration: One month, with potential extensions based on performance and fit.
• Location: Remote and open worldwide.
• Work type: Contractor and part-time.
• Language: English.
• Required: An active license to practice medicine in any specialty.
• Required: Current clinical practice experience and experience with clinical decision-making.
• Required: A strong foundation in evidence-based medicine.
• Required: Strong analytical and communication skills.
• Required: Interest in using AI in clinical practice.
• Helpful: Familiarity with medical AI, clinical decision support, or evaluating reasoning quality.

How it works

Apply on OpenTrain with your resume and then complete the application on the hiring site.

About AI training work

AI training is the human work behind systems that learn from examples, including reviewing model answers and evaluating reasoning. OpenTrain AI handles this engagement so experienced medical professionals can apply their knowledge to help improve AI systems.