Skip to content
OpenTrain AIFor AI Companies

Healthcare AI Rubric Evaluation Specialist

Apply now

Job details

Pay

$100 USD/hr

Engagement

Contract, part-time

Location

Remote

United States only

Posted

Sep 30, 2026

OpenTrain on Trustpilot

The work

About OpenTrain. OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role. OpenTrain is recruiting a Healthcare AI Rubric Evaluation Specialist to support AI training through expert question writing, clinical-content evaluation, and detailed rubric design. You will use healthcare expertise to define accurate, complete, safe, and professionally appropriate responses while identifying reasoning gaps, clinically important omissions, unsupported assumptions, and inappropriate certainty.

What you will do. You will create challenging healthcare questions that expose limitations in advanced AI responses, design scoring rubrics covering essential concepts, reasoning steps, caveats, acceptable alternatives, and critical errors, and evaluate AI-generated clinical content for factual accuracy, completeness, safety, reasoning quality, calibration, and professional relevance. You will distinguish minor omissions from failures that materially affect clinical quality or safety, critique rubrics created by other healthcare professionals, and develop expert reference answers, explanations, critiques, and gold-standard content for RL and SFT workflows.

Required skills. You should hold a professional degree, license, credential, or recognized qualification in medicine, dentistry, pharmacy, nursing, or another healthcare discipline. You should have demonstrated expertise in a clearly defined healthcare specialty or subspecialty, at least three years of professional or clinical experience, strong professional English proficiency at C1 level or higher, and the ability to create challenging questions requiring genuine expertise and nuanced judgment. You should be able to write detailed rubrics and assess clinical reasoning, safety, completeness, and appropriate uncertainty while identifying clinically important omissions and acceptable alternative approaches.

Helpful background. Experience with medical education, examination writing, competency assessment, clinical guidelines, teaching, peer review, quality assurance, AI evaluation, RLHF, SFT, data annotation, model benchmarking, or rubric-based scoring can help you succeed.

Work arrangement and pay. This is a remote hourly contractor opportunity for healthcare professionals in the United States. The advertised rate is up to $100 per hour. OpenTrain helps freelancers consolidate AI training opportunities across platforms and build a durable AI training portfolio.

Requirements

  • Languages: English

Open to applicants in

  • United States

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmuns5xt000020agmmcrnt6yc

Keep exploring

Similar open roles

View all AI training jobs

Medical AI Evaluation Researcher

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Medical/Health ResearchEnterprise AIBenchmarkingAgentic Medical Workflows
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $96.15–$120.19/hr

Posted Nov 18, 2026

Registered Nurse AI Evaluation Expert

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Text
Remote · United Arab Emirates, Antigua and Barbuda, Albania +92 more
English
Flexible hours
Hourly · $68/hr

Posted Nov 7, 2026

Healthcare Quality Specialist Expert

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Text
Remote · United Arab Emirates, Antigua and Barbuda, Albania +92 more
English
Flexible hours
Hourly · $80–$120/hr

Posted Nov 7, 2026

Health Information Management AI Evaluation Expert

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Documents
Remote · United Arab Emirates, Antigua and Barbuda, Albania +92 more
English
Flexible hours
Hourly · $70–$95/hr

Posted Nov 7, 2026

Clinical Researcher AI Evaluation Specialist

Review AI-generated clinical research and biomedical content, rate model responses, and write expert explanations. This remote contractor project offers up to $40 to $70 per hour, requires 20+ hours weekly, and needs 5+ years of clinical research experience.

BiasBias & confoundingBiostatisticsClinical trial designConfoundingData interpretation
Generative AI & RLHF
Text
Remote · Bangladesh, Bhutan, Brazil +14 more
English
Part-time · Flexible
Intermediate level
Hourly · $40–$70/hr

Posted Jul 8, 2026