About OpenTrain. OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role. OpenTrain is recruiting a Healthcare AI Rubric Evaluation Specialist to support AI training through expert question writing, clinical-content evaluation, and detailed rubric design. You will use healthcare expertise to define accurate, complete, safe, and professionally appropriate responses while identifying reasoning gaps, clinically important omissions, unsupported assumptions, and inappropriate certainty.
What you will do. You will create challenging healthcare questions that expose limitations in advanced AI responses, design scoring rubrics covering essential concepts, reasoning steps, caveats, acceptable alternatives, and critical errors, and evaluate AI-generated clinical content for factual accuracy, completeness, safety, reasoning quality, calibration, and professional relevance. You will distinguish minor omissions from failures that materially affect clinical quality or safety, critique rubrics created by other healthcare professionals, and develop expert reference answers, explanations, critiques, and gold-standard content for RL and SFT workflows.
Required skills. You should hold a professional degree, license, credential, or recognized qualification in medicine, dentistry, pharmacy, nursing, or another healthcare discipline. You should have demonstrated expertise in a clearly defined healthcare specialty or subspecialty, at least three years of professional or clinical experience, strong professional English proficiency at C1 level or higher, and the ability to create challenging questions requiring genuine expertise and nuanced judgment. You should be able to write detailed rubrics and assess clinical reasoning, safety, completeness, and appropriate uncertainty while identifying clinically important omissions and acceptable alternative approaches.
Helpful background. Experience with medical education, examination writing, competency assessment, clinical guidelines, teaching, peer review, quality assurance, AI evaluation, RLHF, SFT, data annotation, model benchmarking, or rubric-based scoring can help you succeed.
Work arrangement and pay. This is a remote hourly contractor opportunity for healthcare professionals in the United States. The advertised rate is up to $100 per hour. OpenTrain helps freelancers consolidate AI training opportunities across platforms and build a durable AI training portfolio.