Healthcare Specialist Rubric Design

You will help train AI systems by creating difficult healthcare questions, writing detailed scoring rubrics, and reviewing AI-generated clinical content. Your work will focus on whether responses are accurate, complete, safe, well reasoned, appropriately uncertain, and professionally relevant.

• Create challenging healthcare questions that reveal weaknesses in advanced AI responses.
• Design rubrics covering essential concepts, reasoning steps, caveats, acceptable alternatives, and critical errors.
• Evaluate clinical content for factual accuracy, completeness, safety, reasoning quality, and appropriate certainty.
• Separate minor omissions from problems that could materially affect clinical quality or safety.
• Review rubrics written by other healthcare professionals.
• Write expert reference answers, explanations, critiques, and gold-standard content for reinforcement learning and supervised fine-tuning workflows.

This is a remote, part-time contractor role for healthcare professionals in the United States.

• Time: 20 or more hours per week.
• Language: Professional English proficiency at C1 level or higher.
• Qualifications: A professional degree, license, credential, or recognized qualification in medicine, dentistry, pharmacy, nursing, or another healthcare discipline.
• Experience: At least three years of professional or clinical experience in a clearly defined healthcare specialty or subspecialty.
• Skills: Create challenging expert-level questions, write detailed rubrics, assess clinical reasoning and safety, judge completeness and uncertainty, and identify important omissions and acceptable alternatives.