Outlier annotation training task: prompt/rubric creation and response evaluation workflow.
Completed an Outlier “Introduction to Outlier” guided workflow focused on AI prompt/response evaluation. Built prompts and rubrics, then rated model responses according to rubric criteria. Performed comparative selection by ranking responses using the same rubric. • Wrote prompts in structured conversational style (natural follow-ups). • Created rubric criteria across multiple dimensions (e.g., helpfulness, truthfulness, writing style). • Evaluated responses against rubric and noted guidance about using “Partially” sparingly. • Selected the best response from side-by-side outputs based on rubric.