AI Data Annotator & RLHF Specialist — Outlier (Scale AI)
Evaluated and ranked AI-generated responses for RLHF-style quality dimensions including instruction following, factual accuracy, completeness, tone, and safety for LLM training pipelines. Performed multi-dimension rating using consistent 1–5 rubrics while avoiding bias (e.g., halo effects) across task batches. Rewrote flawed responses with written justifications to create higher-quality training references and prompt inputs. • Ranked responses across multiple quality dimensions. • Applied 1–5 rubric scoring with quality consistency. • Authored diverse prompts to expand training coverage. • Rewrote and justified improvements to weak model outputs.