AI Data Specialist — Independent Contractor (Remote) (AI response evaluation, ranking, and guideline-driven labeling)
Evaluated and ranked over 18,000 English AI responses using pairwise comparison and rubric-based grading. Graded outputs to improve factuality, safety, and overall quality across successive model iterations. Maintained consistent batch quality while meeting project SLAs. • Performed response preference evaluation for RLHF-style selection quality • Used rubric checks to rate model outputs • Tracked acceptance rates and throughput against SLAs • Authored and refined annotation guidelines to reduce ambiguity and disputes