DataAnnotation Tech — Data Labeling Specialist (LLM Evaluation & R&R)
Data Annotation Tech
TextBilled as a DataAnnotation Tech independent contractor performing LLM output rating, comparison, and rater quality audits across multiple confidential projects. Conducted rubric adherence scoring, multi-model pairwise evaluation, and English rationale generation with translated Chinese evidence and formatting constraints. Delivered both initial ratings and R&R corrections using project-specific rules and evidence-based reasoning. •Rated multiple model responses to Chinese-language multi-turn prompts across 8+ rubric dimensions (including instruction following, completeness, factuality, fluency/localization, conciseness, and opener quality). •Ran head-to-head 4-response and 6-pairwise comparison workflows using tier-based logic and flagged items requiring expert knowledge or containing PII (CBR). •Audited other contributors’ submissions with binary (Good/Bad) and three-tier (Good/OK/Bad) schemes, flagging rubric violations such as fluff, inconsistency, factual errors, and contradictory pairwise choices. •Edited and rewrote rationales to fix format, spelling, missing evidence, and missing dimension links while preserving the original rater’s voice, tone, and required [Turn N] structure.