AI Trainer & Model Evaluator (Freelance, Remote) — AI response evaluation, RLHF-style preference labeling, and dataset annotation
Text
RLHFPerformed AI response evaluation and preference labeling to support RLHF-style training, using side-by-side (SxS) comparisons and structured feedback. Reviewed outputs for accuracy, grounding, helpfulness, coherence, and instruction-following, and documented issues such as hallucinations and flawed inferences. Also contributed to multi-turn prompt testing by generating prompt variations and evaluating model behavior against defined intents. • Data labeled included model responses and preference/ranking judgments • Labeling involved RLHF-style reward model feedback and SxS evaluation • QA checks covered hallucination detection, factual accuracy, and fluency • Work included written rationales to justify judgments and findings
2024 - Present