AI Data Analyst - LLM Evaluation & Annotation Specialist
Labelbox
Text
RLHFAs an AI Data Analyst & LLM Evaluation Specialist at Scale AI, I contributed to LLM training, optimization, and evaluation using advanced annotation tools. I performed response evaluation, bias detection, and dataset quality assurance to enhance model accuracy and safety. My work included structured human feedback workflows to fine-tune and validate AI models for real-world deployments. • Conducted LLM response ranking and evaluation using rubric-based methods • Used Labelbox and CVAT for annotation of text and 2D/3D sensor data • Responsible for bias/safety detection and hallucination identification in AI-generated outputs • Performed dataset quality analysis and optimization for AI model improvement
2024 - Present