AI Operations & Delivery Lead — AI Safety and Frontier Model Training (Scale AI / Outlier)
Scale AI
Text
RLHFLed frontier model training and evaluation pipelines using RLHF and rubrics-based quality assessment for multiple assistant capabilities. Owned end-to-end labeling operations including program design, contributor workflow, quality monitoring, and risk mitigation for safety and performance. Coordinated red teaming to identify vulnerabilities and implemented adversarial testing protocols to improve training outcomes.• Managed RLHF and evaluation label schemas across instruction following, conversational quality, factuality, and cultural relevance.• Directed contributor capacity planning, automated workflow allocation, and targeted feedback protocols to raise quality scores quickly.• Oversaw agentic behavior and product fine-tuning programs, including audio and image annotation coordination as part of multimodal pipelines.• Produced stakeholder reporting and monitoring for roadmaps, timelines, SLAs, and crisis response.