AI Data Specialist (RLHF) - Evaluating and ranking AI responses at Outlier
You evaluated and ranked AI-generated responses using RLHF-style criteria focused on helpfulness, accuracy, safety, relevance, and coherence. You followed detailed evaluation frameworks to score outputs and maintained quality under deadlines. You identified edge cases, inconsistencies, and weaknesses to improve training data quality. • Evaluate and rank responses across multiple quality dimensions. • Design and assess prompts to improve large language model performance. • Provide detailed feedback to refine AI system behavior. • Maintain high scoring consistency while meeting productivity targets.