Generalist AI Expert — Model Response Evaluation (MRN)
Mercor
Text
RLHFAs a Generalist AI Expert, I assessed LLM-generated responses using a structured multi-dimensional rubric, contributing expert annotations to reward model training for advanced AI systems. My role involved rigorous fact-checking, logical soundness verification, and written justifications across diverse subjects. I demonstrated high consistency and quality under regular performance monitoring.• Performed pairwise evaluation of AI model outputs across multiple rubrics and dimensions. • Conducted comprehensive fact-checking and multi-step logical analysis of model responses. • Authored concise, rationale-driven preference annotations for learning model enhancement. • Adhered to strict quality metrics while managing a high volume of evaluated tasks.
Computer Programming/Coding
Evaluation/Rating