Toloka Annotators — LLM response evaluation and rubric grading
Evaluated large language model responses by applying predefined guidelines and rubrics to compare and assess output quality. Synthesized information across large volumes of model responses, flagging edge cases and recurring failure modes. Produced structured written rationales to support benchmarking and inform product and model refinement. • Compared LLM outputs against rubric-based guidelines. • Flagged edge cases and identified recurring patterns or failure modes. • Provided detailed structured justification for scoring decisions. • Performed quality-focused benchmarking support and QA documentation.