Jarosz D.

AI Workflow Engineer (Data Annotation & Evaluation Pipelines)

TolokaText

As an AI Workflow Engineer, I specialized in designing and building data annotation and evaluation pipelines for large language models using Toloka. My responsibilities included developing structured text classification, preference ranking, and response evaluation tasks with detailed worker instructions. I implemented multi-step labeling workflows, quality assurance checklists, and automated output validation systems for consistent and accurate results. • Designed Toloka annotation interfaces and golden sets for text classification and ranking • Developed prompt frameworks and evaluation scoring rubrics for LLM outputs • Built validation layers to ensure high-quality data labeling and reduce ambiguity • Integrated automated and crowdworker-based quality assurance processes