Bilingual AI Response Evaluation, RLHF & Quality Review
Worked on bilingual (English/Polish) AI training and RLHF evaluation tasks involving comparative assessment, testing, rating, validation, and quality review of AI-generated responses. Work included, amongst others, evaluating outputs for relevance, clarity, factual consistency, reasoning quality, localization, instruction adherence, and overall response quality using detailed project guidelines, as well as developing challenging prompts designed to test model limitations and failure cases. Tasks required consistent application of evaluation criteria across nuanced language and reasoning scenarios, careful interpretation of complex instructions, and strong analytical judgement, accuracy, consistency, and attention to detail. Experience also included identifying inconsistencies, ambiguities, and other quality issues within AI-generated content and contributor evaluations, assessing evaluation reliability, and ensuring annotation standards and project guidelines were applied consistently across language-based tasks.