LLM Prompt Evaluation and AI Trainer
Text
Evaluation/RatingThe project focused on the large-scale evaluation and refinement of Large Language Model (LLM) responses. The primary objective was to improve the model's ability to generate accurate, helpful, and human-like text by subjecting it to rigorous human-in-the-loop testing. This involved analyzing how the AI interpreted various user prompts and ensuring its outputs aligned with specific safety and quality benchmarks. I performed prompt evaluation and comparative analysis of AI-generated responses. My tasks included scoring model outputs based on relevance, factual accuracy, and linguistic naturalness. I was responsible for identifying and logging technical flaws such as hallucinated information, unnatural phrasing, and tone inconsistencies. Additionally, I provided detailed, evidence-based justifications for my ratings to help developers understand the specific "why" behind a model’s success or failure. This was a high-volume, continuous evaluation project spanning two months. I handled hundreds of unique prompt-response pairs, contributing to a vast dataset used to fine-tune the model's conversational capabilities and technical reliability. I maintained high data integrity by adhering to strict Inter-Annotator Agreement (IAA) standards and Gold Standard benchmarks. My work was subject to regular quality audits to ensure consistency with the project's complex rubrics. I focused on qualitative precision, moving beyond simple binary choices to provide nuanced feedback loops that directly informed the model's iterative training process.