AI data annotation contractor (Agentic coding annotator and evaluation) at Vetto AI
Created high-quality coding and evaluation training data in agentic coding environments while keeping model blindness and session independence across runs. Verified LLM outputs by reviewing generated code, executing terminal commands, inspecting logs, and checking produced artifacts. Produced evidence-based rationales for trajectory rankings grounded in observed model behavior. • Designed multi-step coding tasks with user intent definition and milestone structure. • Built and refined task-specific rubrics and binary evaluation criteria. • Conducted model assessments using guardrails and evaluation checklists. • Collaborated on scenario design and conversation simulation workflows.