AI Data Annotation & Model Evaluation Contractor (Handshake AI)
Text
Classification
TrackingAI Data Annotation & Model Evaluation contractor responsible for evaluating AI-generated responses across software development, app evaluation, coding, STEM, finance, and text-based tasks. Performed rubric-based scoring and blind model comparisons to select higher-quality outputs with evidence-backed rationales. Conducted quality and safety assessments including factuality, instruction following, reasoning quality, relevance, and hallucination/failure-mode identification. • Evaluated instruction following, factual accuracy, reasoning quality, and safety/relevance using structured rubrics. • Reviewed evidence such as screenshots, code files, command outputs, and test results to justify ratings and selections. • Identified failure modes including hallucinated claims, missing verification, prompt noncompliance, and code regressions. • Authored and reviewed realistic prompts, evaluation rubrics, and quality criteria for AI training workflows including multi-modal and agentic tasks.