Kelly P.

Key Skills

Software

Data Annotation Tech

Deep Systems

Datature

Top Subject Matter

LLM training data (RLHF/SFT)

multilingual NLP annotation

evaluation & QA

Top Data Types

Text

Document

Image

Top Task Types

RLHF

Fine-tuning

Classification

Freelancer Overview

AI Training Data Specialist — Annotation & Labeling Infrastructure. Brings 8+ years of professional experience across legal operations, contract review, compliance, and structured analysis. Core strengths include Don't disclose, Internal, and Proprietary Tooling. Education includes Bachelor of Technology, Federal University of Technology, Owerri (FUTO) (2019). AI-training focus includes data types such as Text and labeling workflows including RLHF, Fine-tuning, and Classification.

Labeling Experience

AI Training Data Specialist — Annotation & Labeling Infrastructure

Don't discloseTextRLHF

Responsible for designing and labeling large instruction-response and preference datasets used in RLHF and SFT training runs for production LLMs. Built gold-standard output definitions, evaluation rubrics, and structured evaluation criteria to guide labeling quality across diverse task types. Performed multi-language annotation and validation, correcting cross-linguistic inconsistencies that could degrade multilingual model performance.• Designed annotation rubrics and gold-standard output definitions for instruction-response and preference data.• Created evaluation scenarios and task prompts to test reasoning under ambiguous and multi-step conditions.• Implemented robust data validation and testing to catch 200+ edge-case labeling failures before training.• Annotated and validated outputs across 12+ languages, improving multilingual consistency and quality.

2023 - Present

Data Labeling Engineer — Annotation Pipeline & Quality Assurance

TextFine-tuning

Built and maintained scalable annotation pipelines producing 50,000+ instruction-response pairs per project cycle for supervised fine-tuning (SFT) of domain-specific LLMs. Implemented annotation QA systems to monitor consistency, detect silent data errors, and reduce training failures. Authored guidelines and testing frameworks for varied data types and out-of-distribution inputs, improving downstream model reliability.• Developed scalable annotation pipelines for 50,000+ SFT-ready instruction-response pairs per cycle.• Designed QA workflows to detect labeling inconsistency and silent data errors, reducing failures by 40%.• Created and maintained annotation guidelines and testing frameworks for diverse and out-of-distribution inputs.• Co-authored documentation adopted as engineering standard across 80+ contributors across three concurrent projects.

2021 - 2022

NLP Data Annotator & Labeling Infrastructure Specialist

Don't discloseTextClassification

Engineered NLP data annotation and labeling pipelines for transformer-based LLM training across conversational AI, search relevance, and text classification tasks. Produced 30,000+ high-quality instruction-tuning samples while maintaining data integrity and consistency across distributed annotation workflows. Identified systemic annotation and data representation issues and documented findings that informed labeling improvements in production model releases.• Built and ran annotation pipelines for conversational AI, search relevance, and text classification workloads.• Developed and labeled 30,000+ instruction-tuning samples with strict data integrity standards.• Investigated systemic annotation and representation issues and documented improvements for production releases.• Applied parallel computing to accelerate annotation throughput, cutting preprocessing time by 25% for multilingual corpora.

2019 - 2021

Education

Work History