Dexter A.

Key Skills

Software

No software listed

Top Subject Matter

NER annotation for clinical/financial/compliance-adjacent enterprise domain entities

LLM evaluation

HITL human review workflows

Top Data Types

Text

Image

Document

Top Task Types

No task types listed

Freelancer Overview

Senior AI Engineer — NER Annotation, Golden Dataset Engineering, HITL Design (Accenture — Enterprise AI Platform). Brings 11+ years of professional experience across legal operations, contract review, compliance, and structured analysis. Core strengths include Prodigy (active learning), LangSmith (golden dataset, and eval mgmt). Education includes Bachelor's Degree, Notre Dame de Namur University (2016). AI-training focus includes data types such as Text and labeling workflows including Entity (NER), Evaluation, and Rating.

Labeling Experience

Senior AI Engineer — Golden Dataset & HITL Human Evaluation, Preference Ranking (Accenture — Enterprise AI Platform)

Text

Designed and maintained LLM evaluation datasets and CI gates, including stratified golden dataset construction with adversarial query buckets. Built HITL human evaluation workflows for clinical and compliance-sensitive outputs using LangGraph with structured reviewer actions. Delivered preference ranking and pairwise evaluation annotation programs to support prompt regression testing and A/B decisions. • Maintained a 100-query golden eval suite with explicit expected outputs, evaluation dimensions, and automated CI blocking thresholds. • Engineered HITL review queues where reviewers approve/modify/reject with confidence selection, source verification, and hallucination flags. • Conducted preference ranking annotation with calibrated anchor examples and Kappa gating to keep annotator drift under control. • Used quarterly refresh and bad-regression discovery via reviewer feedback to drive golden dataset updates and prompt iteration cycles.

2024 - 2026

Senior AI Engineer — NER Annotation, Golden Dataset Engineering, HITL Design (Accenture — Enterprise AI Platform)

Text

Led an enterprise NER annotation program for domain-specific entity extraction and coverage engineering. Built Prodigy-based active learning to prioritize uncertain spans, while establishing annotator guidelines and inter-annotator agreement calibration using Cohen's Kappa. Maintained label quality through held-out evaluations and per-entity-type quality reviews that informed targeted annotation batches. • Labeled 2,400 documents across 12 entity types using Prodigy (NER annotation) and custom span rules. • Authored annotation guidelines including boundary decision trees, exclusion logic, and structured disagreement resolution. • Implemented an active learning loop (uncertainty/confidence thresholds) to reduce annotation volume while improving entity recall. • Ran weekly/bi-weekly calibration and monthly IAA tracking, improving Kappa from 0.71 to 0.89 and entity recall to 91% on held-out tests.

2024 - 2026

AI/ML Engineer — LLM Evaluation Dataset Curation (Informulate)

Text

Created and maintained a golden evaluation dataset for an Azure OpenAI RAG platform to support prompt regression testing and pre-deployment quality gates. Ran weekly bad-answer triage sessions to classify failure modes and feed recurring issues into an adversarial evaluation bucket. Used root-cause distributions and retrieval/generation/prompt attribution to drive iterative pipeline improvements. • Built a 50-query golden dataset across eight query types with domain-expert expected outputs and minimum quality scores. • Implemented weekly logging and review of thumbs-down reports from 50+ daily users, capturing query/response/user feedback. • Classified failures into retrieval, generation, prompt, or expectation mismatch categories and tracked changes over time. • Promoted repeated bad examples into adversarial evaluation sets after independent occurrence to prevent regressions.

2019 - 2023

AI/ML Engineer — NLP Annotation, Eval Dataset Engineering (Informulate)

Text

Built and operated an NER annotation pipeline for an enterprise knowledge platform across multiple verticals, using Prodigy active learning. Defined multi-vertical entity schemas, ran seed annotation to bootstrap model baselines, and used uncertainty-driven batch selection for continued annotation. Measured annotation quality through iterative retraining and fixed held-out evaluations to identify lagging entity types. • Labeled 2,400+ documents with entity types including project entities, policy references, technical terms, and person-role-organization triples. • Used Prodigy uncertainty sampling and confidence-based span review to reduce redundant labeling and improve recall on validation. • Implemented a detailed entity boundary guide to resolve compound/hyphenated term ambiguity and improve precision on technical entities. • Tracked per-entity-type precision/recall trends after each annotation batch and added targeted batches for low-performing types.

2019 - 2023

Education

Work History