Evance O.

Key Skills

Software

CloudFactory

Data Annotation Tech

Dataloop

CVAT

Clickworker

Appen

Figure Eight

iMerit

Labelbox

Lionbridge

Micro1

Remotasks

Top Subject Matter

LLM response evaluation

RLHF-style rubric evaluation

and agentic reinforcement learning task design

Top Data Types

Text

Image

Video

Top Task Types

Data Collection

RLHF

Classification

Bounding Box

Segmentation

Entity (NER) Classification

Point/Key Point

Polygon

Polyline

Cuboid

Object Detection

Text Generation

Question Answering

Text Summarization

Fine-tuning

Red Teaming

Transcription

Evaluation/Rating

Computer Programming/Coding

Function Calling

Prompt + Response Writing (SFT)

Freelancer Overview

I've spent about three years, mostly full-time, working in AI training and evaluation with DataAnnotation Tech.I've spent about three years, mostly full-time, working in AI training and evaluation with DataAnnotation Tech. My focus is on reviewing model responses for accuracy, helpfulness, and instruction-following, and developing the rubrics that reviewers use to score work consistently. This spans single responses as well as longer multi-turn conversations and agentic tasks built around realistic scenarios, across text, image, and video. Earlier, at CloudFactory, I began as a data annotator on projects involving self-driving, medical imaging, and inventory systems, and later led a quality assurance team. That experience gave me a solid grounding in annotation and in identifying quality issues early, before they affect downstream results.

Labeling Experience

Senior Reviewer and Production Coordinator - DataAnnotation.Tech

TextRLHF

Evaluated AI model responses for instruction-following, factual correctness, helpfulness, and safety against the user’s intent and reasonable alternatives. Designed realistic evaluation scenarios with nuanced constraints and measurable success criteria. Applied deep understanding of prompt engineering, quality assurance, and automated rubric-based assessment to support production review workflows. • Reviewed model outputs across correctness, helpfulness, and safety • Built multimodal evaluation tasks with strict output formats • Authored and edited rubrics for single-turn and long-context conversations • Coordinated with teams as a senior reviewer and production coordinator

2021 - Present

AI Training and Evaluation (Independent Contractor, Remote)

Don't discloseTextFine-tuning

Evaluated language model responses for instruction-following, factual correctness, helpfulness, and safety against the primary user intent and plausible alternatives. Designed and applied rubric-based assessments for single-turn and long-context conversations, including dimensions like coherence, memory/inference consistency, factuality, and verbosity. Created realistic multimodal and agentic evaluation tasks that require structured outputs (e.g., JSON/XML) and correct reasoning under nuanced constraints. • Response quality judging using rubrics across correctness/helpfulness/safety and instruction-following • Anti-hacking decomposition of vague prompts into atomic criteria with metadata and dodged-bullet checks • Design of multimodal evaluation tasks (image description, visual QA, and in-image analysis) with format conformance • Failure-targeted agentic RL environment scenarios to measure measurable success vs. shallow coverage

2021 - Present

QA Team Lead - CloudFactory

ImageRLHF

Led quality assurance efforts for machine learning datasets and client delivery timelines while supervising a team of annotators. Partnered with project managers to streamline review workflows and address recurring quality issues at scale. Applied benchmarking and quality control practices to maintain consistent annotation standards for downstream use. • Managed annotator team performance against client deadlines • Maintained accuracy benchmarks and quality baselines • Streamlined review workflows with project managers • Addressed recurring quality issues through process improvements

2019 - 2021

Data Annotation and Quality Assurance Team Lead (CloudFactory, Remote)

CloudFactoryVideoPoint/Key Point

Performed data annotation on machine learning datasets spanning self-driving car data, medical imaging, and business inventory. Promoted from annotator to QA Team Lead, supporting accuracy benchmarks while meeting client deadlines. Worked with project management to streamline review workflows and systematically reduce recurring quality issues. • Annotated dataset examples across domains including medical imaging • Supported QA processes and quality assurance benchmarks for labeled data • Coordinated team workflow with project managers to address quality issues • Managed annotation delivery against client timelines

2019 - 2021

Education

Work History