Courtney S.

Key Skills

Software

Mercor

Mindrift

Top Subject Matter

Insurance domain (regulated P&C/operations)

AI search/retrieval quality and factuality

General AI content evaluation and factuality

Top Data Types

Text

Document

Top Task Types

Classification

Freelancer Overview

AI Insurance Domain Expert (Contract) - Train/optimize AI and evaluate insurance-domain outputs. Brings 7+ years of professional experience across legal operations, contract review, compliance, and structured analysis. Core strengths include Mercor, Handshake AI, and Telus Digital. Education includes Bachelor of Arts, University of Cincinnati and Associate of Arts, Oakland Community College. AI-training focus includes data types such as Text and labeling workflows including Evaluation, Rating, and Classification.

Labeling Experience

AI Search Rater (Freelance)

Text

Evaluated AI-generated search results for relevance, usefulness, and factual correctness. Compared multiple candidate AI outputs for the same user query to determine the best response using rating guidelines. Identified misinformation, bias, and low-quality answers to support quality improvement. • Rated search quality using relevance/usefulness/factual correctness criteria • Performed head-to-head comparisons of AI outputs for query resolution • Flagged misinformation, bias, and quality issues • Ensured consistent evaluation accuracy in remote freelance work

2026 - Present

AI Insurance Domain Expert (Contract) - Train/optimize AI and evaluate insurance-domain outputs

MercorText

Evaluated and refined AI outputs for insurance workflows, policy structures, and operational processes. Assessed responses for regulatory accuracy, compliance alignment, and correct insurance-domain terminology while identifying workflow gaps and edge cases. Provided guidance intended to improve model reliability and real-world standard adherence for downstream use. • Rated or reviewed AI outputs against regulatory and industry criteria • Translated domain knowledge into structured, model-ready guidance • Flagged inconsistencies, missing coverage, and edge cases affecting performance • Ensured outputs reflected real-world insurance standards and terminology

2026 - Present

AI Data Evaluator - factual accuracy and guideline adherence assessment

Text

Evaluated AI-generated content for factual accuracy, clarity, tone, and adherence to provided guidelines. Conducted fact-checking and source verification across claims and contextual details to ensure correctness. Identified inconsistencies, unsupported assertions, and policy deviations and returned structured feedback to improve future model outputs. • Verified factual claims and contextual details using sources • Assessed outputs for clarity, tone, and guideline compliance • Flagged inconsistencies and unsupported statements • Provided structured feedback for model performance improvement

2025 - Present

AI Data Annotation & Evaluation Specialist (Freelance Projects)

TextClassification

Labeled and evaluated AI outputs for accuracy, relevance, and intent alignment. Applied strict annotation and quality standards across evaluation tasks to ensure consistent labeling results. Supported training data refinement through high-precision review of outputs used for model improvement. • Labeled outputs for accuracy and relevance • Assessed intent alignment with user/task requirements • Followed strict annotation quality standards • Performed high-precision review to refine training data

2026

AI Content Evaluator (Freelance Projects)

MindriftText

Reviewed AI-generated content for accuracy and reasoning quality. Assessed outputs for bias, ambiguity, and unclear framing, then provided evaluator notes for model improvement. Focused on identifying quality and correctness issues that degrade user trust. • Evaluated correctness and quality of reasoning • Assessed bias and clarity of framing • Identified ambiguity and quality gaps • Delivered evaluator notes to enable model updates

2026

Education

Work History