Sophia R.

Key Skills

Software

Other

Top Subject Matter

LLM evaluation

Instruction-following Domain Expertise

safety/quality QA

Top Data Types

Text

Image

Top Task Types

Prompt + Response Writing (SFT)

Entity (NER) Classification

Freelancer Overview

Senior AI Trainer & Data Annotation Specialist. Core strengths include Other. Education includes Doctor of Philosophy, Carnegie Mellon University (2025) and Master of Arts, Princeton University (2024). AI-training focus includes data types such as Text and labeling workflows including Prompt + Response Writing (SFT), Evaluation, and Rating.

Labeling Experience

Senior AI Trainer & Data Annotation Specialist

OtherTextPrompt + Response Writing (SFT)

Created realistic, user-like prompts across general domains and evaluated model instruction-following behavior and output quality. Reviewed and edited AI-generated responses for clarity, factual correctness, tone, and usefulness to improve overall response quality. Authored human reference answers and prompt variations to support supervised fine-tuning and evaluation benchmarks. • Produced prompt variations and controlled test sets to measure nuanced output differences and edge-case behavior. • Ranked and compared multiple model responses under length/style constraints and task-specific instructions using preference-style labeling. • Identified failure cases such as hallucinations, ambiguity, and safety issues and documented findings for QA updates. • Supported evaluation workflows by generating high-quality reference content aligned with selected prompts.

2021 - Present

Linguistic Quality Assurance & Media Annotation Analyst

OtherText

Performed guideline-based QA of annotated datasets by enforcing consistency and supporting strong inter-annotator agreement. Evaluated model outputs and labeled responses for relevance, sentiment, and contextual intent across diverse content types. Flagged ambiguity and recurring error patterns to refine guidelines and reduce rework. • Conducted linguistic review of multilingual content for accuracy, tone alignment, and usability for downstream model training. • Labeled and assessed outputs for contextual intent and sentiment to improve training signal quality. • Documented ambiguous cases and recurring failure modes to contribute to clearer annotation standards. • Supported dataset quality processes focused on maintaining labeling consistency across annotators.

2019 - Present

AI Data Analyst & STEM Content Validator

OtherTextEntity (NER) Classification

Validated STEM and reasoning-focused training data by checking correctness, clarity, and logical consistency of explanations. Reviewed AI-generated responses against task instructions and rewritten prompts to improve instruction-following accuracy. Collaborated with QA leads to refine annotation criteria and reduce ambiguity-related errors in datasets. • Verified that STEM training examples and explanations met quality thresholds for downstream learning. • Improved response alignment by auditing outputs relative to task instructions and updated prompts. • Participated in dataset governance by supporting scalable growth while maintaining turnaround targets and strict quality standards. • Assisted annotation criteria refinement to reduce ambiguity and improve labeling reliability.

2017 - Present

Education

Work History