Jeffrey W.

Key Skills

Software

Other

Appen

Remotasks

Top Subject Matter

Multi-domain AI language model evaluation

factuality verification

and prompt/response quality scoring

Top Data Types

Text

Image

Document

Top Task Types

Fine-tuning

Entity (NER) Classification

Freelancer Overview

AI Language Model Evaluator & Trainer — Outlier AI (August 2025 – Present). Brings 4+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Other, Appen, and Remotasks. Education includes Master of Science, Carnegie Mellon University (2024) and Bachelor of Science, University of California, San Diego (2022). AI-training focus includes data types such as Text and Image and labeling workflows including Evaluation, Rating, and Fine-tuning.

Labeling Experience

AI Language Model Evaluator & Trainer — Outlier AI (August 2025 – Present)

OtherText

Evaluates LLM-generated outputs for factual accuracy, logical consistency, and contextual relevance across multiple subject domains. Refines prompts to improve response clarity, specificity, and alignment with diverse user intents while supporting standardized calibration with evaluation teams. Conducts systematic web-based fact-checking of AI-generated claims using authoritative sources and applies output validation and quality scoring to maintain benchmark standards. • Labeling includes accuracy, coherence, and contextual relevance judgments • Applies response quality scoring and validation/rubrics • Performs web-based claim verification • Collaborates to standardize annotation workflows and calibration criteria

2025 - Present

AI Data Trainer & Evaluator — Scale AI (December 2024 – July 2025)

OtherTextFine-tuning

Curates and annotates large-scale datasets used to fine-tune and improve next-generation language models. Performs intent-checking, structured quality scoring, and response ranking across thousands of model outputs to support model improvement cycles. Provides detailed written rationales for evaluation decisions and identifies systematic model failure patterns to improve training data quality. • Dataset curation and annotation for fine-tuning • Intent checking and instruction-following assessment • Response ranking and structured quality scoring • Written rationales and failure pattern analysis

2024 - 2025

AI Content Evaluator — Appen (January 2024 – November 2024)

AppenTextEntity (NER) Classification

Assesses AI-generated text for coherence, tone, safety, and adherence to project-specific quality guidelines. Executes annotation tasks including sentiment labeling and named-entity recognition, along with comparative response ranking. Applies domain expertise in cognitive science to evaluate model outputs related to psychology, behavior, and decision-making while meeting daily throughput targets. • Sentiment labeling and named-entity recognition • Comparative response ranking • Safety/tone/coherence guideline adherence checks • Daily throughput and consistent quality maintenance

2024 - 2024

AI Data Annotator & Evaluator — Remotasks (February 2023 – January 2024)

RemotasksImage

Annotates training data for computer vision and NLP models using rigorous validation protocols. Evaluates AI-generated text responses for factual grounding and logical soundness using structured rubrics. Completes structured LLM evaluation tasks assessing outputs for safety, helpfulness, and harmlessness while maintaining high accuracy and consistent benchmark performance. • CV and NLP training data annotation • Factual grounding and logical soundness evaluation • Safety/helpfulness/harmlessness assessment • Structured rubric-based quality validation and benchmarking

2023 - 2024

Education

Work History