Callagan S.

Key Skills

Software

Other

Appen

Clickworker

Top Subject Matter

LLM output evaluation and structured text annotation

RLHF evaluation and preference labeling for text generation

Top Data Types

Text

Top Task Types

RLHF

Freelancer Overview

AI Model Evaluator & Data Annotator (Freelance). Brings 3+ years of professional experience across complex professional workflows, research, and quality-focused execution. Core strengths include Other. Education includes Bachelor of Science, Machakos University (2026). AI-training focus includes data types such as Text and labeling workflows including RLHF, Evaluation, and Rating.

Labeling Experience

AI Output Evaluation Practice (Relevant Project)

OtherText

Rated and labeled AI output examples based on clarity, correctness, relevance, and tone as part of RLHF practice. Ranked responses and recorded preference-style labels to reflect quality differences. Used categorical quality labels with explanations to support training signals for machine learning workflows.• Labeled over 150 text examples for RLHF-style clarity/correctness/relevance/tone criteria.• Performed response ranking to generate preference signals.• Applied quality categories (poor/good/excellent) with written rationales.• Documented labeling outcomes to support downstream evaluation and training exercises.

2024 - Present

AI Model Evaluator & Data Annotator (Freelance)

OtherTextRLHF

Evaluated and scored AI-generated text outputs for accuracy, fluency, coherence, and user-intent alignment using RLHF-style feedback criteria. Classified and annotated structured text data across technical, instructional, and conversational domains. Applied rubrics to assess factuality, helpfulness, and potential biases, while resolving gray areas to maintain consistency.• Rated more than 200 AI-generated outputs using an established evaluation rubric.• Performed structured annotation of text datasets for later model training.• Monitored inter-annotator agreement by following guidelines and adjudicating inconsistencies.• Compiled labeled datasets into Excel sheets with proper labels for downstream ML workflows.

2024 - Present

Education

Work History