Mary P.

Key Skills

Software

Appen

Remotasks

Scale AI

Labelbox

CVAT

Toloka

Other

Top Subject Matter

LLM Training

Conversational AI

Text Data

Top Data Types

Text

Image

Document

Top Task Types

Classification

RLHF

Freelancer Overview

AI Data Annotator & Quality Reviewer (Freelance, Remote). Brings 3+ years of professional experience across legal operations, contract review, compliance, and structured analysis. Core strengths include Appen, Remotasks, and Scale AI. Education includes Nigerian Certificate in Education, College of Education, Afaha Nsit (2011). AI-training focus includes data types such as Text and labeling workflows including Evaluation, Rating, and Classification.

Labeling Experience

AI Data Annotator & Quality Reviewer (Freelance, Remote)

AppenText

As an AI Data Annotator & Quality Reviewer (Freelance), I annotated and labeled diverse text datasets, including prompts, LLM-generated responses, and conversational exchanges for AI model training and evaluation. I performed RLHF tasks, evaluated and ranked AI-generated responses, conducted multi-label text classification, and executed named entity recognition (NER) using detailed rubrics. I consistently maintained accuracy above 95% while delivering high annotation volume within tight deadlines. • Performed annotation of text, including sentiment, intent, toxicity, factual accuracy, and policy compliance categories. • Conducted NER by tagging people, organizations, locations, dates, and domain-specific entities in large corpora. • Identified and escalated hallucinated, biased, or policy-violating outputs to ensure clean training data. • Documented annotation rationale and maintained inter-annotator agreement through calibration reviews.

2024 - Present

RLHF Response Ranking Practice — LLM Evaluation

OtherTextRLHF

In my RLHF Response Ranking Practice project, I evaluated pairs of AI-generated responses using a personal scoring rubric aligned with RLHF methodologies. I assessed quality dimensions such as factual accuracy, instruction following, helpfulness, safety, and tone. I maintained structured logs documenting preference justifications for each decision, refining my judgment on ambiguous model outputs. • Ranked response pairs using structured, multi-criteria preference methods. • Applied RLHF evaluation principles as used in training large language models. • Logged decisions and preferences for transparency and quality review. • Developed expertise in RLHF workflows by practicing with publicly accessible model outputs.

2024 - 2024

Text Classification & Sentiment Labeling — Self-Directed Project

OtherTextClassification

In my self-directed Sentiment & Text Classification project, I created a personal annotation dataset with over 300 labeled samples across sentiment, intent, and topic categories. I followed industry-standard annotation rubrics and maintained structured documentation of decisions and edge cases. This practice project simulated professional workflows, focusing on consistency and calibration exercises. • Built and annotated datasets using consistent guidelines modeled after major industry platforms. • Labeled sentiment (positive, negative, neutral), intent (question, complaint, request, feedback), and topic. • Documented annotation decisions and simulated audit-ready logs. • Conducted self-calibration and consistency reviews to ensure reliability.

2024 - 2024

AI Output Evaluator & Community Knowledge Reviewer

OtherText

As an AI Output Evaluator & Community Knowledge Reviewer with Digital Witch, I reviewed and assessed AI-generated content for quality, factual accuracy, tone, and potential bias in a technical support community. I categorized and labeled responses across helpfulness, clarity, and safety to improve the available AI-assisted resources. I identified error patterns and communicated structured evaluation findings to enhance resource quality. • Applied structured annotation-style evaluation methods to community AI output. • Labeled AI responses by multiple evaluation criteria and presented concise summaries to the community. • Provided structured feedback and error pattern recognition reports to leadership. • Reinforced data quality and critical assessment skills among community members.

2024 - 2024

Education

Work History