Diana P.

Key Skills

Software

Labelbox

Snorkel AI

Internal/Proprietary Tooling

Top Subject Matter

Education-HigherEducation

Linguistics-Sociolinguistics

Social Sciences - Cultural Studies

Top Data Types

Image

Video

Text

Top Task Types

Text Generation

Fine-tuning

Evaluation/Rating

Question Answering

Transcription

RLHF

Freelancer Overview

I have experience creating, evaluating, and refining training data for large language models and other AI systems through multilingual annotation, expert review, adjudication, and benchmark development projects. I have assessed AI-generated and human-authored content for factual accuracy, reasoning quality, self-containment, rubric compliance, and alignment with project guidelines across graduate-level tasks in domains including Iberian and Latin American Cultures, Anthropology, Sociology, Linguistics, and Indigenous Studies. My work has involved identifying subtle errors in model reasoning, distinguishing between recall-based and analytical tasks, adjudicating conflicting reviews, and ensuring that evaluation items genuinely test disciplinary expertise rather than pattern matching or memorization. In addition, I have contributed to multilingual AI training initiatives involving Basque, Spanish, and English language data through translation evaluation and localization projects. I have analyzed machine-translated and post-edited outputs, compared human and AI-generated translations, identified linguistic and cultural inaccuracies, and provided quality assessments to improve model performance. I have also completed audio annotation and validation tasks for speech technologies, including transcription review, audio–text alignment, and spoken language quality evaluation for large-scale AI systems. Drawing upon more than 15 years of experience in translation, localization, language assessment, and higher education, I bring strong analytical skills, meticulous attention to detail, and the ability to apply complex annotation guidelines consistently to produce high-quality training data for AI applications.

Labeling Experience

LLM Training Data Evaluation and Expert Adjudication

TextText Generation

Conduct expert evaluation, review, and adjudication of training data used to improve the performance and alignment of large language models (LLMs). Assess both AI-generated and human-authored content for factual accuracy, reasoning quality, self-containment, rubric compliance, and adherence to project specifications across graduate-level benchmark tasks. Apply domain expertise in Anthropology, Sociology, Linguistics, Iberian and Latin American Studies, and related disciplines to identify subtle reasoning errors, detect hallucinations, evaluate distractor quality, and ensure that assessment items measure analytical thinking rather than simple recall or pattern matching. Provide detailed feedback to resolve reviewer disagreements, maintain annotation consistency, and support the creation of high-quality datasets for model evaluation, alignment, and instruction tuning. Applied extensive experience in higher education assessment design and rubric development to guarantee rigorous quality standards in large-scale AI training initiatives.

2025 - Present

Multilingual AI Data Annotation and LLM Evaluation (Basque, Spanish, English)

TextRed Teaming

Contributed to multilingual AI training and evaluation projects involving Basque, Spanish, and English language data. Evaluated AI-generated translations and post-edited machine translation outputs to improve linguistic quality, accuracy, fluency, and adherence to style guidelines. Performed comparative assessments of human and machine-generated content, identifying translation errors, inconsistencies, terminology issues, and cultural or contextual inaccuracies to support the development and refinement of language models. Additionally, completed audio annotation and quality assessment tasks for speech-related AI systems, including transcription review, validation of audio–text alignment, and evaluation of spoken language data. Worked with detailed annotation protocols to ensure consistency and high inter-annotator agreement across large datasets. Brought over 15 years of professional experience in translation, localization, and linguistic quality assurance to support the creation of high-quality multilingual training data for AI applications in machine translation, speech technologies, and large language models.

2022 - 2024

Education

Work History