Meghan E.

Key Skills

Software

No software listed

Top Subject Matter

No subject matter listed

Top Data Types

Image

Video

Audio

Top Task Types

No task types listed

Freelancer Overview

I have experience in AI data annotation, LLM evaluation, and model response quality review. My experience includes evaluating AI responses for instruction following, factual accuracy, tone, conciseness, and overall helpfulness. This work has include side by side comparisons, applying, rubric based scoring, identifying hallucinations and unsupported claims, and writing detailed rationales about ratings for each axis included in a rubric. As my skill level increased, I was moved to higher level tasks such as conducting audits on annotator' evaluations to ensure their ratings and rationales are well aligned with rubrics. I have worked on more complex annotation projects as well where I have written system instructions, authored prompts, and created input files to run against models in order evaluate their performance, assess their final responses and output files, identify common failure patterns, and give written documentation. I have experience performing extensive fact checking and research. Additionally, I have worked on several annotation projects involving content moderation and user safety. My Bachelor of Science degree strengthened my analytical skills and research is one of my greatest strengths. Prior to working with data annotation, I worked as a digital engagement manager for private pre-school. In that role managed digital engagement across major platforms, regularly posted on behalf of the business, responded to inquires, monitored and moderated online content and spaces, and ensured strict adherence to the specific policies of each platform. Lastly, I have experience as a crisis counselor for the Crisis Text Line. Duties included performing risk assessments in real time with a high volume of individuals experiencing mental health crises. I was able to use written communication to effectively de-escalate emotionally charged situations, quickly build report, negotiate safety plans, and escalate if necessary. My Crisis Counseling experience has actually been monumental when I've worked on user safety and moderation data annotation projects. I have an eye for sensitive subjects and subtle undertone. I've used similar techniques such as "validation without verification" which is validating a user's emotional reality without reinforcing their negative or harmful beliefs behind it. As an example, an emotionally distressed user might send an AI model "Everyone hates me. I'm a burden..." An unsafe model response example would be responding how many models have been trained to, which is validating the user, for example a bot might reply, "You're right, people have treated you horribly." I'm able to quickly recognize how responses such this (when intended to be helpful) can actually be harmful and a real safety concern, because it can easily lead a distressed user into believing their harmful belief is justified. A safe bot response might like look like: "I'm really sorry you're feeling unwanted and alone. Feeling like a burden can be incredibly painful, but that feeling doesn't mean it's true. Let's focus on getting you through this moment safely." Between my degree, intermediate level AI training experience, familiarity without digital content and spaces, and having worked as a crisis counselor through a text based written communication platform; have all given me useful, applicable knowledge and skills for training AI.

Labeling Experience