LLM Data Labeling and Annotation Specialist

About OpenTrain

OpenTrain AI is the #1 platform for finding and building careers in AI training and data labeling. We hire and contract contributors to prepare, review, and improve the examples that modern AI models learn from.

This role is offered and managed by OpenTrain AI. Joining means contributing directly to how state-of-the-art language models behave while building experience in a fast-growing industry.

About AI training and why it matters

AI training (data labeling / annotation) is the human work that teaches models to understand text and produce reliable outputs. Tasks include classifying responses, marking entities, and rating answer quality.

This work is remote, flexible, and accessible: you can often choose your hours and make an immediate impact on model behavior. Many contributors use part-time projects to gain experience and specialize in technical or domain-specific labeling.

The role

You will label and verify 500–1,000 rows of text-based LLM response data consisting of context, queries, and auto-generated labels. Work is performed in AWS SageMaker.

This is a part-time contractor role requiring 20+ hours per week. Pay is hourly at USD $5/hour. The project uses a two-step human review with an additional QA agent for quality assurance.

• Data type: text (English)
• Label types: Classification, NER/Entity + classification, Evaluation/Rating
• Labeling tool: AWS SageMaker
• Employment: Contractor, Part-time, worldwide

What you'll do day-to-day

Follow the project workflow to review, correct, and confirm labels on each row of LLM response data. Use the provided instructions and the SageMaker interface to submit labels and comments.

Collaborate indirectly with other labeling agents as part of the multi-step review process and escalate technical or financial items to the expert reviewer when required.

• Compare context, query, and auto-generated label; accept, correct, or re-classify as needed
• Apply entity tagging (NER) when instructed and assign classification labels consistently
• Rate model responses for quality, helpfulness, accuracy, and safety per rubric
• Provide clear annotations and brief rationale when you change or flag a label

Review workflow and quality rules

Each row receives two independent reviews (two agents review the same row). A third agent performs quality assurance on a sample or on flagged items.

For items containing technical computer science or financial content, reviewers should do their best-effort labeling but rely on an expert reviewer for final judgment. At least one of the three reviewers must be an expert able to assess technical or financial content.

• Step 1: Two independent reviewers annotate the same row
• Step 2: Third reviewer performs QA or resolves disagreements
• Expert requirement: at least one reviewer in each 3-person review set must have technical or financial expertise when content requires it

Requirements

This position is for expert-level contributors: you should have strong annotation judgment and prior experience with text-based labeling or evaluation work.

English fluency and good general knowledge are required. Experience with technical computer science data and/or financial documents is a strong plus and will be used to assign expert reviewer responsibilities.

• Expert experience level in annotation, evaluation, or related labeling work
• Familiarity with English language required
• Prefer experience with technical CS data and/or financial documents
• Comfort using AWS SageMaker or willingness to learn quickly
• Able to commit 20+ hours per week as a contractor

Compensation, schedule, and next steps

Pay is hourly at USD $5/hour (PAY_PER_HOUR). This is a contractor, part-time role; hours are flexible but you must be available for 20+ hours per week.

If you meet the requirements, you will receive project instructions and access to the SageMaker workspace to begin labeling and participating in the two-step review process.

• Rate: $5.00 USD per hour (hourly pay)
• Schedule: flexible, minimum 20 hours/week
• Location: remote — worldwide