AI Training & Evaluation Specialist (Remote, Expert)

About OpenTrain

OpenTrain is the #1 platform for people building careers in AI training and data labeling. OpenTrain AI hires and contracts contributors directly to work on real model training and evaluation projects — enabling flexible, remote work that shapes how modern AI systems behave.

• Work fully remote and build experience in a fast-growing field.
• OpenTrain focuses on AI training, evaluation, and human feedback work across industries.

About AI training and human feedback work

AI training (also called data labeling or human feedback) is the human side of building intelligent systems: people annotate data, evaluate outputs, and rate responses so models learn correct behavior. This role sits at the intersection of evaluation, RLHF, and text labeling — essential work that directly improves model performance.

• Tasks include response ranking, prompt evaluation, question answering checks, and quality review.
• Many projects are flexible and accessible; specialist expertise is valued and often required for expert-level tasks.

The role

As an AI Training & Evaluation Specialist you will complete onboarding and client qualification assessments, then work on assignments matched to your strengths. Work focuses on evaluating and ranking AI-generated outputs, running natural language and reasoning assessments, and contributing to text labeling and quality review to improve models.

• Contractor, part-time role with an expected minimum commitment of 20 hours per week.
• Project duration expected to run for 6+ months.

What you'll do

You will provide consistent, high-quality human feedback used to train and refine generative models. Typical activities include evaluation rating, RLHF-style feedback, question-answer validation, prompt assessment, and general text labeling and QA. Assignments are tailored to your demonstrated strengths after qualification.

• Evaluate and rank AI-generated outputs for correctness, helpfulness, and safety.
• Perform prompt evaluations and reasoning assessments.
• Label text data and conduct quality reviews to improve model training sets.
• Complete client onboarding and qualification assessments (some allow only one attempt).

Requirements

You must meet the core qualifications and follow assessment rules exactly. We do not allow reliance on major generative AI tools to produce answers during qualification tasks — follow the assessment AI-usage policy.

• Prior experience on remote AI platforms and demonstrated AI training, evaluation, annotation, ranking, or labeling experience.
• Experience with text labeling workflows and with Evaluation Rating, Question Answering, and RLHF tasks.
• Excellent English reading and writing skills; fluent English proficiency required.
• Reliable computer and internet connection; strong independence and attention to detail.
• Ability to complete client qualification/onboarding assessments (some with one attempt).
• Expected weekly commitment: at least 20 hours.
• Project duration: expected 6+ months.
• Software engineer who knows how to code extensively.

Who should apply

Apply if you are an experienced evaluator or annotator comfortable working independently on expert-level evaluation and RLHF tasks. Candidates with software engineering or coding backgrounds who can reason about model outputs and follow strict assessment rules are especially welcome.

• Ideal for people building a career in AI training, ML evaluation, or human feedback.
• Good fit for experienced contractors who want stable, part-time remote work in English.

How it works & pay

OpenTrain AI hires contributors as contractors and assigns work after onboarding and qualification. You will be paid per hour as a contractor and must follow all onboarding and assessment rules. Keep in mind the stated hourly rate when evaluating this opportunity.

• Employment type: Contractor, Part-time.
• Time requirement: 20+ hours per week (minimum).
• Duration: expected 6+ months.
• Langauge: English required.
• Pay: USD $1.00 per hour (PAY_PER_HOUR).
• Worldwide applicants accepted.