LLM Red-Teamer for Adversarial Testing and Evaluation

About OpenTrain

OpenTrain is the #1 platform for people building careers in AI training and data labeling. We help freelancers discover projects, build an AI training portfolio, and grow durable freelance careers in a fast-growing field. Creating an OpenTrain account is free and connects you to remote, flexible contracting work in AI.

About AI Training Work (Why this matters)

AI training — also called data labeling, annotation, or human feedback — is the human work that teaches modern models how to behave. Contributors annotate, evaluate, and adversarially test model behavior so AI systems are safer, more useful, and aligned to human expectations.

This kind of work is often remote, flexible, and accessible: many projects need strong attention to detail and domain-specific skills rather than technical credentials. As an LLM Red-Teamer you’ll be on the front lines shaping how language models handle adversarial or edge-case interactions.

The Role

We are hiring an LLM Red-Teamer to design adversarial evaluation packages and test frontier language models. This is a remote contractor, part-time role (20+ hours/week) with pay between $40 and $65 per hour depending on experience. OpenTrain AI is the contracting organization for this project and contributors work independently while maintaining calibration with project leads.

• Position type: Contractor, part-time (20+ hours/week).
• Pay: $40–$65 USD per hour, based on experience.
• Language: Fluent written English required.
• Location: Worldwide (remote).

What You'll Do

You will create adversarial multi-turn conversations, author evaluation rubrics, test model outputs, and deliver polished task packages that teams can use for training and evaluation.

• Develop complex, adversarial multi-turn conversations and task-based scenarios aligned to detailed specifications.
• Author clear, precise evaluation rubrics (including binary rubrics) to assess model responses against defined behavioral targets.
• Iteratively test conversations against frontier LLMs, increasing difficulty and nuance until tasks meet quality targets.
• Deliver comprehensive task packages: transcripts, target behaviors, rubrics, rationale, and evidence of failure modes.
• Validate and document model strengths and failure patterns relative to the project specification.
• Maintain calibration with team leads and quality-control contacts as project requirements evolve.

Requirements

Candidates must demonstrate strong written English and the ability to produce clear, structured rubrics and adversarial scenarios. This role emphasizes independent work, critical thinking, and meticulous attention to detail.

• Exceptional written English: clarity, precision, and strong structural organization.
• Ability to design adversarial multi-turn conversations and task scenarios.
• Ability to evaluate binary rubrics against model outputs and document failure modes.
• Ability to work autonomously and interpret complex specifications with minimal oversight.
• Deep familiarity with large language models and common LLM failure patterns.

Helpful Experience (not required)

Prior experience in AI human-data environments or writing- and analysis-focused roles will help you ramp faster, but applicants without formal AI experience who meet the requirements are encouraged to apply.

• Experience with RLHF, SFT, model evaluations, annotation, or prompt engineering.
• Experience designing evaluation items or rubrics.
• Backgrounds such as research, editorial work, technical writing, or QA are relevant.