Skip to content
OpenTrain AIFor AI Companies

Intelligence Analyst, AI Model Evaluation

Apply now

Job details

Pay

$80–$170 USD/hr

Engagement

Contract, part-time

Location

Remote

52 countries

Posted

Oct 23, 2026

OpenTrain on Trustpilot

The work

  • Develop and map analytic cases to defined taxonomies across multiple intelligence evaluation areas.
  • Write complex test cases that examine source handling, analytic rigor, uncertainty expression, and practical usefulness.
  • Assess and grade AI-generated military and government reports against standards for reasoning, source use, limitations, and usefulness.
  • Identify cases where an AI model refuses legitimate analytic exploration, and document the issue clearly.
  • Record edge cases and disagreements between analysts to support stronger evaluation methods.
  • Work with project leads to refine source-evaluation procedures and confidence-level assignments.
  • Share expert insights that help calibrate analytic standards and improve AI benchmarks.

Requirements

  • Language: Professional fluency in English.
  • Required background: Demonstrated all-source analytic experience or deep expertise in a specific analytic discipline.
  • Required knowledge: Strong proficiency in source evaluation, confidence-level assessment, established analytic standards, structured analytic techniques, and intelligence tradecraft standards.
  • You must be able to identify and explain nuanced model limitations and over-refusal scenarios.
  • Strong written and verbal communication, with a focus on clarity and precision, is required.
  • Experience teaching, mentoring, or leading analytic reviews is valuable.
  • A civilian intelligence community background or comparable government and defense analysis experience can provide useful context.
  • Skills: Analytic rigor, Source handling, Over-refusal detection, All-Source Intelligence Analysis

Open to applicants in

  • Algeria
  • Argentina
  • Austria
  • Bahrain
  • Bangladesh
  • Belgium
  • Brazil
  • Canada
  • Chile
  • Colombia
  • Denmark
  • Egypt
  • France
  • Germany
  • Ghana
  • Greece
  • India
  • Indonesia
  • Italy
  • Japan
  • Jordan
  • Kazakhstan
  • Kenya
  • Kuwait
  • Kyrgyzstan
  • Lebanon
  • Malaysia
  • Mexico
  • Morocco
  • Netherlands
  • Nigeria
  • Oman
  • Pakistan
  • Palestine
  • Peru
  • Philippines
  • Portugal
  • Qatar
  • Saudi Arabia
  • Singapore
  • South Africa
  • Spain
  • Sri Lanka
  • Switzerland
  • Taiwan
  • Thailand
  • Tunisia
  • United Arab Emirates
  • United Kingdom
  • United States
  • Uzbekistan
  • Vietnam

Preferred

  • Previous participation in AI evaluation or model assessment work is helpful but not required.

How to apply

  1. Press Apply now. You create a free OpenTrain account and add your resume.
  2. Finish your application on the hiring company's site. The company decides who it hires, so it runs its own form. It asks for your resume again, your LinkedIn profile, your phone number, your location and answers to 8 screening questions.
  3. Take a short AI interview.

OpenTrain on Trustpilot

Job ID: cmvldtwoh003508jpvg3zuxf2

Keep exploring

Similar open roles

View all AI training jobs

Biology and Life Sciences AI Evaluation Expert

Create and assess difficult biology content used to test and improve large language models. This remote, project-based role is open in the US and requires 20+ hours weekly, advanced biology knowledge, and clear scientific writing.

Generative AI & RLHF
Text
Remote · United States
English
Part-time · Flexible

Posted Nov 21, 2026

Generalist (Japan) for E-Commerce AI Testing

Test AI-powered shopping features through real transactions on Japanese e-commerce platforms. Place, track, and cancel orders, review AI outputs, and give written and verbal feedback for $30 to $49 per hour.

GeneralistTesterEnglish communication
Generative AI & RLHF
Text
Remote · Japan
Japanese, English
Part-time · Flexible
Hourly · $30–$49/hr

Posted Nov 21, 2026

Generalist (France) AI E-Commerce Testing

Test e-commerce AI in France by placing, tracking, and canceling orders, reviewing model outputs, and writing clear feedback. The role pays $30 to $49 per hour and requires fluent French, strong English, and local purchase-task equipment.

GeneralistTesterEnglish communication
Generative AI & RLHF
Text
Remote · France
French, English
Part-time · Flexible
Hourly · $30–$49/hr

Posted Nov 21, 2026

E-commerce AI Testing Generalist (German, Germany)

Test German e-commerce transactions and review AI outputs for accuracy and cultural relevance. This part-time contractor role pays $30 to $49 per hour and requires fluent German, strong English, and residence in Germany.

GeneralistTesterEnglish communication
Generative AI & RLHF
Text
Remote · Germany
German, English
Part-time · Flexible
Hourly · $30–$49/hr

Posted Nov 21, 2026

Generalist (Brazil) E-Commerce AI Testing Specialist

Test Brazilian e-commerce workflows, review AI outputs, and document clear feedback for $30 to $49 per hour. This part-time contractor role is open to candidates in Brazil and requires fluent Portuguese, strong English, and a computer.

GeneralistTesterEnglish communication
Generative AI & RLHF
Text
Remote · Brazil
Portuguese, English
Part-time · Flexible
Hourly · $30–$49/hr

Posted Nov 21, 2026