Skip to content
OpenTrain AIFor AI Companies

ML Engineer, LLM and Agentic Evaluations

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

3 countries

Posted

Nov 18, 2026

OpenTrain on Trustpilot

The work

  • Design, build, and run evaluation suites for reasoning, coding, mathematics, tool use, and long-context tasks.
  • Create benchmarks, task specifications, rubrics, and grading criteria for multi-step planning, function calling, code execution, and browser or computer-use agents.
  • Curate, clean, and annotate evaluation datasets while comparing open-source and frontier model outputs.
  • Build programmatic checks and model-based graders, then validate them against human reference judgments.
  • Investigate failure cases, identify error patterns, and write reproducible reports with practical recommendations.
  • Integrate evaluations into harnesses, CI pipelines, and dashboards.
  • Improve documentation and evaluation tools.

Requirements

This is a remote contractor opportunity for qualified contributors. The engagement is planned for 3 months.

You should be pursuing or have recently completed a bachelor's, master's, or PhD in computer science, machine learning, data science, mathematics, or a related quantitative discipline.

  • Part-time engagement requiring 20+ hours per week.
  • Fluent English.
  • Strong Python skills with familiarity with NumPy, Pandas, Jupyter, and standard machine learning workflows.
  • Understanding of how modern LLMs are trained, prompted, and evaluated.
  • Experience using LLM APIs or open-source models through tools such as Hugging Face or vLLM.
  • Strong analytical, technical writing, and independent problem-solving skills.
  • Ability to collaborate remotely with overlapping working hours.

Open to applicants in

  • India
  • Pakistan
  • Türkiye

Preferred

  • Experience designing model benchmarks, evaluating agentic workflows, building automated graders, or analyzing human reference judgments is helpful.

How to apply

  1. Press Apply now. You create a free OpenTrain account and add your resume.
  2. Finish your application on the hiring company's site. The company decides who it hires, so it runs its own form.

OpenTrain on Trustpilot

Job ID: cmwmk86q8001a07jpjipi0oew

Keep exploring

Similar open roles

View all AI training jobs

Member of Technical Staff, Enterprise AI (English)

Evaluate enterprise AI systems through dataset design, experiments, benchmarking, and failure analysis. This part-time contract role pays $100 to $130 per hour and requires strong machine learning evaluation skills.

Research Signal JudgmentML-Oriented Data DesignOps-to-Research TranslationEnterprise AI
Generative AI & RLHF
Document
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $100–$130/hr

Posted Nov 18, 2026

Travel Operations Specialist (English)

Use your travel booking and itinerary skills to create and review realistic examples for AI training. This remote, part-time contractor role pays $20 to $44 per hour and requires fluent English.

Travel BookingCustomer ExperienceItinerary PlanningReservations
Generative AI & RLHF
Document
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $20–$44/hr

Posted Nov 18, 2026

Counterintelligence Technology & Program SME (English)

Evaluate AI responses through a counterintelligence lens, author case studies, and review security-focused outputs. This remote, part-time contract pays $90 to $140 per hour and requires senior counterintelligence experience.

CI Threat JudgementLegitimate/prohibited discriminationProgram and technology senseSource handling and uncertainty
Generative AI & RLHF
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $90–$140/hr

Posted Nov 18, 2026

Member of Technical Staff, Enterprise AI (English)

Research enterprise AI systems by analyzing failures, designing evaluations, benchmarking agentic workflows, and building practical tools. This part-time contractor role pays $100 to $130 per hour and requires English fluency.

Research Signal JudgmentML-Oriented Data DesignOps-to-Research TranslationEnterprise AI
Generative AI & RLHF
Document
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $100–$130/hr

Posted Nov 18, 2026

Member of Technical Staff, Economics Research

Review AI outputs against economics research guidelines and explain your judgments with clear evidence. This remote contractor role pays $96.15 to $120.19 per hour and requires a PhD plus applied research experience.

Economic ResearchEnterprise AIBenchmarkingAgentic Economic Workflows
Generative AI & RLHF
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $96.15–$120.19/hr

Posted Nov 18, 2026