Skip to content
OpenTrain AIFor AI Companies

ML Engineer, LLM and Agentic Evaluations (English)

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

23 countries

Posted

Nov 18, 2026

OpenTrain on Trustpilot

The work

  • Design, build, and run evaluation suites for large language models across reasoning, coding, mathematics, tool use, and long-context tasks.
  • Develop benchmarks and grading rubrics for multi-step planning, function calling, code execution, and browser or computer-use agents.
  • Curate, clean, and annotate evaluation datasets, then write clear task specifications and grading criteria.
  • Compare open-source and frontier models, analyze results, and produce structured reports with practical recommendations.
  • Build programmatic and LLM-based graders, validate them against human reference judgments, and investigate recurring error patterns.
  • Integrate evaluations into engineering harnesses, continuous integration pipelines, and dashboards.
  • Contribute to documentation and tooling improvements.

Requirements

  • Plan to work 20+ hours per week in a remote environment. The assignment lasts 3 months and may be adjusted based on the engagement.
  • Pursuing or recently completed a Bachelor's, Master's, or PhD in computer science, machine learning, data science, mathematics, or a related quantitative field.
  • Strong Python skills with familiarity with NumPy, Pandas, Jupyter, and standard machine learning workflows.
  • Understanding of how modern LLMs are trained, prompted, and evaluated, plus experience using hosted or open-source LLM APIs.
  • Fluent in English, with strong analytical and technical writing skills and the ability to work independently.

Open to applicants in

  • Argentina
  • Belize
  • Bolivia
  • Brazil
  • Chile
  • Colombia
  • Costa Rica
  • Dominican Republic
  • Ecuador
  • El Salvador
  • Guatemala
  • Guyana
  • Haiti
  • Honduras
  • Mexico
  • Nicaragua
  • Panama
  • Paraguay
  • Peru
  • Puerto Rico
  • Suriname
  • Uruguay
  • Venezuela

Preferred

  • Helpful background includes LLM benchmark design or use, agent frameworks, tool-using agents, LLM-as-a-judge methods, inter-annotator agreement, statistical analysis, prompt engineering, structured outputs, RLHF, RLAIF, and post-training data pipelines.

How to apply

  1. Press Apply now. You create a free OpenTrain account and add your resume.
  2. Finish your application on the hiring company's site. The company decides who it hires, so it runs its own form.

OpenTrain on Trustpilot

Job ID: cmwmk86w2001c07jplth6amdh

Keep exploring

Similar open roles

View all AI training jobs

Member of Technical Staff, Enterprise AI (English)

Evaluate enterprise AI systems through dataset design, experiments, benchmarking, and failure analysis. This part-time contract role pays $100 to $130 per hour and requires strong machine learning evaluation skills.

Research Signal JudgmentML-Oriented Data DesignOps-to-Research TranslationEnterprise AI
Generative AI & RLHF
Document
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $100–$130/hr

Posted Nov 18, 2026

Travel Operations Specialist (English)

Use your travel booking and itinerary skills to create and review realistic examples for AI training. This remote, part-time contractor role pays $20 to $44 per hour and requires fluent English.

Travel BookingCustomer ExperienceItinerary PlanningReservations
Generative AI & RLHF
Document
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $20–$44/hr

Posted Nov 18, 2026

Counterintelligence Technology & Program SME (English)

Evaluate AI responses through a counterintelligence lens, author case studies, and review security-focused outputs. This remote, part-time contract pays $90 to $140 per hour and requires senior counterintelligence experience.

CI Threat JudgementLegitimate/prohibited discriminationProgram and technology senseSource handling and uncertainty
Generative AI & RLHF
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $90–$140/hr

Posted Nov 18, 2026

Member of Technical Staff, Enterprise AI (English)

Research enterprise AI systems by analyzing failures, designing evaluations, benchmarking agentic workflows, and building practical tools. This part-time contractor role pays $100 to $130 per hour and requires English fluency.

Research Signal JudgmentML-Oriented Data DesignOps-to-Research TranslationEnterprise AI
Generative AI & RLHF
Document
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $100–$130/hr

Posted Nov 18, 2026

Member of Technical Staff, Economics Research

Review AI outputs against economics research guidelines and explain your judgments with clear evidence. This remote contractor role pays $96.15 to $120.19 per hour and requires a PhD plus applied research experience.

Economic ResearchEnterprise AIBenchmarkingAgentic Economic Workflows
Generative AI & RLHF
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Part-time · Flexible
Hourly · $96.15–$120.19/hr

Posted Nov 18, 2026