Member of Technical Staff, Frontier AI Evaluation Lead
Lead model evaluation, failure analysis, and ML data systems remotely for 20+ hours per week, earning $115.38 to $168.27 per hour. Apply through OpenTrain to begin.
Posted Nov 19, 2026
About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting for a full-time Member of Technical Staff to join a frontier AI research lab focused on evaluating and improving AI systems. This hands-on role sits at the intersection of machine learning, data research, and evaluation engineering. You will collaborate with researchers and domain experts to develop benchmarks, reinforcement learning environments, and high-quality datasets that advance model capabilities and performance.
What you will do
- Own research and evaluation initiatives end-to-end, from task design and dataset development to benchmark validation and performance analysis.
- Design and validate capability evaluations, RL environments, and benchmark suites for computer use, agentic workflows, and emerging model capabilities.
- Develop high-quality datasets, task specifications, evaluation rubrics, and methodologies to improve post-training data quality and model assessment.
- Analyze model behavior through failure analysis, error taxonomies, and capability gap identification to inform research directions.
- Build reproducible evaluation pipelines and feedback loops to support iterative model development.
- Contribute to benchmarks, research papers, technical reports, and open research artifacts.
Required skills
- Strong technical background in computer science, machine learning, AI, or a related field.
- Hands-on experience designing ML evaluations, benchmark suites, research datasets, or experimental frameworks.
- Proficiency in Python and experience building data pipelines, evaluation tools, or ML research infrastructure.
- Understanding of LLMs, foundation models, post-training techniques, and evaluation methodologies.
- Ability to translate complex real-world workflows into structured tasks, measurable criteria, and research experiments.
- Strong analytical skills in model failure analysis, experimental design, and evaluation quality.
Helpful background
- Experience with reinforcement learning environments, reward modeling, simulators, or feedback-driven training systems.
- Experience evaluating computer use agents, tool use, browser automation, or multi-step agentic workflows.
- Familiarity with multimodal reasoning, long-context evaluation, instruction following, or deep research benchmarks.
- Experience with human-in-the-loop evaluations, expert-generated datasets, or post-training data.
- Contributions to AI research publications, open-source benchmarks, or evaluation frameworks.
You apply here in about two minutes and finish on the hiring platform. The next step is a short AI interview (technical roles include a coding challenge). Passing it is how you qualify for paid tasks, so take it soon after you are invited.
The hiring platform's application for this job asks for your resume, your LinkedIn profile, your phone number, your location and answers to 4 screening questions.
Open to applicants in
OpenTrain on Trustpilot
Job ID: cmwkasuqq00dz08l6e98q1zz0
Keep exploring
Lead model evaluation, failure analysis, and ML data systems remotely for 20+ hours per week, earning $115.38 to $168.27 per hour. Apply through OpenTrain to begin.
Posted Nov 19, 2026
Earn $67.31 to $86.54 per hour in a remote Enterprise Client Partner role focused on frontier AI partnerships, model evaluation, and human-data programs. Work 20+ hours per week and apply through OpenTrain.
Posted Nov 19, 2026
About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…
Posted Nov 18, 2026
About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…
Posted Nov 18, 2026
About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…
Posted Nov 17, 2026
Browse related job pages
Categories
Locations
Languages