About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting an ML Engineer focused on evaluating large language models and agentic AI workflows. You will help measure model capabilities, identify failure modes, and improve how AI systems are trained, benchmarked, and refined. The work combines software development, research-oriented analysis, dataset curation, and structured model feedback.
What you will do
Design, build, and run evaluation suites for reasoning, coding, mathematics, tool use, and long-context tasks. Develop benchmarks, task specifications, rubrics, and grading criteria for multi-step planning, function calling, code execution, and browser or computer-use agents. Curate, clean, and annotate evaluation datasets while comparing open-source and frontier model outputs. Implement programmatic checks and model-based graders, then validate them against human reference judgments. Investigate failure cases, characterize error patterns, and produce reproducible reports with actionable recommendations. Integrate evaluations into harnesses, CI pipelines, and dashboards, and contribute to documentation and tooling improvements.
Required skills
You should be pursuing or have recently completed a bachelor's, master's, or PhD in computer science, machine learning, data science, mathematics, or a related quantitative discipline. Strong Python programming skills with familiarity with NumPy, Pandas, Jupyter, and standard machine learning workflows are required. You should understand how modern LLMs are trained, prompted, and evaluated, and have experience interacting with LLM APIs or open-source models through tools such as Hugging Face or vLLM. Strong analytical, technical writing, and independent problem-solving skills are important, as is the ability to collaborate remotely with overlapping working hours.
Helpful background
Experience designing model benchmarks, evaluating agentic workflows, building automated graders, or analyzing human reference judgments is useful for this role.
Engagement details
This is a remote contractor opportunity for qualified contributors located in India, Pakistan, or Turkey. The engagement is planned for three months. OpenTrain helps freelancers consolidate AI training opportunities across platforms and build a durable AI training portfolio.
Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.