Machine Learning Benchmark Evaluator

About OpenTrain

OpenTrain AI is the hiring and contracting organization for this role. OpenTrain is the leading platform for people building careers in AI training and data labeling; contributors find specialized projects, build a unified portfolio, and grow into durable freelance careers. Creating an OpenTrain account is free.

• Work with real AI engineering projects that shape how models are evaluated and improved.
• Flexible, remote contract opportunities that fit around your schedule and location.

About AI training and benchmark evaluation

AI training (data labeling and human evaluation) is the human side of building modern AI: people prepare, run, and review the examples and evaluation needed to measure and improve models. Benchmark-driven evaluation is a core part of that work—running training and inference pipelines, collecting metrics, finding failure modes, and validating model behavior on real-world tasks.

• This work places you at the intersection of software engineering and ML research, directly improving model correctness and robustness.
• Many projects are remote and flexible, making them ideal for experienced engineers seeking part-time contracting work.

The role

We are hiring experienced Machine Learning Engineers to perform benchmark-driven evaluation and engineering work on production-like ML systems. You will work in codebases that include training, evaluation, and inference pipelines; prepare datasets and metrics; debug and refactor systems for correctness and performance; and evaluate model behavior and edge cases that matter to benchmarks.

• Contract, part-time engagement focused on ML benchmark validation and engineering.
• Work directly on model training/evaluation pipelines and production-like ML code.

What you'll do

• Build, run, and modify model training, evaluation, and inference pipelines.
• Prepare datasets, features, and metrics for ML benchmarking and validation.
• Debug, refactor, and improve production-like ML systems to ensure correctness and performance.
• Evaluate model behavior, failure modes, and edge cases relevant to benchmark tasks.
• Write clean, reproducible, well-documented Python code for ML workflows and participate in code reviews.

Requirements

You must meet the core experience and skill requirements below to be considered.

• 3+ years of experience as a Machine Learning Engineer or Software Engineer focused on ML.
• Strong Python proficiency for machine learning and data workflows.
• Hands-on experience with model training, evaluation, and inference pipelines.
• Familiarity with ML frameworks such as PyTorch, TensorFlow, JAX, or equivalents.
• Ability to navigate, debug, and modify complex, real-world ML codebases.
• Solid understanding of ML fundamentals, evaluation metrics, and optimization.
• Excellent spoken and written English communication skills.

Logistics, who should apply, and how it works

This role is offered as a contractor, part-time engagement with a time expectation of 20+ hours per week. Candidates must be able to communicate fluently in English. OpenTrain accepts applicants from the listed countries below; only applicants located in those countries should apply. OpenTrain AI is the contracting organization for this work.

If you're an experienced ML engineer who enjoys hands-on benchmarking, debugging production-like code, and improving model evaluation, this role is a strong fit. To apply, create a free OpenTrain account, complete your profile, and submit your application through the OpenTrain platform.

• Time requirement: 20+ hours per week (part-time, contractor).
• Allowed locations: IN, PK, NG, KE, EG, GH, BD, TR, BR, MX (applicants should be located in these countries).
• Language: English required.
• No pay details are provided in this listing; compensation and contract terms will be shared during the application process.