About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting for a Python ML Quality Assurance Lead to review AI-generated Python code, machine learning workflows, model explanations, and trainer QA work. In this remote contractor role, you will monitor quality, evaluate output against project guidelines and rubrics, give clear written feedback, and help contributors maintain high standards across Python machine learning training work. The role also includes answering questions on Python syntax, package usage, data leakage, model validation, metrics, reproducibility, notebooks, and rubric interpretation. You will support contributor communication, documentation, onboarding materials, calibration tasks, and process improvements that strengthen quality across technical review workflows.
What you will do
- Review AI-generated Python code, ML pipelines, preprocessing steps, model training workflows, evaluation logic, and explanations for correctness and reproducibility.
- Spot-check work for data leakage, statistical validity, debugging accuracy, formatting, readability, maintainability, and instruction-following.
- Provide precise feedback to contributors and help escalate recurring or critical quality issues.
- Update guidance, trackers, FAQs, examples, honeypots, and onboarding materials.
- Support onboarding and training calls for Python ML contributors.
- Help maintain scalable QA processes for remote technical teams.
Required skills
- Strong Python fundamentals, including data structures, functions, classes, iterators, comprehensions, exception handling, virtual environments, package management, testing, and debugging.
- Strong ML knowledge across supervised and unsupervised learning, feature engineering, train/test splits, cross-validation, model selection, data leakage, regression, classification, clustering, metrics, bias and variance, regularization, and reproducibility.
- Ability to evaluate ML content against detailed rubrics and identify flawed methodology, wrong metrics, hallucinated APIs, misleading conclusions, and incomplete explanations.
- Strong English communication skills for clear technical feedback and team coordination.
- Experience with AI training, data annotation, LLM evaluation, code QA, ML QA, or rubric-based technical review is a strong plus.
- Experience leading or supporting remote teams of trainers, reviewers, engineers, data scientists, or ML researchers is preferred.