Data Science QA Lead — Remote Contractor

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We help contributors discover projects, build a unified training portfolio, and grow freelance careers working on the human side of AI.

This role is hired and contracted by OpenTrain AI. Creating an OpenTrain account is free and is how you apply and manage work.

About AI training work

AI training (data labeling, annotation, and human feedback) is the human backbone of modern AI. Contributors review and improve model outputs—annotating, rating, and giving feedback—so that models learn correct, reliable behavior.

This work is remote, flexible, and accessible: you can shape cutting-edge AI systems while working part time or freelance, often without prior industry experience. Specialist roles, like this one, reward domain knowledge and technical skill.

The role

You will lead QA for AI-generated data science content and trainer QA work. Your core responsibility is to assess statistical accuracy, model choice, code correctness, reproducibility, metric interpretation, business context, instruction following, and rubric adherence, then provide precise written feedback that raises quality across projects.

You will also identify recurring errors, maintain and improve documentation and guidelines, support onboarding and calibration for contributors, and help evolve QA processes and tooling.

What you'll do

• Review AI-generated data science explanations, Python, R, and SQL snippets, modeling workflows, dashboards, experiment designs, and step-by-step reasoning.
• Detect data leakage, flawed assumptions, incorrect metrics, weak methodology, non-reproducible code, and misleading conclusions.
• Provide clear, actionable written feedback tied to rubrics and guidelines.
• Communicate guideline changes and workflow updates to trainers and QAs.
• Create and maintain style guides, trackers, FAQs, examples, honeypots, calibration tasks, and onboarding materials.
• Support onboarding and training calls for contributors and help improve QA processes and tooling.

Requirements

• Degree in data science, statistics, computer science, machine learning, mathematics, economics, engineering, or a related quantitative field.
• 3+ years experience in data science, analytics, machine learning, statistical modeling, experimentation, data engineering, technical review, or data science education.
• Strong understanding of statistics, model evaluation, experimentation, regression, classification, clustering, and validation methods.
• Familiarity with Python, pandas, NumPy, scikit-learn, SQL, Jupyter, matplotlib, R, Spark, Git, MLflow, notebooks, dashboards, and cloud or data platforms.
• Strong English communication skills for concise, technical written feedback and team coordination.
• Ability to review analytical work against rubrics and provide consistent evaluation ratings (EVALUATION_RATING tasks).

Helpful background

• Experience leading or supporting remote teams of trainers, annotators, analysts, data scientists, engineers, educators, or QAs.
• Experience with AI training, data annotation, LLM evaluation, or rubric-based technical review is a strong plus.
• Comfort using Discord, Google Sheets, Google Docs, trackers, dashboards, GitHub, and project management systems.
• Highly organized and experienced maintaining documentation, examples, and calibration materials.

Compensation & logistics

This is an hourly, remote contractor role (part-time). Advertised pay is up to $110 USD per hour. The position targets 20+ hours per week.

Open to candidates based in the United States only. Work will be conducted in English. Tasks are text-focused (DATA TYPE: TEXT) and involve evaluation/rating work (LABEL TYPE: EVALUATION_RATING).

How it works / Apply

Create a free OpenTrain account to apply and manage work. Applications are submitted through your OpenTrain profile.

If selected, you'll participate in onboarding and calibration activities, including training calls, sample reviews, and documentation review to align with project rubrics and quality standards.