Data Science QA Lead

About OpenTrain

OpenTrain AI is the hiring and contracting organization for this role. OpenTrain is the #1 platform for finding and building careers in AI training and data labeling, helping contributors discover projects, build a professional profile, and apply to opportunities in minutes.

Creating an OpenTrain account is free, and this opportunity offers a way to contribute directly to the development and evaluation of modern AI systems.

About AI Training and Data Evaluation

AI training is the human side of building artificial intelligence. People review examples, evaluate model outputs, check technical accuracy, and provide feedback that helps AI systems become more useful and reliable.

In this role, your data science expertise will support evaluation of AI-generated explanations, analytical workflows, code, experiments, and conclusions. The work is remote and can be a flexible way to apply specialized skills to cutting-edge AI projects.

The Data Science QA Lead Role

OpenTrain AI is recruiting a Data Science QA Lead to review AI-generated data science content and trainer QA work. You will assess statistical accuracy, model selection, code correctness, reproducibility, metric interpretation, business context, instruction following, and rubric adherence.

You will also provide precise written feedback, identify recurring quality issues, help update trainers and QAs, support contributor onboarding, maintain quality documentation, and improve QA processes. This is a remote hourly contractor role for US-based contributors working 20 or more hours per week.

• Advertised rate of up to $110 per hour
• Part-time contractor engagement
• Remote work available in the United States
• English-language role requiring strong written communication

What You’ll Do

You will evaluate both the technical substance and communication quality of data science work. Your reviews will help ensure that AI-generated content is accurate, reproducible, methodologically sound, and aligned with project requirements.

• Review AI-generated data science explanations, Python, R, and SQL snippets, modeling workflows, dashboards, experiment designs, and step-by-step reasoning.
• Check for data leakage, flawed assumptions, incorrect metrics, weak methodology, non-reproducible code, and misleading conclusions.
• Assess analytical work against rubrics covering statistical accuracy, model selection, code correctness, business context, instruction following, and rubric adherence.
• Communicate guideline changes and workflow updates to trainers and QAs.
• Create and maintain style guides, trackers, FAQs, examples, honeypots, calibration tasks, and onboarding materials.
• Support onboarding and training calls for contributors.
• Identify recurring issues and help improve QA processes.

Required Qualifications and Skills

The role requires a strong quantitative background and the ability to review analytical work against detailed rubrics. Candidates should be comfortable explaining technical findings clearly in written English and coordinating with distributed contributors.

• Degree in data science, statistics, computer science, machine learning, mathematics, economics, engineering, or a related quantitative field.
• Strong English communication skills for clear technical feedback and team coordination.
• At least 3 years of experience in data science, analytics, machine learning, statistical modeling, experimentation, data engineering, technical review, or data science education.
• Strong understanding of statistics, model evaluation, experimentation, regression, classification, clustering, and validation methods.
• Familiarity with Python, pandas, NumPy, scikit-learn, SQL, Jupyter, matplotlib, R, Spark, Git, MLflow, notebooks, dashboards, and cloud or data platforms.
• Experience with AI training, data annotation, LLM evaluation, data science QA, or rubric-based technical review is a strong plus.

Helpful Background

Experience supporting distributed teams and maintaining clear quality resources will help you succeed. The work involves coordinating updates, organizing documentation, and keeping review standards consistent across projects.

• Experience leading or supporting remote teams of trainers, annotators, analysts, data scientists, engineers, educators, or QAs.
• Comfort using Discord, Google Sheets, Google Docs, trackers, dashboards, GitHub, and project management systems.
• Strong organizational skills and the ability to maintain documentation and quality resources.
• Availability for 20 or more hours per week.

How to Apply Through OpenTrain

Create a free OpenTrain account to build your AI training profile and apply to this opportunity. If selected, you will work remotely as a contractor and contribute technical review expertise to the quality of AI-generated data science content.

• Review the role requirements and confirm your US availability.
• Highlight your data science, statistics, machine learning, analytics, or technical review experience.
• Show how you communicate precise feedback and evaluate analytical work against standards.
• Apply through OpenTrain and manage your opportunity from your OpenTrain profile.