Data Science AI Evaluation Expert

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We connect qualified contributors with opportunities to help shape modern AI systems, build a professional profile, and grow their experience in a fast-moving field.

OpenTrain AI is recruiting a Data Science AI Evaluation Expert for a talent network supporting future AI evaluation projects. This is a remote, hourly contractor opportunity with flexible part-time availability.

• Remote work available worldwide
• Contractor and part-time engagement
• Compensation of $100 to $150 per hour
• English-language work
• 20+ hours per week, with a default commitment of 40 hours per week

About AI Training and Evaluation

AI training is the human side of building artificial intelligence. Experts review examples, assess model outputs, and provide structured feedback that helps AI systems become more accurate, reliable, and useful.

In this role, your data science expertise will help evaluate how effectively AI systems perform real-world analytical work. Your assessments can help establish clearer standards for data science reasoning, experimentation, modeling, and communication.

• Contribute to the development of cutting-edge AI systems
• Use professional expertise to assess complex technical work
• Work remotely with a flexible schedule
• Help make AI evaluation more consistent and evidence-based

The Role

As a Data Science AI Evaluation Expert, you will evaluate AI-generated or human-created data science deliverables against established criteria. You will design precise grading standards, score submitted work, explain your decisions, and refine evaluations using structured feedback from senior reviewers.

• Role focus: data science evaluation for AI
• Work type: evaluation and rating
• Experience level listed: entry level
• Engagement: hourly contractor
• Data format: text

What You'll Do

You will assess a broad range of data science outputs and communicate your reasoning clearly. The work requires careful attention to technical quality, consistency, and the evidence supporting each evaluation.

• Design task-specific grading criteria for exploratory data analyses
• Evaluate statistical modeling work and machine learning pipelines
• Assess experimentation and A/B test write-ups, including causal inference considerations
• Review feature engineering and technical reports or notebooks
• Evaluate AI-generated and human-created data science work
• Provide detailed written justifications for scores and evaluations
• Apply consistent, evidence-based judgment so assessments are reproducible and defensible
• Incorporate structured feedback from senior reviewers and iterate on submitted work

Requirements

This opportunity is intended for candidates with professional data science experience and a strong technical foundation. You should be able to evaluate complex analytical work and explain technical findings in clear written English.

• At least 1 year of professional data science experience
• Experience at a leading technology, research, or quantitative firm, such as a top FAANG company, AI lab, top-tier quantitative fund, or equivalent
• Strong command of Python and SQL
• Strong command of statistical modeling and machine learning
• Experience with experimentation and causal inference
• Exceptional written communication skills
• A detail-oriented and consistent approach to evaluating complex work
• Comfort receiving feedback and calibrating judgment against established standards

Who Should Apply

Apply if you have the professional data science background to distinguish rigorous, well-supported work from incomplete or unreliable analysis. This role may be especially well suited to data scientists who enjoy reviewing technical deliverables, developing evaluation standards, and communicating precise feedback.

• Data scientists with experience in technology, research, or quantitative organizations
• Professionals comfortable reviewing notebooks, reports, models, and experiments
• Candidates who can make consistent judgments across varied data science tasks
• Experts who welcome reviewer feedback and ongoing calibration

How to Apply Through OpenTrain

Create a free OpenTrain account to build your AI training profile and apply in minutes. OpenTrain helps contributors discover opportunities across the AI training industry and develop a durable career working on projects that shape how AI is built.

This role is part of a talent network for future projects, so project availability and specific assignments may vary. The listed compensation is $100 to $150 per hour, and the role requires at least 20 hours per week with a default commitment of 40 hours per week.

• Apply through OpenTrain
• Showcase your data science experience and technical strengths
• Indicate your availability for 20+ hours per week
• Prepare to complete evaluations using established standards and reviewer feedback