MLE Bench Data Analyst

You will analyze data from machine learning training, inference, and evaluation workflows. Your work will help assess model performance, identify failures, and improve the reliability of machine learning systems.

You will work with ML engineers and researchers on real-world evaluation problems, using clear and reproducible analysis to support benchmark-driven reviews.

• Analyze structured and unstructured data from ML training, inference, and evaluation pipelines.
• Define, calculate, and validate model performance and behavior metrics.
• Investigate data distributions, edge cases, model outputs, and benchmark failures.
• Use Python and SQL to analyze data, create reports, and support evaluation workflows.
• Check data quality, consistency, and correctness across datasets and experiments.
• Create clear analytical documents and reproducible workflows.
• Communicate findings clearly in spoken and written English.

This is a remote contractor assignment with part-time hours. The contract is planned for three months and may be adjusted based on engagement.

• Hours: At least four hours per day and 20 hours per week.
• Schedule: At least four hours of overlap with PST is required.
• Experience: At least three years as a data analyst or analytics-focused engineer.
• Technical skills: Strong Python for data analysis and solid SQL experience with relational datasets.
• ML experience: Experience analyzing machine learning outputs and evaluation metrics.
• Other requirements: Strong statistical understanding, analytical reasoning, and comfort with large, complex datasets.
• Code quality: Ability to write clean, readable, well-documented analytical code.
• Language: Professional fluency in English.
• Level: Intermediate.