Data Engineering and Data Science AI Evaluation Engineer

As a Data Engineering and Data Science AI Evaluation Engineer, you will create and check data workflows used to measure how well advanced AI systems perform. You will work with production-like structured and unstructured data, Python code, and real-world data problems.

You will also help design difficult, realistic data engineering and data science tasks for AI systems. The work includes local experiments, code reviews, and careful checks for correct and repeatable results.

• Support SWE Bench-style evaluation tasks with structured and unstructured datasets.
• Design, build, and validate data pipelines for benchmarking and evaluation workflows.
• Process and analyze data, prepare features, and validate data science workflows.
• Write, run, and modify Python code for data processing and local experiments.
• Check data quality, transformations, and outputs for correctness and reproducibility.
• Create clean, documented, reusable workflows suitable for benchmarking.
• Take part in code reviews focused on code quality and maintainability.
• Work with researchers and engineers to develop real-world data engineering and data science tasks for AI systems.

What it pays and takes

This is a part-time contractor role with a three-month contract. The contract length may be adjusted based on the engagement. The posting does not specify a pay rate.

The listing is marked entry level, but it requires at least three years of overall experience in data engineering, data science, or data-focused software engineering.

• Pay: Not specified in the posting.
• Schedule: At least 4 hours per day and at least 20 hours per week.
• Availability: At least 4 hours of overlap with Pacific Time, or PST.
• Location: Open to candidates in India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Brazil, and Mexico.
• Language: Excellent spoken and written English.
• Python: Strong proficiency for data engineering and data science workflows.
• Data experience: Data processing, analysis, model-related workflows, and work with structured and unstructured data.
• Technical ability: Understand, navigate, and modify complex real-world codebases.
• Code quality: Write readable, reusable, maintainable, and well-documented code.
• Problem solving: Experience with algorithmic or data-intensive problems.
• Helpful background: SWE Bench or similar benchmark projects, or experience designing AI model evaluation tasks.
• Contract type: Contractor and part time.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work uses human-created data, code, and feedback to test and improve artificial intelligence systems. Experienced contributors are needed to check difficult technical tasks, identify errors, and help make model results more useful and reliable.