AI-Generated Python Code Reviewer and Rater

You will review AI-generated Python code in real open-source projects and judge its quality, performance, and ability to fix bugs. The work includes running code locally, checking tests, and helping researchers find difficult tasks for AI models.

• Analyze and triage GitHub issues in active open-source libraries.
• Set up and configure repositories, including Docker environments.
• Review unit test coverage and assess test quality.
• Modify and run codebases locally to evaluate AI performance on bug-fixing tasks.
• Work with researchers to identify repositories and issues that challenge AI coding systems.

What it pays and takes

This is a remote contractor role with part-time classification and a short project period.

• Pay: $20 per approved task.
• Time: At least 20 hours per week, with four hours of overlap with Pacific Time.
• Location: Open to candidates in India, Pakistan, Nigeria, Egypt, Ghana, Bangladesh, Türkiye, or Mexico.
• Language: Professional fluency in English.
• Contract length: Two to three weeks, with an expected start next week.
• Strong Python experience.
• Proficiency with Git, Docker, and basic software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfort running, modifying, and testing real-world projects locally.
• Experience with open-source projects, LLM research or evaluation, developer tools, or automation agents is helpful.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind modern artificial intelligence, including reviewing model responses and evaluating computer code. OpenTrain helps people find these projects, build an AI training profile, and apply in minutes.