Python AI Model Evaluation Developer

You will help improve large language models through Python coding, training-data creation, and response evaluation. The work focuses on preparing and reviewing examples rather than building or fine-tuning the models themselves.

• Write and maintain efficient Python code for AI training and evaluation workflows.
• Create Python solutions, supervised fine-tuning examples, model responses, and evaluation rationales.
• Compare and rank model responses for quality, relevance, accuracy, and alignment.
• Benchmark model performance, analyze results, and support evaluation strategy design.
• Refine responses for clarity, relevance, and technical accuracy.
• Support reinforcement learning from human feedback and reward-model improvement.
• Review code and documentation, refactor code safely, and provide constructive technical feedback.
• Work with researchers and annotators on tools and methods for AI training.

What It Pays And Takes

This is a fully remote contractor assignment lasting one month. Compensation is not disclosed in the role information.

• Pay: Compensation not disclosed.
• Hours: Choose a commitment of 20, 30, or 40 hours per week.
• Daily schedule: At least 4 hours per day is required, including 4 hours of overlap with Pacific Time.
• Location: Fully remote and open worldwide.
• Language: Fluent conversational and written English.
• Experience: At least three years of strong Python programming experience.
• Technical skills: Familiarity with Python frameworks and libraries, software-development quality standards, formatting, architecture, and best practices.
• Testing: Experience with unit, integration, and property-based testing in Python.
• Programming concepts: Understanding of multithreading and asynchronous programming.
• Problem solving: Ability to diagnose memory or concurrency issues and refactor code without introducing regressions.
• Communication: Ability to write clear evaluation rationales and technically accurate responses.

How It Works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI Training Work

AI training is the human work behind systems that learn from examples, including writing code, rating model responses, and preparing feedback. OpenTrain helps people discover these roles, build a profile, and apply in one place, while experienced contributors can use specialized skills to work on advanced AI projects.