About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting for a Python AI Model Evaluation Developer to support the improvement of large language models through hands-on coding, data generation, and evaluation work. You will create high-quality examples for supervised fine-tuning, produce Python solutions to code-based questions, compare responses from different language models, and provide detailed feedback that can improve model quality. This role focuses on generating and evaluating training data rather than building or fine-tuning the models themselves.
What you will do
Design, develop, and maintain efficient, high-quality Python code for AI training and evaluation workflows.
Conduct evaluations to benchmark model performance and analyze results for continuous improvement.
Evaluate and rank AI model responses against defined quality, relevance, accuracy, and alignment criteria.
Create task-specific supervised fine-tuning datasets, model responses, code solutions, and evaluation rationales.
Create and refine responses to improve clarity, relevance, and technical accuracy.
Contribute to reinforcement learning from human feedback activities and reward-model refinement.
Design evaluation strategies, review code and documentation, and provide constructive technical feedback.
Collaborate with researchers and annotators while exploring tools and methods that strengthen AI training processes.
Required skills
At least three years of strong Python programming experience and familiarity with Python frameworks and libraries.
Knowledge of software-development quality, formatting, architecture, and best practices.
Experience with unit, integration, and property-based testing in Python.
Understanding of multithreading and asynchronous programming.
Ability to refactor code safely without introducing regressions and diagnose memory or concurrency issues.
Ability to write clear evaluation rationales and technically accurate responses.
Fluent conversational and written English communication.
Schedule and engagement
This is a fully remote contractor assignment lasting one month. A commitment of at least 20 hours per week and at least 4 hours per day is required, with 4 hours of overlap with Pacific Time. Available commitments are 20, 30, or 40 hours per week. Compensation is not disclosed in the role.
Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.