Skip to content
OpenTrain AIFor AI Companies

Python AI Model Evaluation Developer

Apply now

Job details

Engagement

Contract, part-time

Location

Remote

Worldwide

Posted

Jul 16, 2026

OpenTrain on Trustpilot

The work

About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role
OpenTrain is recruiting for a Python AI Model Evaluation Developer to support the improvement of large language models through hands-on coding, data generation, and evaluation work. You will create high-quality examples for supervised fine-tuning, produce Python solutions to code-based questions, compare responses from different language models, and provide detailed feedback that can improve model quality. This role focuses on generating and evaluating training data rather than building or fine-tuning the models themselves.

What you will do
Design, develop, and maintain efficient, high-quality Python code for AI training and evaluation workflows.
Conduct evaluations to benchmark model performance and analyze results for continuous improvement.
Evaluate and rank AI model responses against defined quality, relevance, accuracy, and alignment criteria.
Create task-specific supervised fine-tuning datasets, model responses, code solutions, and evaluation rationales.
Create and refine responses to improve clarity, relevance, and technical accuracy.
Contribute to reinforcement learning from human feedback activities and reward-model refinement.
Design evaluation strategies, review code and documentation, and provide constructive technical feedback.
Collaborate with researchers and annotators while exploring tools and methods that strengthen AI training processes.

Required skills
At least three years of strong Python programming experience and familiarity with Python frameworks and libraries.
Knowledge of software-development quality, formatting, architecture, and best practices.
Experience with unit, integration, and property-based testing in Python.
Understanding of multithreading and asynchronous programming.
Ability to refactor code safely without introducing regressions and diagnose memory or concurrency issues.
Ability to write clear evaluation rationales and technically accurate responses.
Fluent conversational and written English communication.

Schedule and engagement
This is a fully remote contractor assignment lasting one month. A commitment of at least 20 hours per week and at least 4 hours per day is required, with 4 hours of overlap with Pacific Time. Available commitments are 20, 30, or 40 hours per week. Compensation is not disclosed in the role.

Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.

Requirements

  • Languages: English

Open worldwide

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmrnidyjc001f04l5473za1zj

Keep exploring

Similar open roles

View all AI training jobs

Senior Python Backend Engineer, AI Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Python Backend Engineer, AI Code Evaluation

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

Senior Python Engineer, LLM Evaluation

About OpenTrain. OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptJavaC++Rust
Code
Remote · United States
English
Part-time · Flexible

Posted Nov 6, 2026

AI Model Evaluation Trainer

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Video AnnotationAttention to DetailData LabelingVideo Annotation ToolsTime Management
Text
Remote · Germany
English
Part-time · Flexible
Intermediate level
Hourly · $14–$36/hr

Posted Sep 23, 2026

AI Coding Evaluation and Training Developer

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

PythonJavaScriptTypescriptNode.jsNest.jsReact
Code
Remote · Worldwide
English
Part-time · Flexible

Posted Jul 17, 2026