About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting for a remote AI Model Evaluation Data Scientist to develop Python solutions and analyze datasets that support AI model training and improvement. You will work across model evaluation, response ranking, supervised fine-tuning, and reinforcement learning with human feedback, combining technical analysis with clear written reasoning. The role is designed for a data scientist or analyst who can translate complex findings into useful improvements for AI systems.
What you will do
Design, develop, and maintain high-quality Python code for training and optimizing AI models. Conduct evaluations to benchmark model performance and analyze results. Evaluate and rank model responses to user queries using predefined criteria. Write comprehensive explanations and rationales for evaluation decisions. Create and maintain task-specific datasets for supervised fine-tuning. Collaborate with researchers and annotators on reinforcement learning with human feedback and reward-model refinement. Create and refine responses for clarity, relevance, and technical accuracy. Review code and documentation, identify issues, and provide constructive feedback. Use public datasets, including data from Kaggle, the UN, and the US government, to answer business questions.
Required skills
Proficiency in Python and strong data analysis ability. A bachelor's or master's degree in engineering, computer science, or equivalent experience. Strong problem-solving skills and sound business judgment when interpreting data. The ability to explain technical reasoning clearly in Jupyter notebooks or comparable formats. Effective communication with researchers and other stakeholders. Fluent conversational and written English.
Work arrangement
This is a fully remote contractor assignment with a one-month contract term. The expected commitment is at least 20 hours per week, with options for 20, 30, or 40 hours per week and four hours of daily overlap with Pacific Time.
Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.