You will create and calibrate evaluation tasks for AI models based on computational work you have personally performed, debugged, or run in a research or technical setting. Tasks may cover data reduction, model fitting, simulation setup, instrument-data processing, numerical solvers, and inverse problems.
You will define the criteria and metrics for a strong solution without publishing exact passing thresholds. You will revise tasks after structured review and may also review and calibrate tasks submitted by other experts in your field.
- Design original evaluation tasks based on authentic scientific computing work.
- Turn personal research experience into clear, well-specified tasks.
- Set evaluation criteria that reflect real scientific difficulty.
- Revise tasks in response to structured review feedback.
- Optionally review and calibrate tasks from other experts.
What it pays and takes
This is remote, flexible, task-based independent-contractor work with no fixed weekly schedule. Each task takes roughly four hours, and the role description states $300 per approved task in addition to the $75 per hour compensation.
- Pay: $75 per hour.
- Work pattern: Flexible task-based assignments with no fixed weekly schedule.
- Location: Remote, open to candidates in the countries listed for this role.
- Language: Professional fluency in English.
- Education: A PhD, or a research-active Master's background supported by thesis work, research assistance, laboratory work, publications, or a clear doctoral trajectory.
- Experience: Active hands-on research or technical work in a scientific, mathematical, or engineering field.
- Technical skills: Working proficiency in Python and the ability to run and debug your own computational work.
- Work style: Ability to work independently and write clear, well-specified evaluation tasks.
Helpful background
Experience with scientific computing, data analysis, simulation, numerical methods, data annotation, data labeling, or AI and machine-learning evaluation may be useful. Relevant fields include biology, ecology, medicine, neuroscience, astronomy, chemistry, materials science, physics, atmospheric science, environmental science, geosciences, ocean science, mathematics, operations research, statistics, and chemical, civil, electrical, or mechanical engineering.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI training work
AI training work uses human judgment to create, review, and evaluate examples that help artificial intelligence systems improve. Researchers with strong subject knowledge are valuable because they can judge whether an AI solution is scientifically sound, not just whether it looks plausible.