About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting an AI Evaluation Engineer focused on engineering simulation and design. You will create rigorous simulation-based design problems across electrical, mechanical, and aerospace engineering to train and evaluate AI agents. The work combines engineering judgment, open-source simulation, Python development, agent evaluation, and objective benchmark design.
What you will do
Author self-contained engineering tasks with competing constraints, optimization targets, validated reference solutions, and objective autograders.
Build, run, and validate simulation environments using open-source tools and custom Python test benches.
Review coding-agent outputs and execution logs across repeated trials to identify failures such as misinterpreted simulator feedback, premature convergence, and physically impossible geometries.
Calibrate benchmark difficulty using empirical model performance while preserving clear requirements and technically valid solutions.
Collaborate with AI researchers, pod leads, and engineering experts to integrate high-rigor benchmarks into evaluation workflows.
Required skills
Bachelor's degree or higher in electrical, mechanical, aerospace, or a closely related engineering discipline.
At least five years of hands-on engineering design experience.
Strong Python skills and proficiency with at least one relevant open-source simulation package, such as ngspice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, OpenModelica, or Gmsh.
Experience with LLMs or coding agents, AI evaluation concepts such as pass@k, trajectory-log auditing, and systematic failure diagnosis.
Careful application of physical plausibility, unit consistency, boundary conditions, convergence criteria, and technical documentation.
Experience in AI evaluation, data annotation, content review, quality assurance, or a related analytical role.
Schedule and engagement
This is a remote contractor engagement lasting up to 12 weeks. Work may be up to 40 hours per week and requires four hours of overlap with Pacific Time, along with weekend on-call availability. Part-time engagement is acceptable.
Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.