You will create and validate difficult engineering design problems used to train and evaluate AI agents. The work combines engineering judgment, simulation, coding-agent review, and benchmark development.
- Write self-contained engineering design tasks with competing constraints, clear optimization goals, validated reference solutions, and automated graders.
- Build, run, and validate simulation environments with open-source tools and custom Python test benches.
- Review coding-agent outputs and execution logs across repeated trials to find reasoning, tool-use, and simulator-feedback failures.
- Adjust benchmark difficulty using model performance data while keeping tasks clear and technically sound.
- Work with AI researchers and engineering specialists to add high-rigor benchmarks to evaluation workflows.
What It Pays and Takes
This is a remote contractor engagement. Compensation is not stated in the role description.
- Schedule: 40 hours per week, including four hours of overlap with Pacific Time; part-time work may be acceptable.
- Engagement length: expected to last up to 24 weeks.
- Availability: weekend on-call work is required.
- Location: worldwide remote; English is required.
- Education and experience: a master's degree or PhD in electrical, mechanical, aerospace, control systems, systems engineering, robotics, or an applied science field, plus at least three years of hands-on engineering design experience.
- Technical skills: strong Python scripting and experience with at least one relevant open-source simulation package, such as ngspice, PySpice, OpenFOAM, FEniCSx, CalculiX, python-control, CadQuery, build123d, OpenModelica, Cantera, or Gmsh.
- AI evaluation experience: practical knowledge of LLMs or coding agents, pass@k, failure-mode analysis, nondeterministic behavior, and trajectory-log review.
- Engineering judgment: careful attention to physical plausibility, unit consistency, boundary conditions, convergence criteria, and technical documentation.
- Equipment: a stable high-speed internet connection and a suitable computer.
How It Works
Apply on OpenTrain with your resume and then complete the application on the hiring site.
About AI Training Work
AI training work is the human work behind systems that learn from examples, including testing model answers, reviewing code, and evaluating how AI agents solve problems. Experienced engineers are paid for applying subject knowledge to create reliable examples, tests, and feedback that improve these systems.
