About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting for an AI Agent Evaluator to assess how two AI agents handle realistic errands and digital tasks from a user's perspective. You will test agents side by side on a smartphone, make decisions at important checkpoints, capture screen recordings, and evaluate the quality and reliability of each experience. Your observations will help improve AI agents that assist with everyday activities such as payments, bookings, shopping, communications, and forms.
What you will do
You will run realistic scenarios involving bills, returns, email follow-ups, bookings, shopping, groceries, travel, family logistics, outings, and forms. You will approve or stop an AI agent at key steps, screen record every attempt using a smartphone, and score completion, quality, control issues, satisfaction, and time for each run. You will also write clear feedback describing what worked, what failed, and how the interaction felt. Some tests may require using personal AI accounts and making real purchases with your own card, with reimbursement afterward.
Required skills
This role requires U.S. residence, a bachelor's degree, two to three years of experience preferably in engineering, business management, or office operations, regular personal use of AI tools, strong attention to detail, and the ability to follow instructions precisely. You should have clear written communication skills, be comfortable documenting each step, and have a smartphone capable of screen recording with an active personal AI account.
Pay and schedule
This remote contractor role pays $25‑$45 per hour. Work is completed on a flexible schedule, and each task run takes approximately 1.5 hours, including evidence capture and scoring.
Helpful background
Generalists are welcome, while experience in a relevant domain can be helpful. Coding experience and a formal AI background are not required. The role is designed for independent contractors who can complete structured mobile testing and provide thoughtful, reliable evaluations.
Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.
You apply here in about two minutes and finish on the hiring platform. The next step is a short AI interview (technical roles include a coding challenge). Passing it is how you qualify for paid tasks, so take it soon after you are invited.
The hiring platform's application for this job asks for your resume, your LinkedIn profile, your phone number, your location and answers to 8 screening questions.