Skip to content
OpenTrain AIFor AI Companies

AI Agent Tester for Real-World Task Evaluation

Apply now

Job details

Pay

$25–$45 USD/hr

Engagement

Contract, part-time

Location

Remote

United States only

Posted

Oct 24, 2026

OpenTrain on Trustpilot

The work

About OpenTrain. OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.

About this role. OpenTrain is recruiting for a contractor role evaluating how well AI agents handle realistic everyday errands. You will use two AI agents side by side on your phone, approve or stop them at important moments, capture screen recordings, score each run, and explain what worked, what failed, and how the experience felt. Your observations will help improve AI agents that support activities such as bill payment, returns, bookings, shopping, travel, forms, and family logistics.

What you will do. You will run realistic scenarios involving bills, returns, email follow-ups, bookings, shopping, groceries, travel, outings, forms, and other everyday errands. You will use your own phone and accounts to test two AI agents in comparable situations, approve or stop an agent when a key action requires user control, and screen record every attempt. You will document the complete interaction, score each run for completion, quality, control issues, satisfaction, and time, and provide clear written feedback about the agent experience.

Required skills. Candidates should reside in the United States, hold a bachelor's degree, use AI in daily life, and have two to three years of experience, preferably in engineering, business management, office, or operations work. Strong attention to detail, precise instruction following, clear written communication, and careful step-by-step documentation are required. Candidates must have an active personal AI assistant account on the Plus tier, active Muse and Dots access, a smartphone capable of screen recording, and a payment card for test purchases that are reimbursed afterward.

Helpful background. Generalists are welcome, while experience in a relevant domain can be helpful. No coding background is required. Familiarity with real-world errands, mobile applications, and documenting user experiences can support strong performance in this work.

Location and pay. This is a U.S.-based contractor opportunity paying $25‑$45 per hour. Work is completed remotely using a smartphone and the required personal accounts and testing tools.

Why work with OpenTrain. OpenTrain connects independent specialists with meaningful AI training opportunities and makes it easier to manage projects, discover new work, and build a durable record of experience. You can contribute practical expertise to systems that are becoming part of everyday life while working as an independent contractor.

You apply here in about two minutes and finish on the hiring platform. The next step is a short AI interview (technical roles include a coding challenge). Passing it is how you qualify for paid tasks, so take it soon after you are invited.

The hiring platform's application for this job asks for your resume, your LinkedIn profile, your phone number, your location and answers to 8 screening questions.

Requirements

  • Languages: English
  • Skills: ChatGPT, Muse, AI agents, Attention to Detail, Following Instructions, Screen Recording

Open to applicants in

  • United States

How to apply

  1. Apply here on OpenTrain. You create a free account, and we send you to the hiring platform.
  2. Complete your application on the hiring platform.

OpenTrain on Trustpilot

Job ID: cmvlpm1md001l08kr1nivrfdc

Keep exploring

Similar open roles

View all AI training jobs

Applied AI Agent Evaluation Research Engineer

About OpenTrain. OpenTrain is the unified marketplace where AI training freelancers find and consolidate opportunities across platforms, build an AI training portfolio, and develop a durable freelance career. We help ca…

AI Agent Evaluation & Benchmark DesignPython & Research EngineeringAgent Failure Analysis & ExperimentationLLM & Agentic Systems
Text
Remote · United Arab Emirates, Argentina, Austria +49 more
English
Flexible hours
Hourly · $96.15–$125/hr

Posted Nov 17, 2026

AI Agent Evaluator

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

ChatGPTMuseAI agentsAttention to DetailFollowing InstructionsScreen Recording
Video
Remote · United States
English
Part-time · Flexible
Hourly · $25–$45/hr

Posted Nov 16, 2026

Python Coding-Agent Benchmark Expert

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Python
Code
Remote · Worldwide
English
Part-time · Flexible

Posted Nov 18, 2026

Python Domain Expert - Coding Agent Benchmarking

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

Python
Code
Remote · Worldwide
English
Part-time · Flexible

Posted Nov 18, 2026

AI Agent Tester

About OpenTrain OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control.…

ChatGPTMuseAI agentsAttention to DetailFollowing InstructionsScreen Recording
Video
Remote · Austria, Belgium, Switzerland +9 more
English
Part-time · Flexible
Hourly · $25–$45/hr

Posted Nov 16, 2026