Retail AI Model Evaluation Specialist

About OpenTrain

OpenTrain is the #1 platform for people building careers in AI training and data labeling. We help freelancers discover and manage specialized AI training work, build a unified portfolio, and grow long-term proof-of-work that demonstrates domain expertise.

• We focus on the human side of AI: real people teaching models through annotation, evaluation, and feedback.
• OpenTrain supports remote, flexible contracting work that can be part-time or scaled into a career.

About AI training work

AI training (data labeling, annotation, and human evaluation) is how modern AI systems learn and improve. Contributors review model outputs, rate correctness, craft high-quality examples, and refine guidelines so models behave more reliably in the real world.

This role sits at the intersection of retail domain expertise and generative-AI evaluation: your retail judgment will directly shape how AI models handle merchandising, category management, and retail operations scenarios.

The role

As a Retail AI Model Evaluation Specialist you will evaluate model outputs against structured rubrics, write clear feedback and solutions grounded in retail practice, and help design and refine scoring guidelines for retail-specific tasks.

This is a US-based contractor, part-time role. Pay is hourly and ranges from $60 to $80 per hour depending on experience and performance.

• Work type: Contractor, part-time
• Location: United States only
• Pay: USD $60–$80 per hour

What you'll do

• Review AI model outputs for retail tasks using structured rubrics and scoring criteria.
• Write accurate, well-reasoned solutions and feedback grounded in real retail practice.
• Design and refine domain-relevant evaluation guidelines, rubrics, and scoring rules.
• Close knowledge gaps in merchandising, category management, and retail operations reasoning.
• Collaborate with other subject matter experts to improve consistency and data quality.

Requirements

You must be able to bring deep, hands-on retail judgment to evaluation work and communicate clearly in writing and verbally. Preserve strong attention to detail when applying scoring rubrics and explaining decisions.

• 8+ years professional experience in retail (merchandising, category management, retail operations, buying/planning).
• Hands-on experience evaluating LLM or AI model outputs against rubrics or structured scoring criteria.
• Ability to write clear, well-reasoned judgments on correctness and reasoning quality.
• Strong written and verbal communication, problem-solving, and interpersonal skills.
• Reliable weekday availability for at least 35 hours per week (role expects an ongoing weekly commitment).
• Language: English required.
• Work authorization: must be located in and authorized to work in the United States.

Helpful background

We welcome candidates with senior retail experience and demonstrable career progression. Experience at major retail organizations is useful but not required.

• Experience at recognized retailers (for example: Amazon, Walmart, Target, Nike, Costco, Home Depot) is a plus.
• Prior roles in merchandising, category leadership, or retail operations that show increasing responsibility.