Sports LLM Evaluation Expert

You will help evaluate how well an AI model understands and answers sports questions. Your work will support model improvement by testing its knowledge, reasoning, accuracy, and handling of difficult cases.

You will work with AI researchers, document your findings, and maintain consistent annotation quality throughout the assignment.

• Create advanced prompts about global sports, leagues, athletes, tournaments, rules, statistics, and analytics.
• Review AI-generated responses for factual accuracy, reasoning quality, completeness, and nuance.
• Find hallucinations, logical inconsistencies, outdated information, and unusual edge cases.
• Build benchmark datasets and adversarial test cases for sports-focused model evaluation.
• Provide reliable references and objective, evidence-backed feedback.
• Maintain clear documentation and high annotation quality.

What It Pays and Takes

This is a remote contractor assignment. The listing does not specify a pay rate. The role is listed as entry level, but it requires advanced education and strong subject knowledge.

• Schedule: 40 hours per week for eight weeks.
• Time zone: At least four hours of overlap with Pacific Time each week.
• Location: Remote and open worldwide.
• Language: Excellent written English.
• Education: A master's degree or higher is required.
• Sports knowledge: Strong knowledge of one or more sports, including major leagues, teams, athletes, tournaments, rules, statistics, and important developments.
• Preferred experience: Three or more years in sports journalism, sports media, sports content, research, analysis, reporting, or a related field.
• Preferred background: Experience with large language models, generative AI, prompt engineering, or AI evaluation.
• Core strengths: Research, analysis, careful attention to detail, and objective evidence-backed reviews.

How It Works

Apply on OpenTrain with your resume and then complete the application on the hiring site.

About AI Training Work

AI training work uses human-written examples and reviews to help artificial intelligence systems produce better results. People with strong subject knowledge are paid to check model outputs, find errors, and explain what a better answer should contain.