Python Domain Expert for LLM Evaluation

You will test and improve large language models, which are AI systems that generate text. Your work will focus on Python and business analysis, helping identify where model responses are inaccurate, unclear, incomplete, or poorly reasoned.

You will create difficult prompts and test cases, review the responses they produce, and explain your findings in enough detail to guide model improvement.

• Create complex prompts that test language-model capabilities.
• Evaluate responses for factual accuracy, reasoning, relevance, and overall quality.
• Find factual errors, logical inconsistencies, knowledge gaps, and other weaknesses.
• Design edge cases that expose limitations in model behavior.
• Provide clear, evidence-based feedback on how responses could improve.

What It Pays and Takes

The listing does not provide pay details. This is a part-time contractor role with a 20-plus-hour weekly commitment, open worldwide.

• Pay: Not provided in the listing.
• Hours: 20 or more hours per week.
• Work arrangement: Part-time contract work.
• Location: Open worldwide.
• Language: Fluent written English.
• Experience level: Entry level.
• Education: A master's degree or higher in a relevant field.
• Experience: At least three years of professional, research, or teaching experience.
• Core skills: Deep knowledge of Python and business analysis.
• Other requirements: Strong analytical and critical-thinking skills, exceptional attention to detail, and clear written communication.

Helpful Background

Experience with large language models or generative AI can help. Familiarity with prompt engineering, AI evaluation, model testing, difficult prompts, edge cases, or stress-testing AI systems is also useful.

• Large language model or generative AI experience
• Prompt engineering
• AI evaluation or model testing
• Designing challenging prompts and edge cases
• Finding subtle factual or reasoning errors

How It Works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI Training Work

AI training is the human work behind systems that generate text and other content. Contributors write prompts, review model responses, and explain errors so AI systems can become more accurate, useful, and reliable.