Tooling Engineer Business Analysis AI Evaluation

You will help evaluate and improve large language models used for business-analysis tasks. The work focuses on written prompts, AI-generated responses, test cases, and detailed feedback.

You will use business-analysis knowledge and careful judgment to find factual and reasoning problems that may affect model reliability.

• Write complex, business-analysis prompts that test language-model capabilities.
• Review AI-generated responses for factual accuracy, reasoning, relevance, and overall quality.
• Find factual errors, logical inconsistencies, knowledge gaps, and other weaknesses.
• Create difficult test cases and edge cases that expose limits in model performance.
• Explain your findings clearly and provide evidence-based feedback for model improvement.

What It Pays And Takes

The role is a remote contract position for an individual contributor. It is marked as part-time and requires at least 20 hours each week.

You should be able to make careful judgments, communicate precisely in writing, and challenge assumptions while spotting subtle factual or reasoning errors.

• Pay: Not provided in the listing.
• Work arrangement: Remote contract work.
• Schedule: Part-time, with a requirement of 20+ hours per week.
• Education: A master's degree or higher in a relevant field.
• Experience: At least three years of professional, research, or teaching experience.
• Language: Excellent written English communication skills.
• Core skills: Business-analysis knowledge, strong analytical and critical-thinking skills, and exceptional attention to detail.
• Helpful background: Experience with large language models, generative AI, prompt engineering, AI evaluation, model testing, difficult prompts, edge cases, or stress-testing AI models.

How It Works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI Training Work

AI training is the human work behind systems that learn from examples and feedback. People write prompts, review model responses, and explain errors so AI systems can become more accurate and reliable.

People with strong subject knowledge are valuable because they can identify subtle mistakes and judge whether an answer is sound, relevant, and well reasoned.