Legal AI Evaluation Attorney

You will review AI-generated responses to U.S. legal hypotheticals across several areas of law. Your evaluations will help measure how well AI systems identify legal issues, apply the law, and explain their reasoning.

You will also document difficult edge cases, reasoning gaps, and blind spots that can support legal AI benchmarks. Detailed annotations and written feedback will help improve model responses, while collaboration with research and project teams will support consistent evaluation standards.

• Review AI-generated responses to U.S. legal hypotheticals.
• Rate issue spotting, legal accuracy, reasoning quality, and application of law using a structured rubric.
• Identify edge cases, gaps, and blind spots in legal reasoning.
• Write detailed annotations and feedback for model improvement.
• Work with research and project teams to align evaluation standards and project goals.

What It Pays And Takes

This is a remote contractor project with a four-week schedule. Previous experience with AI annotation, model evaluation, or structured feedback is helpful but not required.

• Pay: Up to $150 per hour, depending on internal evaluation and location.
• Project length: 40 total hours over four weeks.
• Work type: Remote, part-time contract project.
• Education: Juris Doctor degree from an accredited U.S. law school.
• Bar admission: Active admission to a U.S. bar required.
• Experience: At least three years of U.S. legal practice.
• Skills: Strong reasoning across core areas of U.S. law, excellent written communication, careful attention to detail, and high ethical standards.
• Helpful background: Experience analyzing complex legal hypotheticals and explaining the basis for legal conclusions.

How It Works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI Training Work

AI training is the human work behind systems that answer questions and generate content. Contributors review model responses, rate their quality, and explain what needs to improve so AI systems can give more accurate and useful answers. Legal professionals are valued for applying careful subject-matter judgment to specialized evaluation tasks.