Legal Research AI Evaluation SME

About OpenTrain

OpenTrain AI is the hiring and contracting organization for flexible work in AI training and data labeling. We help people build careers teaching and evaluating artificial intelligence through specialized projects that can be completed remotely.

In this role, your legal judgment will help improve how AI systems research issues, handle jurisdictional differences, select authority, and explain conclusions.

• Remote contract work with hiring across Germany, India, and the United States
• Part-time schedule of 20+ hours per week
• Hourly work supporting the development and evaluation of AI systems

About AI Training and Legal Evaluation

AI training is the human side of building modern artificial intelligence. Subject-matter experts review model outputs, identify errors, write strong examples, and provide feedback that helps systems become more accurate and useful.

Legal research evaluation applies this process to issue analysis, authority selection, citation support, rule application, and the defensibility of written conclusions.

• Evaluate multiple AI responses against expert standards
• Create high-quality examples that demonstrate sound legal reasoning
• Help identify misleading certainty, bias, jurisdictional mistakes, and unsupported claims

The Role

OpenTrain AI is seeking a Legal Research AI Evaluation SME to review AI-generated legal research outputs and produce expert legal research content. You will assess reasoning quality across issue analysis, jurisdiction handling, authority selection, citation accuracy, and rule application.

The role requires precise written feedback, primary-authority support checking where required, model research memos, authority summaries, and careful ratings of response correctness, clarity, completeness, and defensibility.

• Employment type: Part-time contractor
• Experience level: Intermediate
• Data type: Text
• Workload: 20+ hours per week
• Language: Professional English proficiency at minimum C1 level
• Advertised rates vary by location, from up to $40/hour to up to $110/hour

What You'll Do

You will evaluate legal research responses with close attention to both substantive accuracy and the quality of the reasoning presented. Your work will include reviewing sources, testing assumptions, and documenting why an answer is reliable or deficient.

• Review AI-generated legal research answers for accuracy, clarity, completeness, and prompt adherence.
• Identify errors in issue spotting, jurisdiction handling, authority selection, citation accuracy, and rule application.
• Write explanations and model research memos that demonstrate careful legal reasoning.
• Create detailed prompts and gold-standard outputs, including issue outlines, research plans, memo templates, and authority summaries.
• Evaluate and rank model responses for correctness, defensibility, citation support, and clarity.
• Test outputs for misleading certainty, jurisdictional errors, bias, and reliability across legal scenarios.

Requirements

You should have a JD, LLB, or equivalent legal research background with substantial experience conducting primary-law research. Strong knowledge of statutes, regulations, and case law is essential, including the ability to distinguish holdings from dicta and assess procedural posture relevance.

You must be able to handle jurisdictional variation, state assumptions and limitations clearly, and produce concise legal writing in professional English.

• Strong experience interpreting statutes, regulations, and case law
• Structured legal research skills from issue framing through authority retrieval and synthesis
• Strong citation hygiene, including quotation verification, pinpoint citations, and proposition support
• Ability to assess whether authorities genuinely support a stated proposition
• Ability to handle multi-jurisdiction questions and document assumptions clearly
• Experience reviewing AI-generated legal research for accuracy, defensibility, and clarity

Preferred Background

Experience with legal research tools such as Westlaw, Lexis, or equivalent platforms is helpful. Prior work in AI training, annotation, editorial quality assurance, knowledge management, or legal publishing is strongly preferred.

You should also be comfortable working across time zones in an hourly remote contractor setting.

• Legal publishing or editorial quality assurance experience
• Prior AI training, data annotation, or model evaluation experience
• Experience comparing authorities and identifying weak or incomplete support
• Experience with multi-jurisdiction legal research
• Comfort creating structured research workflows and high-quality written examples

How to Stand Out

Strong candidates demonstrate disciplined legal research judgment and concise written analysis. Highlight examples of comparing authorities, checking citations, distinguishing procedural contexts, and identifying unsupported conclusions.

When applying through OpenTrain, emphasize any experience evaluating AI outputs or producing reliable legal research content, along with the jurisdictions and research tools you know best.

• Show careful analysis of authority strength and proposition support
• Demonstrate clear handling of jurisdictional assumptions and limits
• Highlight experience with AI evaluation, annotation, QA, or legal publishing
• Explain how you identify errors while preserving clarity and defensibility