About OpenTrain
OpenTrain is the centralized, open platform where AI training and data-labeling freelancers find work, consolidate opportunities across platforms, and build a unified AI training portfolio they control. We make specialized AI training work easier to track, apply for, and grow into a durable freelance career.
About this role
OpenTrain is recruiting for a Legal AI Evaluation and Rubric Design Expert to create challenging legal and compliance scenarios, evaluate AI-generated analysis, and translate professional legal judgment into structured evaluation criteria. You will help produce reliable training data by assessing whether model responses identify material issues, apply appropriate rules, recognize exceptions, distinguish facts from assumptions, address risks, and communicate uncertainty responsibly. The work can cover contracts, privacy, financial regulation, employment matters, corporate compliance, regulatory interpretation, policy analysis, and related specialties.
What you will do
Develop realistic legal and compliance questions that require expert analysis and professional judgment. Design rubrics covering issues, applicable standards, reasoning steps, completeness, exceptions, jurisdiction, risk, and critical errors. Evaluate AI-generated responses for accuracy, relevance, completeness, reasoning quality, and professional rigor. Identify unsupported conclusions, incorrect rule application, missing exceptions, faulty assumptions, and poor risk prioritization. Review peer rubrics for ambiguity, redundancy, excessive subjectivity, hidden assumptions, and inaccurate legal expectations. Create reference analyses, answer frameworks, critiques, preference data, and reference-answer pairs for RL and SFT workflows. Maintain clear distinctions between factual analysis, legal interpretation, professional judgment, and uncertainty.
Required skills
A JD, LLB, LLM, law degree, compliance qualification, or equivalent professional background. At least three years of experience in law, regulatory compliance, legal research, policy, risk, privacy, financial compliance, contracts, or a related field. Demonstrated expertise in a clearly defined legal, regulatory, or compliance domain. Strong professional English proficiency at minimum C1 level. Ability to identify controlling issues, assess rule application, weigh material risks, and recognize legitimate alternative interpretations. Strong rubric-writing skills, including the ability to define required issues, standards, exceptions, omissions, and reasoning quality. Careful attention to terminology, authority, jurisdiction, factual assumptions, materiality, and conclusions.
Helpful background
Experience with legal education, examination writing, peer review, legal editing, quality assurance, policy drafting, compliance testing, supervisory review, AI evaluation, legal AI, data annotation, RLHF, model benchmarking, or rubric-based scoring is valuable.
Why work with OpenTrain
OpenTrain gives freelancers one place to manage AI training opportunities instead of rebuilding proof of work on every platform. A stronger OpenTrain profile helps you show credible experience, discover roles that match your skills, and turn AI training and data-labeling work into a long-term portfolio.