Finance LLM Evaluation & Rubric Design Expert

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We help people start and grow durable freelance careers teaching AI by consolidating specialized AI training opportunities and enabling contributors to build a unified portfolio of work they control.

OpenTrain AI is the hiring organization for this role. We focus on high-quality, subject-matter-driven training data that improves how state-of-the-art AI systems perform in specialized domains like finance.

Why AI training work matters in finance

AI training (also called data labeling or human-in-the-loop evaluation) is the human side of building intelligent systems. For finance applications, expert human judgment is essential to teach models accurate reasoning about valuation, risk, accounting, and decision-making.

Contributors in this field work remotely, often on flexible schedules, and directly influence how models behave in real-world financial situations. This role puts your practical finance experience at the center of improving model safety and usefulness.

Role overview

We are recruiting a Finance LLM Evaluation & Rubric Design Expert to design finance-specific evaluation tasks, author model solutions, build and refine scoring rubrics, and evaluate AI outputs with clear, actionable written feedback.

This is a remote contractor role open to candidates based in the United States. OpenTrain AI expects reliable weekday availability and a weekly commitment of 35 hours.

• Position type: Contractor, part-time (35 hours/week expected)
• Location: Remote, US-based candidates only
• Pay: $65–$90 per hour

What you'll do

Your day-to-day work centers on creating rigorous, domain-relevant evaluation artifacts and applying your finance expertise to judge model outputs against structured criteria.

• Design challenging finance tasks and write accurate, well-reasoned solutions grounded in real financial practice.
• Evaluate AI-generated responses using structured rubrics and provide detailed written feedback on correctness, judgment, and reasoning.
• Develop and refine evaluation guidelines and scoring rubrics tailored to finance scenarios.
• Collaborate with other subject-matter experts to ensure scoring consistency and high-quality training data.
• Work with research and engineering teams to identify and close gaps in models' financial reasoning and analysis.

Requirements

Candidates must meet the core professional and availability requirements listed below. We preserve these requirements exactly as part of the role.

• 8+ years of dedicated professional finance experience at a recognized, top-tier organization (examples listed in the job description).
• Prior hands-on experience evaluating LLM/AI model outputs against rubrics or other structured scoring criteria.
• Demonstrable career progression (e.g., Analyst → Associate → VP/Director).
• Ability to engage reliably for at least 35 hours per week during weekdays.
• Strong verbal and written communication, problem-solving, and interpersonal skills.

Helpful background (nice-to-have)

These skills will strengthen your application but are not listed as strict requirements.

• Experience designing or refining scoring rubrics for complex, open-ended tasks.
• Familiarity with financial modeling, valuation, or corporate finance frameworks.

Compensation, schedule, and logistics

This is an hourly contractor role paying between $65 and $90 per hour. A commitment of 35 hours per week is expected and should be during weekdays.

Work is fully remote and available to candidates located in the United States. The role involves evaluating text outputs and designing text-based tasks and rubrics.

• Pay: $65–$90 / hour (hourly position)
• Weekly commitment: 35 hours (weekday availability required)
• Data type: Text — label types include evaluation rating and text generation

How the work is done & how to apply

OpenTrain contributors use our platform to receive tasks, access rubrics and guidelines, submit evaluations, and document feedback. You will be expected to produce high-quality written feedback and follow scoring guidelines closely while also proposing rubric improvements when necessary.

To apply, create or update your OpenTrain profile, highlight your finance career progression and rubric evaluation experience, and submit any requested samples or assessments. OpenTrain will review and invite qualified candidates to onboarding and calibration tasks.

• Workflows include designing tasks, scoring model outputs, writing model solutions, and refining rubrics.
• Onboarding typically includes calibration exercises to align scoring with other subject-matter experts.