Psychology Quality Assurance Lead

About OpenTrain

OpenTrain is the #1 platform for people building careers in AI training and data labeling. We help freelancers discover projects, consolidate proof of work, and grow a durable portfolio of AI training experience.

As the hiring organization for this role, OpenTrain provides long-term, remote opportunities that put you on the cutting edge of how AI is trained and evaluated.

About AI training and this work

AI training (also called data labeling, annotation, or human feedback work) is the human side of building intelligent systems. People prepare, review, and rate examples—like written explanations or safety checks—that models learn from.

This role focuses on psychology-focused evaluation: you will assess clarity, accuracy, ethical awareness, mental-health safety, and rubric adherence in AI-generated psychology outputs, helping shape safer and more research-aligned models.

The role

You will lead quality assurance for psychology-centered AI training projects: review AI-generated explanations, mental-health content, research summaries, behavioral guidance, and applications of psychological theory.

As a contractor and part-time contributor, you will evaluate trainer and QA work, write corrective feedback, update documentation and style guides, support onboarding, and help improve quality workflows across distributed teams.

What you'll do day-to-day

• Review psychology-focused AI outputs and trainer/QA annotations for accuracy, clarity, safety, and consistency.
• Assess work against detailed rubrics and flag unsupported or unsafe claims.
• Provide clear, actionable written feedback and identify recurring issues for retraining.
• Maintain and update style guides, calibration materials, and documentation.
• Communicate guideline and workflow changes to trainers and QA teams.
• Support onboarding and calibration of new reviewers to ensure consistent ratings.

Required skills and experience

You must have a strong background in psychology, clinical psychology, counseling, behavioral science, or a closely related field and be comfortable evaluating mental-health content against research-informed standards.

• Experience with psychology research, mental-health content review, or related workflows.
• Strong written English skills for producing clear, constructive feedback and documentation.
• Ability to evaluate content against rubrics and identify ethical or diagnostic concerns.
• Familiarity with CBT, behavioral interventions, mental-health literacy, research design, statistics, and ethical guidelines is a plus.

Helpful qualifications

• Experience leading or supporting remote teams or QA programs.
• Prior work with psychology QA, LLM evaluation, or rubric-based review systems.
• Comfort working with text-only data labeling and evaluation tasks (label type: evaluation/rating).

Schedule, location, and compensation

This is remote contractor work for applicants based in the United States. Expected time commitment is 20+ hours per week, with flexible scheduling for future projects.

Compensation is up to $80 per hour. Employment types: contractor, part-time.

Who should apply and how it works

This role is a fit if you want flexible, impactful work that directly shapes how models handle psychology and mental-health topics. Typical contributors use OpenTrain to build a unified portfolio of AI training experience and grow into recurring projects.

To apply, create an OpenTrain account (free), complete your profile with relevant psychology experience, and submit your application. If selected, you will participate in onboarding and calibration before beginning reviews.