Multimodal AI Annotation Quality Operations Lead

You will lead quality operations for AI annotation projects involving chatbot comparisons and final assessments. Your work will help keep evaluation results consistent, accurate, and on schedule.

• Lead Senior QA Leads and QA Leads across shifts.
• Set review priorities, sampling plans, and quality coverage for evaluation work.
• Create and run audits, scoring rubrics, and calibration processes.
• Track accuracy, agreement, error rates, review throughput, and turnaround time.
• Find root causes and drive corrective actions with team leads.
• Resolve guideline questions and document quality decisions.
• Share quality insights after each delivery and report performance to the Quality Manager.
• Support quality reviews and quality-based incentive programs.
• Compare multiple chatbot responses and provide consistent final assessments.

Schedule and requirements

This is a remote, project-based freelance role lasting 16 weeks. It is a contract, part-time position with a schedule of five working days each week, including weekends when required, and two weekdays off.

• Work 20 or more hours per week.
• Work from 7:00 a.m. to 4:00 p.m. Pacific Time.
• Open to candidates worldwide.
• Be fluent in English.
• Bring five to seven years of overall professional experience.
• Have two to three years of experience leading QA or quality operations teams.
• Understand sampling, auditing, calibration, and root-cause analysis.
• Have strong attention to detail and the ability to identify systemic issues.
• Communicate clearly and manage stakeholder relationships effectively.
• Experience with annotation, data labeling, or AI training data projects is preferred.
• Familiarity with multimodal AI annotation workflows and quality measurement is useful.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training work is the human work behind systems that understand text, images, audio, and conversations. People compare model responses, label data, and review quality so AI systems can improve, and experienced contributors are needed for complex evaluation and quality work.