Cybersecurity AI Evaluation Expert

About OpenTrain

OpenTrain AI is the #1 platform for finding and building careers in AI training and data labeling. We help contributors discover specialized projects, build an AI training profile, and apply in minutes. Creating an OpenTrain account is free.

About AI Evaluation Work

AI training is the human side of building artificial intelligence. Experienced professionals create scenarios, write strong reference responses, and evaluate model outputs so AI systems can become more accurate, useful, and reliable in real-world settings.

• Work remotely on cutting-edge AI systems.
• Apply specialized professional judgment to realistic evaluation tasks.
• Help shape how advanced models perform in cybersecurity contexts.

The Role

OpenTrain AI is seeking a Cybersecurity AI Evaluation Expert for an ongoing hourly contractor engagement. You will build evaluation tasks for AI systems operating in security operations, incident response, and risk management contexts.

This remote role pays $80-$90 per hour and is described as a 40-hour-per-week commitment. The structured project details also identify a requirement of 20 or more hours per week. There are no geographic restrictions.

• Engagement type: Contractor
• Work arrangement: Remote, worldwide
• Pay: $80-$90 per hour in USD
• Schedule: 20+ hours per week; role description specifies a 40-hour-per-week commitment
• Data type: Text
• Primary language: English

What You'll Do

You will design challenging, realistic cybersecurity tasks that test whether AI systems can demonstrate authentic security judgment rather than generic textbook or certification-exam-level recall.

• Construct scenarios covering security operations center monitoring, incident response and forensics, vulnerability management, and security architecture design.
• Develop tasks across threat detection, digital forensics, vulnerability and penetration testing, security architecture and engineering, and governance, risk, and compliance.
• Apply threat modeling, incident response playbooks, and risk assessment methods.
• Work with US and international standards tracks, including NIST CSF, SOC 2, ISO 27001, and NIS2.
• Produce reference incident reports, security assessments, architecture designs, and compliance documentation.
• Write evaluation rubrics that distinguish expert security reasoning from surface-level answers.

Required Experience

This role is intended for an experienced cybersecurity professional with substantial ownership of enterprise security programs. The source listing identifies five or more years as a security engineer or CISO at a major company or security firm as a requirement.

• 5+ years as a security engineer or CISO at a major enterprise or security firm.
• Direct ownership of incident response programs, security architecture, or compliance initiatives.
• Fluency with SIEM, EDR, vulnerability scanning, and GRC platforms.
• Experience with tools such as Splunk, Microsoft Sentinel, CrowdStrike, Tenable, Qualys, and GRC platforms.
• Strong understanding of NIST CSF, SOC 2, ISO 27001, NIS2, and the current threat landscape.
• A credential such as CISSP, CISM, or an international equivalent is strongly preferred.
• Experience designing cybersecurity scenarios and writing AI evaluation rubrics.

Helpful Background

Experience creating training data or evaluating AI models will help you contribute effectively. Familiarity with enterprise security program scoping and risk assessment is also useful.

• AI model evaluation or training data creation
• Enterprise security program scoping
• Risk assessment

How This Work Shapes AI

Every major AI system depends on carefully prepared examples and expert review. By creating cybersecurity scenarios and evaluating model reasoning against clear standards, you will help develop AI that is more dependable in high-stakes security environments.

• Create an OpenTrain profile and apply in minutes.
• Bring your professional cybersecurity expertise to specialized AI training work.
• Work remotely as part of a fast-growing field at the intersection of security and artificial intelligence.