Application Security AI Evaluation Expert

You will create realistic security fixtures and evaluation assets that help train and assess AI systems. The work combines application security judgment with coding, testing, technical documentation, peer feedback, and improvements to evaluation pipelines.

• Design and implement small application fixtures with embedded vulnerabilities or secure solutions.
• Create scenarios involving sensitive information disclosure and excessive agency.
• Develop deterministic tests and security-context manifests to assess application security consistently.
• Build hidden answer keys for evaluating vulnerable and secure code.
• Improve synthetic security tasks so they remain realistic and challenging.
• Document technical findings, methods, insights, and practical recommendations.
• Review peer contributions and support agent-based security architecture and evaluation-pipeline improvements.

What It Pays and Takes

This is a remote contractor role with flexible scheduling. The listing marks the experience level as entry level, while the requirements call for substantial application security or secure software development experience.

• Pay: $40 to $80 per hour.
• Work style: Remote contractor role with flexible scheduling.
• Language: Fluent English.
• Location: Open to candidates in the United Arab Emirates, Argentina, Austria, Bangladesh, Belgium, Bahrain, Brazil, Canada, Switzerland, Chile, Colombia, Germany, Denmark, Algeria, Egypt, Spain, France, the United Kingdom, Ghana, Greece, Indonesia, India, Italy, Jordan, Japan, Kenya, Kyrgyzstan, Kuwait, Kazakhstan, Lebanon, Sri Lanka, Morocco, and Mexico.
• Location: Also open to candidates in Malaysia, Nigeria, the Netherlands, Oman, Peru, the Philippines, Pakistan, Palestine, Portugal, Qatar, Saudi Arabia, Singapore, Thailand, Tunisia, Taiwan, the United States, Uzbekistan, Vietnam, and South Africa.
• Experience: Extensive application security, secure software development, or comparable CyberSWE experience, ideally at a five-year or greater level.
• Knowledge: Strong knowledge of OSWE, LLM security, AI-powered static analysis, and application security best practices.
• Skills: Ability to design or evaluate security architecture for traditional and agent-based applications, build and test codebases, identify and fix advanced vulnerabilities, and communicate precise technical judgments.
• Useful background: Familiarity with synthetic security fixtures and deterministic testing approaches.
• Prior AI training experience is not required.

How It Works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI Training Work

AI training is the human work behind systems that learn from examples, including code reviews, evaluations, and feedback on model outputs. People with strong technical expertise help make those examples accurate, consistent, and useful.