Computer Science AI Benchmark Specialist

About OpenTrain

OpenTrain AI is the hiring and contracting organization for this opportunity. OpenTrain is the #1 platform for finding and building careers in AI training and data labeling, helping contributors discover projects, build a professional profile, and apply in minutes.

Creating an OpenTrain account is free, and this role offers a chance to turn specialized technical expertise into meaningful AI training work.

• Fully remote contract work
• Part-time schedule with flexible asynchronous participation
• Opportunity to contribute to cutting-edge AI evaluation

About AI Training and Benchmark Work

AI training is the human side of building artificial intelligence. Specialists create, review, and evaluate examples that help AI systems reason more accurately, follow instructions, and perform reliably in technical domains.

In this role, your questions and evaluations will help measure advanced computer science capability. The work combines subject-matter expertise, careful judgment, and precise written communication.

• Help shape how advanced AI systems are evaluated
• Work with challenging technical text and question-answer content
• Contribute from anywhere with a computer and internet connection

The Role

OpenTrain AI is recruiting a Computer Science AI Benchmark Specialist to create and review academic assessment content used in AI research benchmarks. You will work across distributed systems, machine learning engineering, databases, computer architecture, formal methods, cloud infrastructure, operating systems, and software development.

Questions and solutions must be rigorous, unambiguous, self-contained, and suitable for evaluating advanced computer science capability. The role is listed as entry level, but it requires advanced knowledge in at least one relevant computer science subdomain.

• Contractor and part-time position
• Fully remote and asynchronous
• Pay of $66 to $84 per hour
• English-language work
• The structured schedule indicates 20+ hours per week; the role description expects at least 10 hours per week

What You'll Do

You will author original multiple-choice questions that test conceptual understanding rather than surface-level recall. Each assessment item should be precise, challenging, and independently understandable.

You will also review existing questions and explain any edits needed to improve their technical and educational quality.

• Write precise, original questions across advanced computer science topics
• Create one correct answer and nine plausible but subtly incorrect alternatives
• Rate questions as medium, hard, or expert based on academic difficulty
• Write clear, step-by-step solutions
• Provide one to five reputable academic references per question
• Review accuracy, clarity, completeness, precision, and solvability
• Explain edits made during the review process

Requirements and Preferred Background

You should have advanced knowledge of computer science theory, algorithms, systems design, machine learning, or a related technical area. Strong judgment is essential for evaluating technical correctness, answer quality, rigor, and whether a problem can be solved as written.

A PhD or doctoral candidacy in computer science, electrical engineering, or a closely related field is preferred. A master's degree may be considered when paired with exceptional depth in a specific subdomain. Research publications, relevant technology industry experience, or a competitive programming background are valuable additional qualifications.

• Advanced computer science knowledge in a relevant subdomain
• Ability to formulate precise, self-contained multiple-choice questions
• Ability to construct plausible distractors
• Skill in rating academic difficulty
• Strong technical judgment about correctness, clarity, rigor, and solvability
• Written English proficiency for clear solutions and academic explanations

Who Should Apply

This opportunity may suit computer science researchers, doctoral candidates, experienced engineers, and technically rigorous problem solvers who can explain complex ideas clearly in written English. It is especially relevant to specialists with depth in systems, machine learning engineering, databases, architecture, formal methods, cloud infrastructure, operating systems, or software development.

• Computer science researchers and doctoral candidates
• Software and systems professionals with deep technical expertise
• Competitive programmers and rigorous technical problem solvers
• Experts who enjoy designing challenging assessment content

How to Apply Through OpenTrain

Create a free OpenTrain account, build your profile around your computer science expertise, and apply for the contract in minutes. Your profile can help demonstrate the specialized experience you bring to AI training and benchmark development.

• Create or update your free OpenTrain profile
• Highlight relevant education, research, publications, and technical experience
• Apply for the Computer Science AI Benchmark Specialist contract
• Complete project onboarding and begin asynchronous remote work if selected