Social Science AI Evaluation Researcher

About OpenTrain

OpenTrain is the #1 platform for finding and building careers in AI training and data labeling. We help people discover specialized projects, build a professional profile, and apply to opportunities that can grow into a lasting AI training portfolio.

OpenTrain AI is recruiting this freelance contractor role for remote work with flexible scheduling.

About AI Evaluation Work

AI training is the human side of building artificial intelligence. Researchers and other specialists create examples, evaluate model responses, and develop structured feedback that helps advanced systems reason more accurately and handle complex real-world work.

In this role, your social science expertise will help assess whether AI systems can interpret evidence, apply research methods, and produce defensible methodological work.

The Role

As a Social Science AI Evaluation Researcher, you will support the development of evaluation tasks for frontier AI capability benchmarking. The work focuses on realistic, advanced research scenarios rather than simplified textbook exercises.

You will create research artifacts and evaluation frameworks designed for rigorous model evaluation and expert review. Assignments are completed asynchronously in collaboration with project leads and reviewers.

• Role type: Part-time freelance contractor
• Workload: 20+ hours per week
• Work location: Worldwide and remote
• Working language: English
• Experience level: Entry level listing with at least two years of relevant hands-on experience required

What You'll Do

You will design and author complex evaluation tasks that reflect the day-to-day complexity of professional social science research. Your work should demonstrate rigor, transparency, methodological judgment, and reproducibility.

• Design tasks involving survey analysis, qualitative coding, literature reviews, and statistical analysis.
• Source, synthesize, and create survey instruments, coded datasets, statistical outputs, and source documents.
• Select appropriate methodological approaches and write clear interpretations grounded in social science best practices.
• Create comprehensive grading rubrics with 35 or more items covering methodological choices, execution fidelity, and interpretation quality.
• Refine tasks, research artifacts, and evaluation frameworks with project leads and reviewers through asynchronous collaboration.
• Maintain accurate documentation and high standards of detail, research quality, transparency, and reproducibility.

Requirements

A bachelor's or master's degree in sociology, economics, psychology, political science, or a related social science discipline is preferred. You should have practical experience producing research content and applying social science methods.

Prior AI experience is not required. The role is based on demonstrated social science knowledge, research practice, and the ability to create reliable evaluation materials.

• At least two years of hands-on experience with survey instrument design, qualitative data coding, literature reviews, and statistical analysis.
• Working knowledge of statistical software such as SPSS, R, or Stata.
• Strong research-source synthesis and data documentation skills.
• Ability to produce accurate, reproducible research content and methodological interpretations.
• Experience developing or applying comprehensive grading rubrics or evaluation guidelines is valuable.
• Proficient written English for technical research and evaluation materials.

Compensation and Workload

Listed compensation is $30-$50 USD per hour. Compensation is output-based and paid for completed tasks that meet project specifications, and minimum submission requirements apply.

Task completion time may vary depending on the assignment and your workflow. The role is part time with an expected commitment of 20 or more hours per week.

Why Build an AI Training Career

AI training and data labeling are among the fastest-growing ways to work in tech. Specialists contribute directly to how modern AI systems interpret information, generate responses, and perform professional tasks.

OpenTrain provides a place to build a profile around your work, discover projects aligned with your expertise, and develop a credible portfolio in this rapidly evolving field.

• Work remotely from anywhere with an internet connection.
• Use specialized social science expertise in cutting-edge AI evaluation.
• Choose flexible project work that can fit around other commitments.
• Build experience and a professional portfolio in AI training.