English-Speaking Story Evaluator and Reviewer

This project has two roles. Labelers rate short story and comedy segments, while Reviewers check labeler work for accuracy and consistency. The work is text-only in a web-based labeling tool, with no audio or video.

You will read fiction that may sometimes include mature or offensive content. You must be comfortable making independent judgments from detailed written guidelines.

• Labeler: Rate each short story or comedy segment from 1 to 9 for suspense or humor.
• Labeler: Compare two possible continuations with a human-written reference, choose which better matches its suspense or humor and which follows more logically, then give a 1 to 5 confidence rating for each choice.
• Labeler: Rate complete stories from 1 to 5 for relevance, coherence, empathy, surprise, engagement, and complexity.
• Reviewer (QC): Check labelers' work, approve accurate work, and return work that does not follow the guidelines with clear, specific feedback.
• Reviewer (QC): Identify patterns of confusion or inconsistency and report them so the guidelines and training can improve.

What It Pays and Takes

This is fully remote, part-time contractor work with flexible hours. The project has a short initial phase of under one month, with more work possible.

• Labelers earn $17.00 per hour.
• Reviewers (QC) earn $19.00 per hour.
• About 50 hours of work are available per person during this phase.
• Work is less than 20 hours per week, with an immediate start.
• You must be a fluent English speaker based in the United States, United Kingdom, Canada, Australia, New Zealand, or Ireland.
• You need strong reading comprehension and a good feel for storytelling, suspense, and humor.
• You must follow detailed written guidelines consistently and make independent judgments.
• You need a reliable computer and internet connection.
• Experience with text evaluation, rating, or data annotation is a plus.
• Reviewer (QC) applicants must have prior reviewer, QA, or QC experience on data annotation, rating, or evaluation projects.
• In your application, say whether you are applying as a Labeler, Reviewer (QC), or either.

How It Works

Apply on OpenTrain. The employer reviews applications there.

About AI Training Work

AI training is the human work behind systems that learn from examples. People read, rate, and review content so AI can better understand language, follow instructions, and produce useful responses; experience with careful evaluation is valuable because consistent judgments help improve the training data.