Design Evaluation Specialist — Visual & Brand Review

About OpenTrain

OpenTrain is the leading platform for people who build careers in AI training and data labeling. We connect skilled contributors with evaluation and annotation work, help them build a unified portfolio, and hire contractors directly for projects where clear, repeatable human judgment matters.

OpenTrain AI is the hiring and contracting organization for this role.

Why AI training matters (quick background)

AI systems learn from examples selected and judged by people. Evaluators and reviewers decide what counts as a good answer or a successful design, which shapes how models behave in real-world creative tasks.

This role gives you the chance to apply professional design expertise to guide AI behavior while doing flexible, remote work that directly impacts model quality.

The role

You will evaluate real-world design deliverables—brand identity systems, campaign and marketing visuals, presentations and layouts, packaging concepts, and digital/social assets—by defining grading criteria, scoring samples, and writing clear justifications for each score.

Work is remote, contractor-based, and expected at 20+ hours per week. OpenTrain hires and pays contractors directly.

• Employment type: Contractor, part-time
• Time commitment: 20+ hours/week
• Data type: Image
• Label type: Evaluation rating
• Languages required: English
• Worldwide applicants accepted

What you'll do day to day

Create task-specific, measurable grading criteria that capture quality across brand identity, layout, typography, composition, color, and packaging.

Score both AI-generated and human samples against those criteria and provide written justifications that explain the evidence behind each score.

Calibrate judgments with senior reviewers and quickly iterate when structured feedback is provided so scores remain reproducible and defensible.

• Define evaluation rubrics and clear pass/fail or scaled criteria
• Assess and rate visual work consistently using evidence-based reasoning
• Write concise, defensible critiques for every rated sample
• Adjust and reapply standards after reviewer calibration

Requirements

You must meet the substantive experience and skills listed below. OpenTrain will evaluate candidates based on these criteria during onboarding and calibration.

Note: the job lists an experience-level field of 'Entry level' but requires substantial professional design experience—please ensure you meet the years-of-experience requirement before applying.

• 5+ years of professional experience in graphic, brand, or visual design
• Demonstrated experience judging brand identity, layout, packaging, and campaign creative
• Strong command of typography, layout, visual hierarchy, color, composition, and brand systems
• Clear written communication with the ability to explain design judgments precisely
• Comfort reviewing creative work against defined standards and calibrating with peers
• Familiarity with AI training, evaluation, or human-data projects is a plus

Who should apply

Experienced visual designers, brand designers, and art directors who can translate subjective design choices into measurable evaluation criteria. This role suits professionals who enjoy critique, teaching through feedback, and maintaining consistent standards.

Prior participation in evaluation or annotation projects is helpful but not required; strong portfolio evidence and clear written critique samples will strengthen your application.

Compensation, how to apply, and next steps

Pay is hourly, between $80 and $150 per hour (hourly pay type). The posted rate range is $80–$150/hr and OpenTrain handles contracting and payment.

To apply, prepare a brief portfolio that highlights relevant brand, packaging, layout, or campaign work and one short written critique (100–300 words) of a design sample showing how you would score it against clear criteria. Applications will be reviewed, and qualifying candidates will go through a short calibration task with senior reviewers.

• Hourly pay: $80–$150 per hour (PAY_PER_HOUR)
• Prepare portfolio + one written critique for application
• Successful candidates complete a calibration evaluation before active work