Skip to content
OpenTrain AIOpenTrain AIOpenTrain AIDocs

Ask OpenTrain

Answers from the documentation, with sources.

What would you like to do with OpenTrain?

AI answers can be mistaken. Check the linked sources. Don’t include private account information.

Open app

DEVELOPER DOCUMENTATION

Screen applicants with a quiz

Add an application screening quiz to a job with the CLI or MCP: create a course with a quiz, link it with stage SCREENING before publishing, review and grade each applicant, rank by screening score, and backfill earlier proposals.

Use with AI
View as MarkdownOpen in ChatGPTOpen in ClaudeOpen in PerplexityAdd to CursorAdd to VS Code

An application screening quiz is an LMS course with a graded quiz that every applicant to a job completes as part of their application. You link a published course to the job with the SCREENING stage. From then on, every new proposal on the job is enrolled automatically: the AI trainer submits the proposal, then completes the course and quiz. You review each enrollment by proposal, rank proposals by screening score, and still make the hiring decision yourself.

This is different from quizzes assigned as Project To-dos, which go to AI trainers you have already hired. For the employer view of the same feature, see Application screening quiz.

Before you start

Section titled “Before you start”
  • Versions: @opentrain-ai/cli or @opentrain-ai/mcp 0.27.0 or later. Per-question applicant results (Step 7) need 0.30.0 or later. The hosted MCP endpoint at https://app.opentrain.ai/mcp exposes the same tools.
  • Permissions: lms:read and lms:write for courses, the screening link, status, and review; proposals:read to list and rank proposals; jobs:write to publish the job. See Scopes and Capabilities.
  • Discovery: call opentrain lms capabilities --json (MCP: opentrain_lms_capabilities) for the current question types, settings enums, and limits instead of relying on remembered values.
  • A job draft: create one with the Post a job flow, and stop before Step 3 (Publish) of that guide.

The full sequence:

1. Create and publish the quiz             lms assessments create / publish
2. Put it in a course and publish it       lms courses create → checkout → push → publish
3. Link the course with stage SCREENING    lms training link --stage SCREENING --max-attempts <n>
4. Publish the job                         jobs publish
5. Watch enrollment                        lms screening status
6. Review, grade, release, decide          lms review / grade / release-result / decide --proposal
7. Rank applicants                         proposals list --sort screening-score
8. Backfill proposals that predate the link   lms screening assign

Placeholders such as <job-id>, <course-id>, <assessment-id>, and <proposal-id> stand for IDs returned by earlier calls. Idempotency keys in the examples are illustrative; use your own unique key per operation.

Step 1: Create and publish the quiz

Section titled “Step 1: Create and publish the quiz”

A quiz is a Native Forms assessment. Save a definition like this as screening-quiz.json:

{
  "title": "Applicant screening quiz",
  "definition": {
    "schemaVersion": 2,
    "questions": [
      {
        "id": "q1",
        "type": "MULTIPLE_CHOICE",
        "prompt": "Which sentiment label fits a sentence that states a fact with no opinion?",
        "required": true,
        "options": [
          { "id": "a", "label": "Positive" },
          { "id": "b", "label": "Neutral" },
          { "id": "c", "label": "Negative" }
        ],
        "grading": { "points": 1, "answerKey": { "optionId": "b" } }
      },
      {
        "id": "q2",
        "type": "PARAGRAPH",
        "prompt": "Explain how you would label a sarcastic product review.",
        "required": true,
        "manualGrading": { "points": 4 }
      }
    ]
  },
  "settings": {
    "attemptPolicy": { "mode": "LIMITED", "maxAttempts": 2 },
    "passRule": { "mode": "PERCENTAGE", "thresholdPercent": 70 },
    "resultRelease": "IMMEDIATE"
  }
}

CLI

Terminal window
opentrain lms assessments create \
  --input screening-quiz.json \
  --key screening-quiz-create-01


opentrain lms assessments publish \
  --assessment <assessment-id> \
  --expected-revision <draft-revision> \
  --confirm-publish \
  --key screening-quiz-publish-01

Run opentrain lms assessments create --help for every question type and grading option.

How the quiz settings behave for applicants:

  • attemptPolicy: the link’s attempt ceiling (Step 3) also applies. Applicants get the lower of the two; an UNLIMITED quiz gets the ceiling.
  • passRule: required for a pass or fail result (passed). Without it, a fully graded attempt has a percentage but no pass or fail.
  • resultRelease: IMMEDIATE shows the applicant their score once the attempt is graded. MANUAL holds the score until you release it in Step 6.
  • manualGrading questions: the score stays pending until you grade them in Step 6. Proposals awaiting grades rank below fully graded ones.

You can also build and publish the quiz in Forms in the OpenTrain app; the published form works the same way.

Step 2: Put the quiz in a course and publish it

Section titled “Step 2: Put the quiz in a course and publish it”

The course needs an ASSESSMENT lesson that references the published quiz. Lessons before the quiz, such as a project overview, are optional. A module with only the quiz looks like this:

{
  "key": "screening",
  "title": "Screening",
  "lessons": [
    {
      "key": "screening-quiz",
      "title": "Screening quiz",
      "kind": "ASSESSMENT",
      "required": true,
      "assessment": {
        "formId": "<assessment-id>",
        "passRequired": true,
        "manualReview": false
      }
    }
  ]
}

With passRequired: true, an applicant who fails can retake the quiz while attempts remain. Set manualReview: true only if you want to record the final pass or fail yourself with decide (Step 6).

CLI

Terminal window
opentrain lms courses create --title "Applicant screening" --key screening-course-create-01
opentrain lms checkout --course <course-id> --dir ./screening-course

Save the module above as ./screening-course/modules/01-screening.json and add that path to moduleFiles in ./screening-course/course.yaml. Then validate, push, and publish:

Terminal window
opentrain lms validate --dir ./screening-course --remote --key screening-course-validate-01
opentrain lms push --dir ./screening-course
opentrain lms publish --dir ./screening-course --confirm-publish --key screening-course-publish-01

If validation returns warnings, publishing requires acknowledging them. See Validate LMS authoring before publishing. An employer can also build the course under Training in the OpenTrain app; link it in Step 3 the same way.

Section titled “Step 3: Link the course to the job as its screening course”

CLI

Terminal window
opentrain lms training link \
  --job <job-id> \
  --course <course-id> \
  --stage SCREENING \
  --max-attempts 2 \
  --confirm-link \
  --key screening-link-01

Rules for a screening link:

  • The course must have a published version when you link it.
  • A job has at most one active screening link.
  • --max-attempts (MCP: screeningMaxAttempts) is an integer from 1 to 100 and defaults to 1. It applies only with the SCREENING stage.
  • Linking enrolls nobody who has already applied. It does not need the job’s Training setting and turns no job settings on.
  • Without --stage, a link that already exists keeps its stage and ceiling. The stage and ceiling are part of the idempotency receipt, so retrying the same key with different values returns 409.
  • Linking from a job’s Training tab in the OpenTrain app creates a post-hire training link, not a screening link.

Confirm the link before you publish:

Terminal window
opentrain lms screening status --job <job-id> --pretty

Make the quiz the only screening step

Section titled “Make the quiz the only screening step”

A screening quiz does not replace the AI interview. By default the AI interview is required, so applicants take both. To screen with the quiz alone, turn the interview requirement off:

CLI

Terminal window
opentrain jobs interview set --job-id <job-id> --not-required

Step 4: Publish the job

Section titled “Step 4: Publish the job”

Publish as usual with opentrain jobs publish --job-id <job-id> (MCP: opentrain_publish_job). See Post a job. Every proposal submitted from now on is enrolled in the screening course.

Step 5: Watch enrollment

Section titled “Step 5: Watch enrollment”

CLI

Terminal window
opentrain lms screening status --job <job-id> --limit 500

The response describes the link and every proposal on the job, newest first:

FieldMeaning
screening.linkThe active screening course, its attempt ceiling (maxAttempts), and its current published version. null when the job has no screening link.
screening.proposals[].enrollmentKeyproposal:<proposal-id>. Pass it as the assignment ID to the review tools in Step 6.
screening.proposals[].binding.statusOPEN (not started), IN_PROGRESS, or SUBMITTED.
screening.proposals[].binding.waitingOnWORKER (the applicant’s move), EMPLOYER (a manual grade or decision is pending), or NONE.
screening.proposals[].binding.courseVersionCurrentfalse when the applicant is on an older course version.
screening.proposals[].binding = nullA binding gap: this proposal is not enrolled.
screening.bindingGapsHow many proposals are not enrolled. Backfill them in Step 8.
screening.staleVersionCountHow many applicants are on an older course version, so their scores may not compare directly.
screening.hasMoretrue when the job has more proposals than limit (default and maximum 500).

Step 6: Review, grade, release, and decide each applicant

Section titled “Step 6: Review, grade, release, and decide each applicant”

The review commands are the same ones used for post-hire training. Address a screening enrollment with --proposal <proposal-id> instead of --assignment <id>. In MCP, pass the enrollment key proposal:<proposal-id> as assignmentId.

CLI

Terminal window
# Read submitted answers, grades, and each lesson's review target
opentrain lms review --proposal <proposal-id>


# Grade manually graded questions
opentrain lms grade --proposal <proposal-id> \
  --lesson screening-quiz --attempt <attempt-id> \
  --grades-json '[{"questionId":"q2","awardedPoints":3,"reason":"Covers tone, misses context."}]' \
  --expected-target <target-hash> \
  --key screening-grade-01


# Make a held (MANUAL release) result visible to the applicant
opentrain lms release-result --proposal <proposal-id> \
  --lesson screening-quiz --attempt <attempt-id> \
  --confirm-release --expected-target <target-hash> \
  --key screening-release-01


# Record a final verdict on a manual-review lesson
opentrain lms decide --proposal <proposal-id> \
  --lesson screening-quiz --decision PASS \
  --confirm-decision --expected-target <target-hash> \
  --key screening-decide-01

opentrain lms attempts list --proposal <proposal-id> lists every attempt; opentrain lms attempts get --proposal <proposal-id> --attempt <attempt-id> shows per-question evidence for one attempt.

Order and safety:

  • Grade first, then release, then decide. Release is blocked while manual grades are outstanding.
  • review returns a reviewTarget with a targetHash for each lesson. Pass it as --expected-target (MCP: expectedTargetHash). If the applicant resubmitted after you reviewed, the call fails with 409 and REVIEW_TARGET_STALE instead of acting on a different attempt.
  • Releasing reveals the score, per-question detail, and grading notes to the applicant.
  • Decisions are final. FAIL requires --reason (MCP: reason), which the applicant sees as feedback.
  • None of these calls hires or declines anyone. The hiring decision stays on the proposal.

Step 7: Rank applicants by screening score

Section titled “Step 7: Rank applicants by screening score”

CLI

Terminal window
opentrain proposals list --job-id <job-id> --sort screening-score --limit 100

The REST equivalent is GET /api/public/v1/jobs/{id}/proposals?sort=screening_score. How the ranking works:

  • Order: highest finalized percentage first, then fully graded attempts without a percentage, then submissions awaiting grades, then enrollments not yet submitted, then proposals with no enrollment.
  • One request returns one ranked page of up to limit proposals (default 25, maximum 100). There is no cursor with this sort.
  • The ranking covers the newest 500 proposals on the job. ranking.rankedCount is how many proposals it covered, ranking.scoredCount is how many have a finalized percentage, and ranking.complete is false when the job has more proposals than the ranking covers. hasMore is true when more ranked rows exist than limit, or when the ranking is incomplete.
  • If screening scores cannot be read, the request fails with a retryable 503 rather than returning a recency order. Retry, or use sort=recent, where each row then reports screeningRead: "FAILED".

Every proposal row, and proposals get (MCP: opentrain_get_proposal), carries a screening block (enrollmentKey, status, waitingOn, courseVersionNumber, courseVersionCurrent, gradeState, earnedPoints, possiblePoints, percentage, passed, submittedAt, released) and screeningRead (OK or FAILED). screening is null when the job has no screening course or the proposal is not enrolled. When screeningRead is FAILED, a null block means unknown, not “not enrolled”. percentage and passed appear only once every submitted quiz is fully graded.

To see which questions applicants miss, read per-question results for the applicant audience. Names are masked as first name and last initial.

CLI

Terminal window
opentrain lms question-results --job <job-id> --audience APPLICANT --course <course-id>
Section titled “Step 8: Backfill proposals submitted before the link”

If the job was published before the screening link, or an enrollment did not complete when a proposal was submitted, the proposal has no enrollment. Find them in Step 5 (binding: null, counted by bindingGaps), then enroll each one:

CLI

Terminal window
opentrain lms screening assign \
  --job <job-id> \
  --proposal <proposal-id> \
  --confirm-assign \
  --key screening-assign-<proposal-id>

The backfilled enrollment is identical to one created at submission and uses the link’s current published course version. The call is idempotent per proposal: created: false means the proposal was already enrolled, and retrying the same key replays the receipt. Typed refusals:

StatusCodeMeaning
409SCREENING_NOT_CONFIGUREDThe job has no active screening link.
409SCREENING_COURSE_UNPUBLISHEDThe linked course has no published version.
400SCREENING_BINDING_INVALIDThe ID is not a proposal on this job, or it is an invite rather than a proposal.

Change or remove the screening course

Section titled “Change or remove the screening course”
  • New course version: publish the course again. New enrollments, including backfills, use the new version. Applicants already enrolled keep their version and count toward staleVersionCount.
  • Stop screening: run opentrain lms training unlink --job <job-id> --course <course-id> --confirm-unlink --key <key> (MCP: opentrain_lms_unlink_job_course). The link is retired with its history kept. Applicants who have not finished then see that the screening course is no longer available.

REST endpoints

Section titled “REST endpoints”
MethodEndpointPurpose
POST/api/public/v1/lms/assessmentsCreate the quiz draft.
POST/api/public/v1/lms/assessments/{formId}/publishPublish the quiz.
POST/api/public/v1/lms/coursesCreate the course draft.
POST/api/public/v1/lms/courses/{courseId}/publishPublish the course.
POST/api/public/v1/lms/jobs/{jobId}/courses/linkLink the course with stage: "SCREENING" and screeningMaxAttempts.
GET/api/public/v1/lms/jobs/{jobId}/screeningRead the link and every proposal’s enrollment.
POST/api/public/v1/lms/jobs/{jobId}/screening/assignBackfill one proposal (jobOfferId, confirmAssign: true).
GET/api/public/v1/lms/assignments/{assignmentId}/reviewReview one enrollment; assignmentId is proposal:<proposal-id>.
POST/api/public/v1/lms/assignments/{assignmentId}/gradesRecord manual grades.
POST/api/public/v1/lms/assignments/{assignmentId}/release-resultRelease a held result.
POST/api/public/v1/lms/assignments/{assignmentId}/decisionRecord a final pass or fail.
GET/api/public/v1/jobs/{id}/proposals?sort=screening_scoreRank proposals by screening score.
GET/api/public/v1/lms/jobs/{jobId}/question-results?audience=APPLICANTPer-question applicant results.

Every write requires an Idempotency-Key header. See List Job Proposals for the ranking response fields.

Completion checklist

Section titled “Completion checklist”

Before you report that screening is set up:

  • the human agreed to screen applicants with a quiz, and the course has a published version;
  • lms screening status shows the link, with the attempt ceiling you intended;
  • the AI interview requirement matches what the human wants (both screens, or the quiz alone);
  • the link was created before publishing, or bindingGaps is 0 after backfilling earlier proposals;
  • you shared how to review results: the Screening quiz column in the job’s Proposals tab, or a ranked proposals list --sort screening-score.
Section titled “Related”
Application screening quiz

The employer view: what applicants see and how results appear in proposal review.

Post a job

Draft and publish the job; link the screening course before Step 3.

Evaluate candidates

Combine screening scores with interview scores, profiles, and pre-hire messages.

CLI: Project To-dos and quizzes

Assign quizzes to AI trainers you have already hired.