Ground a Quiz in Instructions, Validate, and Restore

A grounded quiz is a quiz whose questions cite the job’s Instructions. The model in your agent writes the questions. OpenTrain supplies the released Instructions, a snapshot of page checksums, validation, a publish fence, and the immutable published version. OpenTrain does not generate questions.

The publish fence protects AI trainers from a quiz that tests rules the Instructions no longer contain: if a page that a question cites has changed since you pinned it, publishing stops and names the questions to revise.

Note

Requires CLI and MCP 0.42.0 or later and SDK 0.39.0 or later. Grounding reads need both project_todos:read and instructions:read.

Workflow

Read the released Instructions

Page reads return the draft by default. Write grounded questions from the released page, the version AI trainers see: GET /api/public/v1/instructions/jobs/{jobId}/pages/{pageId}?released=true, or getJobInstructionPage(jobId, pageId, { released: true }) in the SDK. Each block carries a stable id in attrs.id. Questions cite those ids.

Take the grounding snapshot

Read the snapshot and copy its grounding block, unchanged, into definition.grounding. The definition must use schemaVersion: 2.

Cite blocks from each question

Give each grounded question up to 20 sourceRefs ({ "pageId", "nodeId" }, where the page is pinned in the grounding) and an optional rationale of up to 2,000 characters. AI trainers never see either.

Validate

Run form_validate on the candidate or the stored draft. Fix every finding it lists.

Publish or roll out

form_publish or quiz_rollout_confirm checks the cited pages again before it writes.

Grounding snapshot

GET /api/public/v1/project-todos/quizzes/grounding?jobId=… returns the job’s released effective Instructions: the effective instruction set id, a page-tree checksum, and each page’s pageId, title, parentPageId, checksum, and stableNodeIdCount. A set inherited from a folder is included and marked inheritedFromFolder: true. Google Doc pages are read live from the linked document.

RESTSDKCLIMCP

REST

curl -s "https://www.opentrain.ai/api/public/v1/project-todos/quizzes/grounding?jobId=job_123" \
  -H "Authorization: Bearer $OPENTRAIN_API_TOKEN" \
  -H "X-OpenTrain-Features: quiz-grounding-draft-differs-v1"

SDK

const snapshot = await opentrain.getProjectTodoQuizGrounding('job_123');
// Copy snapshot.grounding into definition.grounding.

CLI

opentrain todos quiz grounding --job-id job_123

MCP

Call opentrain_quiz_grounding_snapshot with jobId.

The grounding block you copy has this shape:

{
  "jobId": "job_123",
  "instructionSetId": "set_789",
  "treeChecksum": "…",
  "pages": [{ "pageId": "page_ppe", "checksum": "…" }],
  "capturedAt": "2026-11-01T10:00:00.000Z"
}

Pages whose draft differs

With the X-OpenTrain-Features: quiz-grounding-draft-differs-v1 header, each page of a released snapshot also carries draftDiffers. It is true when the page’s unreleased draft differs from the released text, no longer has the page, or could not be read. For those pages, write questions from the released read, not from the draft. Google Doc pages always report false. The SDK, CLI, and MCP send the header for you.

No released Instructions

A job with no released Instructions answers 409 CONFLICT with details.reason INSTRUCTIONS_NOT_RELEASED. Release the Instructions first, or use a draft preview.

Draft preview

Use a draft preview only when the person asks to ground on edits that are not released yet. Pass draftPreview=true (--draft-preview in the CLI, { draftPreview: true } in the SDK and MCP). The result carries "source": "draft_preview" and a previewLabel. Show that label to the person.

A draft-preview grounding never publishes. form_validate reports GROUNDING_DRAFT_PREVIEW, the rollout plan lists it as a blocker, and form_publish refuses it with 409 GROUNDING_DRAFT_PREVIEW. To publish, release the Instructions, take the snapshot again without the option, copy it into definition.grounding, and publish. OpenTrain never promotes a preview pin for you. A stored draft that holds a preview grounding shows its previewLabel on the form reads, so a later session sees it before it tries to publish.

Validate without writing

form_validate reports every schema finding, publication-rule finding, form-state gate, and grounding finding at once. It needs only project_todos:read and no idempotency key, and it writes no content. Pass formId to check the stored draft, or jobId plus definition and settings to check a candidate before you create it, never both. A grounded definition also needs instructions:read.

RESTSDKCLIMCP

REST

curl -s -X POST "https://www.opentrain.ai/api/public/v1/project-todos/quizzes/manage" \
  -H "Authorization: Bearer $OPENTRAIN_API_TOKEN" -H 'content-type: application/json' \
  -H "X-OpenTrain-Features: quiz-grounding-draft-differs-v1" -d '{
  "action": "form_validate",
  "formId": "form_456"
}'

SDK

const report = await opentrain.validateProjectTodoQuiz({ formId: 'form_456' });
if (!report.publishable) console.log(report.affectedQuestionIds, report.findings);

CLI

# Stored draft
opentrain todos quiz validate --form-id form_456
# Candidate, before it exists
opentrain todos quiz validate --job-id job_123 \
  --definition-file quiz.json --settings-file settings.json

A report with publishable: false exits 0. It is a successful report, not a failure.

MCP

Call opentrain_validate_quiz_draft with formId, or with jobId, definition, and settings.

The response has publishable, findings[], publication (total points and question counts, including groundedQuestionCount), grounding (status is NONE, VERIFIED, or SOURCE_CHANGED, plus its findings and the live page checksums), and affectedQuestionIds. Each finding has a source:

source	Codes
SCHEMA	INVALID_DEFINITION, INVALID_SETTINGS, with a path.
PUBLICATION	Publication rules (for example, a pass rule needs at least one answerable scored question) and form-state gates such as FORM_ARCHIVED, ASSET_UNUSABLE, LEARNING_PLAN_MISMATCH, and GROUNDING_DRAFT_PREVIEW.
GROUNDING	code: "SOURCE_CHANGED" with a severity, a reason, the pageId and nodeId, and the questionIds that cite it.

Grounding reason values:

Reason	Severity	Meaning
PAGE_CHANGED, PAGE_MISSING, NODE_MISSING	Error	A cited page changed or is gone, or a cited block no longer exists.
JOB_MISMATCH, NO_INSTRUCTIONS, INSTRUCTION_SET_CHANGED	Error	The grounding belongs to another job, the job has no Instructions now, or the effective set changed.
PAGE_UNREADABLE	Error	A cited page could not be read now. Try again.
TREE_CHANGED	Warning	Pages were added, moved, or removed elsewhere in the tree. Does not block.
DRAFT_DIFFERS, DRAFT_UNREADABLE	Warning	A cited page has unreleased draft edits, or its draft could not be read to compare. Check those questions against the released text. Does not block. Sent only with the feature header.

Only pages that some question cites are read again. A pinned page that no question cites never blocks publishing.

The publish fence

form_publish and quiz_rollout_confirm read the cited pages again. Publishing a grounded draft also needs instructions:read. An ungrounded publish needs only project_todos:write.

Response	What to do
409 CONFLICT, details.reason SOURCE_CHANGED, with details.affectedQuestionIds and details.findings	Read those pages again, revise those questions, pin the pages at the checksum you just read, and save the draft with acknowledgeSourceChanges: { "questionIds": [...] } naming the revised questions. Then publish again.
409, details.reason SOURCE_CHANGED_UNACKNOWLEDGED, with details.movedPins	You pinned a cited page at a new checksum without naming the questions that cite it. Add them to acknowledgeSourceChanges.
409, details.reason GROUNDING_DRAFT_PREVIEW	See Draft preview.
503, details.reason PAGE_UNREADABLE	The page could not be read right now. Retry.

opentrain todos quiz update --form-id form_456 --expected-revision 1 --title "Onboarding quiz" \
  --definition-file quiz.json --settings-file settings.json \
  --acknowledge-source-changes q1,q4 --confirm-live

The published version stores the grounding it was checked against, pruned to the pages questions cite. AI trainers never see grounding, sourceRefs, or rationale.

Restore an archived quiz

form_restore brings an archived form back. Restoring a form that is not archived does nothing. Published versions are never changed. It needs project_todos:write, a claimed account, and an idempotencyKey.

RESTSDKCLIMCP

REST

curl -s -X POST "https://www.opentrain.ai/api/public/v1/project-todos/quizzes/manage" \
  -H "Authorization: Bearer $OPENTRAIN_API_TOKEN" -H 'content-type: application/json' -d '{
  "action": "form_restore",
  "idempotencyKey": "restore-safety-quiz-1",
  "formId": "form_456"
}'

SDK

await opentrain.manageProjectTodoQuiz({
  action: 'form_restore',
  idempotencyKey: 'restore-safety-quiz-1',
  formId: 'form_456',
});

CLI

opentrain todos quiz restore --form-id form_456 --confirm-live

MCP

Call opentrain_restore_quiz_form with formId and idempotencyKey.

Related

• Quiz-as-code and templates explains how grounding travels with an exported document.
• Roll out a quiz shows where SOURCE_CHANGED and GROUNDING_DRAFT_PREVIEW appear as blockers.