gencat: Generative computerized adaptive testing

Benchmarks and datasets

No benchmark or dataset names were extracted from the available abstract.

Reported metrics

No metric terms were extracted from the available abstract.

Key takeaways

• Existing computerized Adaptive Testing (CAT) frameworks are typically built on predicting the correctness of a student response to a question.
• Although effective, this approach fails to leverage textual information in questions and responses, especially for open-ended questions.
• In this work, we propose GENCAT (\textbf{GEN}erative \textbf{CAT}), a novel CAT framework that leverages Large Language Models for knowledge estimate and question selection.

Researcher actions

• Compare this paper against nearby papers in the same arXiv category before using it for protocol decisions.
• Check the full text for explicit evaluation design choices (raters, protocol, and metrics).
• Use related-paper links to find stronger protocol-specific references.

Caveats

• Generated from abstract + metadata only; no PDF parsing.
• Signals below are heuristic and may miss details reported outside the abstract.

Recommended queries

Contribution summary

• In this work, we propose GENCAT (GENerative CAT), a novel CAT framework that leverages Large Language Models for knowledge estimate and question selection.
• First, we develop a Generative Item Response Theory (GIRT) model that enables us to estimate student knowledge from their open-ended responses and predict responses to unseen questions.
• Second, we introduce three question selection algorithms that leverage the generative capabilities of the GIRT model, based on the uncertainty, linguistic diversity, and information of sampled student responses.

Why it matters for eval

• We train the model in a two-step process, first via Supervised Fine-Tuning and then via preference optimization for knowledge-response alignment.