QSTN: A Modular Framework for Robust Questionnaire Inference with Large Language Models

Research Summary

Contribution Summary

• We introduce QSTN, an open-source Python framework for systematically generating responses from questionnaire-style prompts to support in-silico surveys and annotation tasks with large language models (LLMs).
• QSTN enables robust evaluation of questionnaire presentation, prompt perturbations, and response generation methods.
• Our extensive evaluation (>40 million survey responses) shows that question structure and response generation methods have a significant impact on the alignment of generated survey responses with human answers.

Why It Matters For Eval

• QSTN enables robust evaluation of questionnaire presentation, prompt perturbations, and response generation methods.
• Our extensive evaluation (>40 million survey responses) shows that question structure and response generation methods have a significant impact on the alignment of generated survey responses with human answers.

Related Papers

• GDGB: A Benchmark for Generative Dynamic Text-Attributed Graph Learning Embedding Neighbor
• GUI-Libra: Training Native GUI Agents to Reason and Act with Action-aware Supervision and Partially Verifiable RL Embedding Neighbor
• MultiModalPFN: Extending Prior-Data Fitted Networks for Multimodal Tabular Learning Embedding Neighbor
• Beyond Static Pipelines: Learning Dynamic Workflows for Text-to-SQL Embedding Neighbor
• Reasoning Up the Instruction Ladder for Controllable Language Models Embedding Neighbor
• TestExplora: Benchmarking LLMs for Proactive Bug Discovery via Repository-Level Test Generation Embedding Neighbor
• Power and Limitations of Aggregation in Compound AI Systems Embedding Neighbor
• Provably Safe Generative Sampling with Constricting Barrier Functions Embedding Neighbor
• RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models Embedding Neighbor
• Inner Speech as Behavior Guides: Steerable Imitation of Diverse Behaviors for Human-AI coordination Tag Overlap
• Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models Tag Overlap
• EARL: Entropy-Aware RL Alignment of LLMs for Reliable RTL Code Generation Tag Overlap