AITutor-EvalKit: Exploring the Capabilities of AI Tutors

Research Summary

Contribution Summary

• We present AITutor-EvalKit, an application that uses language technology to evaluate the pedagogical quality of AI tutors, provides software for demonstration and evaluation, as well as model inspection and data visualization.
• This tool is aimed at education stakeholders as well as *ACL community at large, as it supports learning and can also be used to collect user feedback and annotation.

Why It Matters For Eval

• We present AITutor-EvalKit, an application that uses language technology to evaluate the pedagogical quality of AI tutors, provides software for demonstration and evaluation, as well as model inspection and data visualization.

Related Papers

• DAVE: A Policy-Enforcing LLM Spokesperson for Secure Multi-Document Data Sharing Embedding Neighbor
• See and Fix the Flaws: Enabling VLMs and Diffusion Models to Comprehend Visual Artifacts via Agentic Data Synthesis Embedding Neighbor
• propella-1: Multi-Property Document Annotation for LLM Data Curation at Scale Embedding Neighbor
• Can Large Language Models Replace Human Coders? Introducing ContentBench Embedding Neighbor
• ATLAS: Adaptive Transfer Scaling Laws for Multilingual Pretraining, Finetuning, and Decoding the Curse of Multilinguality Embedding Neighbor
• On the Existence and Behavior of Secondary Attention Sinks Embedding Neighbor
• EvalSense: A Framework for Domain-Specific LLM (Meta-)Evaluation Embedding Neighbor
• gencat: Generative computerized adaptive testing Embedding Neighbor
• DocSplit: A Comprehensive Benchmark Dataset and Evaluation Approach for Document Packet Recognition and Splitting Embedding Neighbor
• Perspectives - Interactive Document Clustering in the Discourse Analysis Tool Suite Tag Overlap
• Orchestration-Free Customer Service Automation: A Privacy-Preserving and Flowchart-Guided Framework Tag Overlap
• FewMMBench: A Benchmark for Multimodal Few-Shot Learning Tag Overlap