Mmlu Benchmark Papers

Research Utility Snapshot

Top Papers On This Benchmark

• Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference

  Bo-Wei Chen, Chung-Chi Chen, An-Zi Yen · Feb 25, 2026

  Automatic Metrics Tool Use

  Experiments on the Massive Multitask Language Understanding (MMLU) benchmark show that our approach achieves accuracy comparable to the largest model while reducing computational costs by 20\% to 40\%.

• D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models

  Shunsuke Ubukata · Feb 25, 2026

  Automatic Metrics Long Horizon

  Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) often induces "overthinking" in Small Language Models (SLMs), leading to performance degradation and excessive token consumption.

• Black-Box Reliability Certification for AI Agents via Self-Consistency Sampling and Conformal Calibration

  Charafeddine Mouzouni · Feb 24, 2026

  Automatic Metrics

  We validate across five benchmarks, five models from three families, and both synthetic and real data.

• ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decomposition

  Xindian Ma, Rundong Kong, Peng Zhang, Ruoxiang Huang, Yongyu Jiang · Feb 24, 2026

  Automatic Metrics

  We evaluate ID-LoRA on five diverse benchmarks: Mathematical Reasoning, Code Generation, MMLU, CommonsenseQA, and Safety Alignment.

• KNIGHT: Knowledge Graph-Driven Multiple-Choice Question Generation with Adaptive Hardness Calibration

  Mohammad Amanlou, Erfan Shafiee Moghaddam, Yasaman Amou Jafari, Mahdi Noori, Farhan Farsi · Feb 23, 2026

  Automatic Metrics

  Results show that KNIGHT enables token- and cost-efficient generation from a reusable graph representation, achieves high quality across these criteria, and yields model rankings aligned with MMLU-style benchmarks, while supporting topic-sp

• Same Meaning, Different Scores: Lexical and Syntactic Sensitivity in LLM Evaluation

  Bogdan Kostić, Conor Fallon, Julian Risch, Alexander Löser · Feb 19, 2026

  Automatic Metrics

  The rapid advancement of Large Language Models (LLMs) has established standardized evaluation benchmarks as the primary instrument for model comparison.

• RoPE-LIME: RoPE-Space Locality + Sparse-K Sampling for Efficient LLM Attribution

  Isaac Picov, Ritesh Goru · Feb 6, 2026

  Automatic Metrics Tool Use

  Explaining closed-source Large Language Model (LLM) outputs is challenging because API access prevents gradient-based attribution, while perturbation methods are costly and noisy when they depend on regenerated text.

• Indic-TunedLens: Interpreting Multilingual Models in Indian Languages

  Mihir Panchal, Deeksha Varshney, Mamta, Asif Ekbal · Jan 29, 2026

  Automatic Metrics

  We evaluate our framework on 10 Indian languages using the MMLU benchmark and find that it significantly improves over SOTA interpretability methods, especially for morphologically rich, low resource languages.

• Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at Scale

  David Acuna, Chao-Han Huck Yang, Yuntian Deng, Jaehun Jung, Ximing Lu · Nov 7, 2025

  Automatic Metrics

  We introduce a framework able to synthesize vision-centric problems spanning diverse levels of complexity, and the resulting dataset with over 1M high-quality problems including: reasoning traces, preference data, and instruction prompts su

• Annotation-Efficient Universal Honesty Alignment

  Shiyu Ni, Keping Bi, Jiafeng Guo, Minghao Tang, Jingtong Wu · Oct 20, 2025

  Automatic Metrics

  To support a large-scale study, we release HonestyBench, a benchmark covering ten free-form QA datasets with 560k training and 70k evaluation instances annotated with correctness and self-consistency signals.

• Evolving Language Models without Labels: Majority Drives Selection, Novelty Promotes Variation

  Yujun Zhou, Zhenwen Liang, Haolin Liu, Wenhao Yu, Kishan Panaganti · Sep 18, 2025

  Automatic Metrics

  Large language models (LLMs) are increasingly trained with reinforcement learning from verifiable rewards (RLVR), yet real-world deployment demands models that can self-improve without labels or external judges.

• Diffusion Language Models Know the Answer Before Decoding

  Pengxiang Li, Yefan Zhou, Dilxat Muhtar, Lu Yin, Shilin Yan · Aug 27, 2025

  Automatic Metrics

  Empirical evaluations of LLaDA-8B and Dream-7B across multiple tasks show that Prophet reduces the number of decoding steps by up to 3.4x while preserving high generation quality.

• Tokens with Meaning: A Hybrid Tokenization Approach for Turkish

  M. Ali Bayram, Ali Arda Fincan, Ahmet Semih Gümüş, Sercan Karakaş, Banu Diri · Aug 19, 2025

  Automatic Metrics

  We further validate practical utility with downstream sentence embedding benchmarks under a strict \emph{random initialization} control to isolate tokenizer inductive bias.

• Enhancing Multilingual LLM Pretraining with Model-Based Data Selection

  Bettina Messmer, Vinko Sabolčec, Martin Jaggi · Feb 14, 2025

  Automatic Metrics

  Training a 1B-parameter Llama model for 70B and 119B tokens, our approach can match the baseline MMLU score with as little as 15% of the training tokens, while also improving across other benchmarks and mitigating the curse of multilinguali

• Humanity's Last Exam

  Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu · Jan 24, 2025

  Automatic Metrics

  Benchmarks are important tools for tracking the rapid advancements in large language model (LLM) capabilities.

Other Benchmark Hubs