HumanEval+ or MATH-500 Benchmark Papers

Researcher Quick Triage

Use this page for benchmark-matched method comparisons and eval protocol selection. Quality band: High .

• 40 papers explicitly name benchmark datasets in the sampled set.
• 27 papers report at least one metric term in metadata extraction.
• Start with the ranked shortlist below before reading all papers.

Primary action: Start with the top 2 benchmark-matched papers, then compare evaluation modes in the protocol matrix.

Start Here (Benchmark-Matched First 6)

Ranked by protocol completeness so you can quickly find papers suitable for comparison studies.

• Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

  Jun 24, 2026 · Citations: 0 · Score: 7.5

  Eval: Automatic Metrics · Metrics: Accuracy

• What RL Brings to Reasoning: A Few Decisions the Base Model Would Not Make

  Oct 17, 2026 · Citations: 0 · Score: 7.0

  Eval: Automatic Metrics · Metrics: Pass@1

• ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks

  Sep 24, 2026 · Citations: 0 · Score: 7.0

  Eval: Automatic Metrics · Metrics: Pass@1

• Base Models Can Reason By Taking a Cue From Training Data

  Oct 5, 2026 · Citations: 0 · Score: 7.0

  Eval: Automatic Metrics · Metrics: Accuracy

• Reasoning with Continuous Latent Diffusion

  Sep 28, 2026 · Citations: 0 · Score: 7.0

  Eval: Automatic Metrics · Metrics: Pass@1

• Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning

  Apr 1, 2026 · Citations: 0 · Score: 7.0

  Eval: Automatic Metrics · Metrics: Error rate

Protocol Matrix (Top 10)

Compare protocol ingredients quickly before deep-reading full papers.

Top Papers On This Benchmark

• What RL Brings to Reasoning: A Few Decisions the Base Model Would Not Make

  Siyu Zhang, Yuquan Lu, Yoshua Y. Li, Yifan Ding, Kun Zeng · Oct 17, 2026 · Citations: 0

  Automatic Metrics

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Evaluating the Robustness of Proof Autoformalization in Lean 4

  Zhengtao Gui, Sheng Yang, Zhouxing Shi · Jun 12, 2026 · Citations: 0

  We build a benchmark with both perturbations on miniF2F and MATH-500, and automatically measure how stable a proof autoformalization's correctness is under global perturbations and how faithfully its output reflects local perturbations.

• BRIDGE: Bridging Reasoning In Distillation Gap Elimination via Structure-Aware Masking

  Bowen Yu, Sheng Zhang, Binhao Wang, Yi Wen, Jingtong Gao · Feb 5, 2026 · Citations: 0

  Automatic Metrics

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• ELF-REG: Scaling Continuous Diffusion Language Models to Reasoning Tasks

  Zeyu Michael Li, William Xingxu Chen, Bingshuo Qian, Jiayin Liu, Xiang Cheng · Sep 24, 2026 · Citations: 0

  Automatic Metrics

  ELF-REG-L achieves 55.96% pass@1 on GSM8K at 64 network function evaluations (NFE), and 13.39% on MATH-500 and 22.56% on HumanEval+ at 128 NFE.

• Base Models Can Reason By Taking a Cue From Training Data

  Sophie L. Wang, Amil Dravid, Rulin Shao, Kevin Farhat, Sewon Min · Oct 5, 2026 · Citations: 0

  Automatic Metrics

  Finally, we extend our study of token cues with a case study in language model safety, finding that different cues elicit distinct refusal and compliance behaviors that correspond to different types of training data.

• Reasoning with Continuous Latent Diffusion

  Xiang Cheng · Sep 28, 2026 · Citations: 0

  Automatic Metrics

  Our supervised models outperform reported results from recent continuous-diffusion baselines at comparable backbone scales on mathematical reasoning and HumanEval+ code generation.

• LOCKS: Page-Local Compact Key Summaries for Efficient Long-Context Decoding

  Junsung Hwang · Jul 27, 2026 · Citations: 0

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Tool Verification for Test-Time Reinforcement Learning

  Ruotong Liao, Nikolai Röhrich, Xiaohan Wang, Yuhui Zhang, Yasaman Samadzadeh · Mar 2, 2026 · Citations: 0

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Moxia: A Trust-First Neuro-Symbolic Execution Architecture for Self-Explaining Mathematical Reasoning

  Alessio Bruno · May 30, 2026 · Citations: 0

  Automatic Metrics

  The rule-only path answers the 20,000-record lm-eval arithmetic benchmark at 100%, 1 ms per record.

• Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens

  Zhenyu Zhao, Sander Land, Daniel M. Bikel, Waseem Alshikh · Apr 29, 2026 · Citations: 0

  Automatic Metrics

  Across three model families and five mathematical reasoning benchmarks, our approach shortens reasoning traces by 8.1% on average; under a TOST equivalence analysis at a +/- 2pp margin, accuracy is equivalent or inconclusive on 13/15 model…

• Learning How to Use Tools, Not Just When: Pattern-Aware Tool-Integrated Reasoning

  Ningning Xu, Yuxuan Jiang, Shubhashis Roy Dipta, Hengyuan Zhang · Sep 27, 2025 · Citations: 0

  Pairwise Preference Automatic Metrics

  We propose a two-stage framework that first builds code competence from both patterns and then aligns pattern selection with teacher preferences.

• JetSpec: Breaking the Scaling Ceiling of Speculative Decoding with Parallel Tree Drafting

  Lanxiang Hu, Zhaoxiang Feng, Yulun Wu, Haoran Yuan, Yujie Zhao · Jun 16, 2026 · Citations: 0

  Automatic Metrics

  Across math, coding, and chat benchmarks on dense and MoE Qwen3 models, JetSpec consistently outperforms bidirectional-head and tree-based SD baselines.

• Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

  Jaeyong Ko, Pilsung Kang, Yukyung Lee · Jun 24, 2026 · Citations: 0

  Pairwise Preference Automatic Metrics

  Across seven models and three mathematical reasoning benchmarks (GSM1K, MATH500, AIME 2025), cliff tokens act as failure triggers; deleting the first cliff token and resampling recovers pass@64 to 1.0, while keeping it limits recovery to…

• ConPress: Learning Efficient Reasoning from Multi-Question Contextual Pressure

  Jie Deng, Shining Liang, Jun Li, Hongzhi Li, Yutao Xie · Feb 1, 2026 · Citations: 0

  Automatic Metrics

  This phenomenon arises from multi-question contextual pressure during generation and consistently manifests across models and benchmarks.

• Blockwise Policy-Drift Gating for On-Policy Distillation

  Liwen Zheng, Haiyun Jiang · Jun 23, 2026 · Citations: 0

  Automatic Metrics

  In a six-variant Qwen3 math reasoning benchmark with a uniform 200-step training budget for all trained variants, we use pass@8 as the primary problem-level solve-rate metric.

• SIGMA: Search-Augmented On-Demand Knowledge Integration for Agentic Mathematical Reasoning

  Ali Asgarov, Umid Suleymanov, Aadyant Khatri · Oct 31, 2025 · Citations: 0

  Automatic Metrics

  We introduce SIGMA (Search-Augmented On-Demand Knowledge Integration for AGentic Mathematical reAsoning), a unified framework that orchestrates specialized agents to independently reason, perform targeted searches, and synthesize findings…

• Reasoning with Sampling: Cutting at Decision Points

  Felix Zhou, Anay Mehrotra, Quanquan C. Liu · May 28, 2026 · Citations: 0

  Across MATH500, HumanEval+, GPQA Diamond, and AIME26, our method consistently improves over baselines and RL-trained models.

• Modeling Hierarchical Thinking in Large Reasoning Models

  G M Shahariar, Erfan Shayegani, Ali Nazari, Nael Abu-Ghazaleh · Oct 25, 2025 · Citations: 0

  Experiments across four benchmarks (AIME25, MATH-500, GSM8k, and GPQA Diamond) using three state-of-the-art open reasoning models demonstrate that Q-Value steering policy achieves significant performance gains with "surgical" efficiency,…

• MUR: Momentum Uncertainty guided Reasoning for Large Language Models

  Hang Yan, Fangzhi Xu, Rongman Xu, Yifei Li, Jian Zhang · Jul 20, 2025 · Citations: 0

  Automatic Metrics

  MUR is comprehensively evaluated against various TTS methods across four challenging benchmarks (MATH-500, AIME24, AIME25, and GPQA-diamond) using different sizes of recent Qwen3 models (1.7B, 4B, and 8B).

• Toward Automated Robustness Evaluation of Mathematical Reasoning

  Yutao Hou, Zeguan Xiao, Fei Yu, Yihan Jiang, Ma Shuguang · Jun 5, 2025 · Citations: 0

  Existing robustness evaluations predominantly rely on hand-crafted templates or a limited set of perturbation rules.

• DeepPrune: Parallel Scaling without Inter-trace Redundancy

  Shangqing Tu, Yaxuan Li, Yushi Bai, Lei Hou, Juanzi Li · Oct 9, 2025 · Citations: 0

  Llm As JudgeAutomatic Metrics

  Our method features a specialized judge model trained with out-of-distribution data (AIME 2022, AIME 2023, and MATH-500) using oversampling techniques to accurately predict answer equivalence from partial reasoning traces, achieving 0.7072…

• Towards Hierarchical Multi-Step Reward Models for Enhanced Reasoning in Large Language Models

  Teng Wang, Zhangyi Jiang, Zhenqi He, Shenyang Tong, Wenhan Yang · Mar 16, 2025 · Citations: 0

  Automatic Metrics

  Empirical results on the PRM800K dataset show that HRM, together with HNC, provides more stable and reliable evaluations than PRM.

• Reliable Control-Point Selection for Steering Reasoning in Large Language Models

  Haomin Zhuang, Hojun Yoo, Xiaonan Luo, Kehan Guo, Xiangliang Zhang · Apr 2, 2026 · Citations: 0

  Automatic Metrics

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning

  Cai Zhou, Zekai Wang, Menghua Wu, Qianyu Julie Zhu, Flora C. Shi · Apr 1, 2026 · Citations: 0

  Automatic Metrics

  Under zero-shot out-of-domain settings, it improves MATH-500 savings from 24.8% of the static calibration baseline to 67.0% while maintaining a low empirical error rate, and the same trend holds across model families and downstream…

• S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models

  Jack Young · Apr 1, 2026 · Citations: 0

  Automatic Metrics

  Using roughly 48 execution-verified HumanEval+ training solutions, tuning a single initial state matrix per recurrent layer, with zero inference overhead, outperforms LoRA by +10.8 pp (p < 0.001) on HumanEval+.

• Locally Confident, Globally Stuck: The Quality-Exploration Dilemma in Diffusion Language Models

  Liancheng Fang, Aiwei Liu, Henry Peng Zou, Yankai Chen, Enze Ma · Apr 1, 2026 · Citations: 0

  Experiments across a range of reasoning benchmarks including MATH500, AIME24/25, HumanEval+, and MBPP show that our approach yields better exploration-quality tradeoff than both random and low-confidence remasking.

• TAPS: Task Aware Proposal Distributions for Speculative Sampling

  Mohamad Zbib, Mohamad Bazzi, Ammar Mohanna, Hasan Abed Al Kader Hammoud, Bernard Ghanem · Mar 27, 2026 · Citations: 0

  Measured by acceptance length, task-specific training yields clear specialization: MathInstruct-trained drafts are strongest on reasoning benchmarks, while ShareGPT-trained drafts are strongest on MT-Bench.

• When to Ensemble: Identifying Token-Level Points for Stable and Fast LLM Ensembling

  Heecheol Yun, Kwangmin Ki, Junghyun Lee, Eunho Yang · Oct 17, 2025 · Citations: 0

  Automatic Metrics

  Our experiments on diverse benchmarks, including MATH500 and BBH, demonstrate that SAFE outperforms existing methods in both accuracy and efficiency, with gains achieved even when ensembling fewer than 1% of tokens.

• KV Cache Transform Coding for Compact Storage in LLM Inference

  Konrad Staniszewski, Adrian Łańcucki · Nov 3, 2025 · Citations: 0

  Automatic Metrics

  We test KVTC with Llama 3, Mistral NeMo, and R1-Qwen 2.5 models across benchmarks including AIME25, GSM8K, LiveCodeBench, LongBench, MATH-500, MMLU, Qasper and RULER.

• DistillGuard: Evaluating Defenses Against LLM Knowledge Distillation

  Bo Jiang · Mar 8, 2026 · Citations: 0

  We introduce a taxonomy of three defense categories -- output perturbation, data poisoning, and information throttling -- and evaluate nine defense configurations using a standardized pipeline with Qwen3-14B as teacher and…

• MathSmith: Towards Extremely Hard Mathematical Reasoning by Forging Synthetic Problems with a Reinforced Policy

  Shaoxiong Zhan, Yanlin Lai, Ziyu Lu, Dahua Lin, Ziqing Yang · Aug 7, 2025 · Citations: 0

  Existing synthesis methods largely rely on transforming human-written templates, limiting both diversity and scalability.

• CyclicReflex: Improving Reasoning Models via Cyclical Reflection Token Scheduling

  Chongyu Fan, Yihua Zhang, Jinghan Jia, Alfred Hero, Sijia Liu · Jun 4, 2025 · Citations: 0

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoning

  Dongxu Zhang, Yujun Wu, Yiding Sun, Jinnan Yang, Ning Yang · Aug 7, 2025 · Citations: 0

  Automatic Metrics

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs

  Ngoc Bui, Shubham Sharma, Simran Lamba, Saumitra Mishra, Rex Ying · Dec 3, 2025 · Citations: 0

  Automatic Metrics

  Across mathematical reasoning (GSM8K, MATH-500, AIME24), procedural generation (LongProc), conversational long-memory benchmarks (LongMemEval), and long-context understanding (LongBenchV2 and SCBench), TRIM-KV consistently outperforms…

• InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models

  Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang · Mar 9, 2025 · Citations: 0

  Experiments across multiple model architectures demonstrate that our approach reduces computational costs while improving performance, with Qwen2.5-Math-7B showing 3-11% improvements across MATH500, AIME24, and GPQA_diamond benchmarks.

• Spurious Rewards: Rethinking Training Signals in RLVR

  Rulin Shao, Shuyue Stella Li, Rui Xin, Scott Geng, Yiping Wang · Jun 12, 2025 · Citations: 0

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Overconfident Errors Need Stronger Correction: Asymmetric Confidence Penalties for Reinforcement Learning

  Yuanda Xu, Hejian Sang, Zhengze Zhou, Ran He, Zhipeng Wang · Feb 24, 2026 · Citations: 0

  Automatic Metrics

  Evaluated on MATH-500 and AIME 2025, ACE composes seamlessly with existing methods and consistently improves the full Pass@k spectrum across all three model families and benchmarks.

• What If We Allocate Test-Time Compute Adaptively?

  Ahsan Bilal, Ahmed Mohsin, Muhammad Umer, Ali Subhan, Hassan Rizwan · Feb 1, 2026 · Citations: 0

  For each problem, the agent runs multiple inference iterations.

• $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts

  Mert Cemri, Nived Rajaraman, Rishabh Tiwari, Xiaoxuan Liu, Kurt Keutzer · Jun 15, 2025 · Citations: 0

  Automatic Metrics

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Parameter-Efficient Fine-Tuning of LLMs with Mixture of Space Experts

  Buze Zhang, Jinkai Tao, Zilang Zeng, Neil He, Ali Maatouk · Feb 16, 2026 · Citations: 0

  Our experiments across diverse benchmarks demonstrate that MoSLoRA consistently outperforms strong baselines, achieving up to 5.6% improvement on MATH500 and 15.9% on MAWPS.

Related Benchmark Hubs