GPQA or HumanEval+ Benchmark Papers

Researcher Quick Triage

Use this page for benchmark-matched method comparisons and eval protocol selection. Quality band: Medium .

• 26 papers explicitly name benchmark datasets in the sampled set.
• 20 papers report at least one metric term in metadata extraction.
• Start with the ranked shortlist below before reading all papers.

Primary action: Start with the top 2 benchmark-matched papers, then compare evaluation modes in the protocol matrix.

Start Here (Benchmark-Matched First 6)

Ranked by protocol completeness so you can quickly find papers suitable for comparison studies.

• When Self-Consistency Backfires: Majority Vote Hurts the Majority of Hard Science Problems for Small LLMs

  Aug 11, 2026 · Citations: 0 · Score: 9.5

  Eval: Automatic Metrics · Metrics: Accuracy

• Decomposing Wrong-Consensus Agreement in LLM Self-Consistency

  Aug 19, 2026 · Citations: 0 · Score: 8.0

  Eval: Automatic Metrics · Metrics: Accuracy

• CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

  Jun 23, 2026 · Citations: 0 · Score: 7.5

  Eval: Automatic Metrics · Metrics: Accuracy

• Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation

  Sep 29, 2026 · Citations: 0 · Score: 7.0

  Eval: Automatic Metrics · Metrics: Throughput

• Theoria: Rewrite-Acceptability Verification over Informal Reasoning States

  Jul 1, 2026 · Citations: 0 · Score: 6.0

  Eval: Not reported · Metrics: Precision

• IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMs

  Jul 1, 2026 · Citations: 0 · Score: 6.0

  Eval: Automatic Metrics · Metrics: Accuracy

Protocol Matrix (Top 10)

Compare protocol ingredients quickly before deep-reading full papers.

Top Papers On This Benchmark

• When Self-Consistency Backfires: Majority Vote Hurts the Majority of Hard Science Problems for Small LLMs

  Utkarsh Bahuguna · Aug 11, 2026 · Citations: 0

  Demonstrations Automatic Metrics

  On the full GPQA Diamond benchmark (198 graduate-level science questions), majority voting reduces per-problem accuracy on a majority of problems for two instruction-tuned models from different families: 56.6% of problems for Qwen2.5-7B and…

• Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation

  Joel Anto Paul, Litu Rout, Aditya Akella, Sanjay Shakkottai · Sep 29, 2026 · Citations: 0

  Automatic Metrics

  Applied to DiffusionGemma-26B, TADM:Post-train improves throughput by approximately 49% to 79% on several math, code, and STEM benchmarks (GSM8K, AIME26, GPQA-Diamond, LiveCodeBench-v6, HumanEval+, MMLU-Pro).

• Decomposing Wrong-Consensus Agreement in LLM Self-Consistency

  Lizhuo Zhang, Mengmeng Tang, Chenfeng Long, Xiaoyong Tang, Xiang Luo · Aug 19, 2026 · Citations: 0

  Pairwise Preference Automatic Metrics

  The mechanical reference is leak-free: each case's preference and accuracy are estimated from its other runs only.

• The Metanym Game: An LLM Benchmark Without Ground Truth That Rises With the Models It Measures

  David Nordfors · Jun 19, 2026 · Citations: 0

  In our benchmark, LLMs compete in generating sets of analogous statements and rate each other's sets on their own understandings of factual correctness, beauty, intelligence, distinctness, length, and structural diversity.

• Epistemic Goggles: A Pretrained Module that Induces an Epistemic Frame via Gradient Editing

  Joshua Penman · Jul 2, 2026 · Citations: 0

  In our evaluations, models trained on documents prefixed and suffixed with such annotations correctly identify the relevant claims as fictional only about 9% of the time.

• Theoria: Rewrite-Acceptability Verification over Informal Reasoning States

  Michael Saldivar, Ben Slivinski · Jul 1, 2026 · Citations: 0

  Formal proof assistants offer certainty but cannot reach most of the problem distribution; scalar LLM judges offer coverage but produce opaque scores that cannot be audited after the fact and are subject to the same coherence issues as any…

• IsoSci: A Benchmark of Isomorphic Cross-Domain Science Problems for Evaluating Reasoning versus Knowledge Retrieval in LLMs

  Samir Abdaljalil, Erchin Serpedin, Hasan Kurban · Jul 1, 2026 · Citations: 0

  Automatic Metrics

  We introduce ISOSCI, a benchmark of isomorphic cross-domain science problem pairs that separates reasoning ability from domain knowledge retrieval in LLM evaluation.

• CALIBER: Calibrating Confidence Before and After Reasoning in Language Models

  Conor Finlay, Joshua Kurien, Saurabh Dash, Marzieh Fadaee, Beyza Ermis · Jun 23, 2026 · Citations: 0

  Automatic Metrics

  Ablations further show that this position-target alignment is most beneficial under distribution shift where it consistently reduces calibration error across all out-of-distribution benchmarks.

• SIGMA: Search-Augmented On-Demand Knowledge Integration for Agentic Mathematical Reasoning

  Ali Asgarov, Umid Suleymanov, Aadyant Khatri · Oct 31, 2025 · Citations: 0

  Automatic Metrics

  We introduce SIGMA (Search-Augmented On-Demand Knowledge Integration for AGentic Mathematical reAsoning), a unified framework that orchestrates specialized agents to independently reason, perform targeted searches, and synthesize findings…

• Reasoning with Sampling: Cutting at Decision Points

  Felix Zhou, Anay Mehrotra, Quanquan C. Liu · May 28, 2026 · Citations: 0

  Across MATH500, HumanEval+, GPQA Diamond, and AIME26, our method consistently improves over baselines and RL-trained models.

• Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought

  Siddharth Boppana, Annabel Ma, Max Loeffler, Raphael Sarfati, Eric Bigelow · Mar 5, 2026 · Citations: 0

  Automatic Metrics

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Modeling Hierarchical Thinking in Large Reasoning Models

  G M Shahariar, Erfan Shayegani, Ali Nazari, Nael Abu-Ghazaleh · Oct 25, 2025 · Citations: 0

  Experiments across four benchmarks (AIME25, MATH-500, GSM8k, and GPQA Diamond) using three state-of-the-art open reasoning models demonstrate that Q-Value steering policy achieves significant performance gains with "surgical" efficiency,…

• Accelerated Test-Time Scaling with Model-Free Speculative Sampling

  Woomin Song, Saket Dingliwal, Sai Muralidhar Jayanthi, Bhavana Ganesh, Jinwoo Shin · Jun 5, 2025 · Citations: 0

  Automatic Metrics

  Extensive evaluations across multiple models and reasoning tasks (AIME-2024, GPQA-Diamond, and LiveCodeBench) demonstrate that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding while maintaining…

• MUR: Momentum Uncertainty guided Reasoning for Large Language Models

  Hang Yan, Fangzhi Xu, Rongman Xu, Yifei Li, Jian Zhang · Jul 20, 2025 · Citations: 0

  Automatic Metrics

  MUR is comprehensively evaluated against various TTS methods across four challenging benchmarks (MATH-500, AIME24, AIME25, and GPQA-diamond) using different sizes of recent Qwen3 models (1.7B, 4B, and 8B).

• Diffusion-Inspired Masked Fine-Tuning for Knowledge Injection in Autoregressive LLMs

  Xu Pan, Ely Hahami, Jingxuan Fan, Ziqian Xie, Haim Sompolinsky · Oct 10, 2025 · Citations: 0

  Automatic Metrics

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• When to Think, When to Speak: Learning Disclosure Policies for LLM Reasoning

  Jiaqi Wei, Xuehang Guo, Pengfei Yu, Xiang Zhang, Wanli Ouyang · May 5, 2026 · Citations: 0

  Automatic Metrics

  Across two Qwen3 architectures/scales (MoE Qwen3-30B-A3B, dense Qwen3-4B) and both in-domain (AIME25) and out-of-domain (GPQA-Diamond) benchmarks, SxS improves accuracy--content-latency Pareto trade-offs under token-level proxies such as…

• DeepPrune: Parallel Scaling without Inter-trace Redundancy

  Shangqing Tu, Yaxuan Li, Yushi Bai, Lei Hou, Juanzi Li · Oct 9, 2025 · Citations: 0

  Llm As JudgeAutomatic Metrics

  Our method features a specialized judge model trained with out-of-distribution data (AIME 2022, AIME 2023, and MATH 500) using oversampling techniques to accurately predict answer equivalence from partial reasoning traces, achieving 0.7072…

• Schema for In-Context Learning

  Pan Chen, Shaohong Chen, Mark Wang, Shi Xuan Leong, Priscilla Fung · Oct 14, 2025 · Citations: 0

  Demonstrations

  Inspired by cognitive science, specifically schema theory, which holds that humans interpret new information by activating pre-existing mental frameworks (schemas) to structure understanding, we introduce Schema-Activated In-Context…

• Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models

  Richard J. Young · Mar 27, 2026 · Citations: 0

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• Top-b: Entropic Regulation of Relative Probability Bands in Autoregressive Language Processes

  Deepon Halder, Raj Dabre · Mar 15, 2026 · Citations: 0

  Automatic Metrics

  Empirical validation on GPQA and GSM8K benchmarks indicates that Top-b significantly reduces generation entropy and inter-decoding variance while maintaining competitive reasoning accuracy, effectively approximating a self-regulating…

• CyclicReflex: Improving Reasoning Models via Cyclical Reflection Token Scheduling

  Chongyu Fan, Yihua Zhang, Jinghan Jia, Alfred Hero, Sijia Liu · Jun 4, 2025 · Citations: 0

  Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.

• D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models

  Shunsuke Ubukata · Feb 25, 2026 · Citations: 0

  Automatic Metrics

  In this study, we propose Disciplined Chain-of-Thought (D-CoT), a novel framework that enforces a structured reasoning process using control tags -- such as <TEMP_LOW> for fact-checking and <TEMP_HIGH> for multi-perspective exploration --…

• InftyThink: Breaking the Length Limits of Long-Context Reasoning in Large Language Models

  Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang · Mar 9, 2025 · Citations: 0

  Experiments across multiple model architectures demonstrate that our approach reduces computational costs while improving performance, with Qwen2.5-Math-7B showing 3-11% improvements across MATH500, AIME24, and GPQA_diamond benchmarks.

• Slm-mux: Orchestrating small language models for reasoning

  Chenyu Wang, Zishen Wan, Hao Kang, Emma Chen, Zhiqiang Xie · Oct 6, 2025 · Citations: 0

  Automatic Metrics

  Additional experiments show that the core principle of SLM-MUX extends to open-ended generation tasks (e.g., HumanEval+) and benefits other model classes, including frontier LLMs and domain-specific fine-tuned SLMs.

• HEART: Emotionally-Driven Test-Time Scaling of Language Models

  Gabriela Pinto, Palash Goyal, Mihir Parmar, Yiwen Song, Souradip Chakraborty · Sep 26, 2025 · Citations: 0

  Automatic Metrics

  We introduce HEART, a framework that uses emotional cues to guide the model's focus, much like how feelings contribute to human decision-making.

• Recursive Concept Evolution for Compositional Reasoning in Large Language Models

  Sarim Chaudhry · Feb 17, 2026 · Citations: 0

  Automatic Metrics

  Large language models achieve strong performance on many complex reasoning tasks, yet their accuracy degrades sharply on benchmarks that require compositional reasoning, including ARC-AGI-2, GPQA, MATH, BBH, and HLE.

Related Benchmark Hubs