HFEPX Benchmark Hub

Reasoning & Math Suite Benchmark Papers

Updated from current HFEPX corpus (Mar 8, 2026). 16 papers are grouped in this benchmark page.

Read Full Context

Updated from current HFEPX corpus (Mar 8, 2026). 16 papers are grouped in this benchmark page. Common evaluation modes: Automatic Metrics, Human Eval. Common annotation unit: Ranking. Frequently cited benchmark: GSM8K. Common metric signal: accuracy. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 21, 2026.

Papers: 16 Last published: Feb 21, 2026 Global RSS

Researcher Quick Triage

Use this page for benchmark-matched method comparisons and eval protocol selection. Quality band: Medium .

High-Signal Coverage

100.0%

16 / 16 sampled papers are not low-signal flagged.

Replication-Ready Set

Papers with explicit benchmark + metric + eval mode fields.

Quality Controls

0.0%

0 papers report calibration/adjudication/IAA controls.

7 papers explicitly name benchmark datasets in the sampled set.
5 papers report at least one metric term in metadata extraction.
Start with the ranked shortlist below before reading all papers.

Primary action: Start with the top 2 benchmark-matched papers, then compare evaluation modes in the protocol matrix.

Why This Matters (Expanded)

Why This Matters For Eval Research

42.9% of papers report explicit human-feedback signals, led by pairwise preferences.
automatic metrics appears in 31.3% of papers in this hub.
GSM8K is a recurring benchmark anchor for cross-paper comparisons in this page.

Protocol Notes (Expanded)

Protocol Takeaways

Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Rater context is mostly unspecified rater pools, and annotation is commonly ranking annotation; use this to scope replication staffing.
Pair this hub with llm_as_judge pages to benchmark automated-vs-human evaluation tradeoffs.

Benchmark Interpretation

GSM8K appears in 42.9% of hub papers (3/16); use this cohort for benchmark-matched comparisons.
MMLU appears in 42.9% of hub papers (3/16); use this cohort for benchmark-matched comparisons.

Metric Interpretation

accuracy is reported in 42.9% of hub papers (3/16); compare with a secondary metric before ranking methods.
cost is reported in 28.6% of hub papers (2/16); compare with a secondary metric before ranking methods.

Start Here (Benchmark-Matched First 6)

Ranked by protocol completeness so you can quickly find papers suitable for comparison studies.

Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
Jun 3, 2025 · Citations: 0 · Score: 7.0

Eval: Automatic Metrics · Metrics: Pass@1
D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models
Feb 25, 2026 · Citations: 0 · Score: 7.0

Eval: Automatic Metrics · Metrics: Accuracy
Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference
Feb 25, 2026 · Citations: 0 · Score: 7.0

Eval: Automatic Metrics · Metrics: Accuracy
Think$^{2}$: Grounded Metacognitive Reasoning in Large Language Models
Feb 21, 2026 · Citations: 0 · Score: 6.5

Eval: Human Eval · Metrics: Not Reported
Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs
Dec 3, 2025 · Citations: 0 · Score: 6.0

Eval: Automatic Metrics · Metrics: Cost
SPARE: Single-Pass Annotation with Reference-Guided Evaluation for Automatic Process Supervision and Reward Modelling
Jun 18, 2025 · Citations: 0 · Score: 5.5

Eval: Automatic Metrics · Metrics: Accuracy

Protocol Matrix (Top 10)

Compare protocol ingredients quickly before deep-reading full papers.

Paper	Eval Modes	Human Feedback	Metrics	Quality Controls
Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback Jun 3, 2025	Automatic Metrics	Critique Edit	Pass@1	Not reported
D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models Feb 25, 2026	Automatic Metrics	Not reported	Accuracy	Not reported
Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference Feb 25, 2026	Automatic Metrics	Not reported	Accuracy, Cost	Not reported
Think$^{2}$: Grounded Metacognitive Reasoning in Large Language Models Feb 21, 2026	Human Eval	Pairwise Preference	Not reported	Not reported
Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs Dec 3, 2025	Automatic Metrics	Not reported	Cost	Not reported
SPARE: Single-Pass Annotation with Reference-Guided Evaluation for Automatic Process Supervision and Reward Modelling Jun 18, 2025	Automatic Metrics	Not reported	Accuracy, Precision	Not reported
Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at Scale Nov 7, 2025	Not reported	Pairwise Preference	Not reported	Not reported
Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought Mar 5, 2026	Not reported	Not reported	Not reported	Not reported
$V_1$: Unifying Generation and Self-Verification for Parallel Reasoners Mar 4, 2026	Not reported	Not reported	Not reported	Not reported
In-Context Environments Induce Evaluation-Awareness in Language Models Mar 4, 2026	Not reported	Not reported	Not reported	Not reported

Researcher Workflow (Detailed)

Checklist

Moderate: Papers with explicit human feedback

Coverage is usable but incomplete (42.9% vs 45% target).
Gap: Papers reporting quality controls

Coverage is a replication risk (0% vs 30% target).
Strong: Papers naming benchmarks/datasets

Coverage is strong (100% vs 35% target).
Strong: Papers naming evaluation metrics

Coverage is strong (71.4% vs 35% target).
Gap: Papers with known rater population

Coverage is a replication risk (0% vs 35% target).
Strong: Papers with known annotation unit

Coverage is strong (42.9% vs 35% target).

Strengths

Most papers provide measurable evaluation context (100% benchmarks, 71.4% metrics).
Agentic evaluation appears in 57.1% of papers.

Known Gaps

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (0% coverage).

Suggested Next Analyses

Pair this hub with llm_as_judge pages to benchmark automated-vs-human evaluation tradeoffs.
Stratify by benchmark (GSM8K vs MMLU) before comparing methods.
Track metric sensitivity by reporting both accuracy and cost.

Recommended Queries

Human Eval Protocols Benchmark Slice: GSM8K Metric Slice: accuracy Recent High-Signal Papers

Known Limitations

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (0% coverage).
Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Snapshot (Detailed)

Evaluation Modes

Automatic Metrics (5)
Human Eval (1)

Human Feedback Mix

Pairwise Preference (2)
Critique Edit (1)

Top Benchmarks

GSM8K (3)
MMLU (3)
AIME (2)
MMLU Pro (2)

Top Metrics

Accuracy (3)
Cost (2)
Pass@1 (1)
Precision (1)

Top Papers On This Benchmark

Think$^{2}$: Grounded Metacognitive Reasoning in Large Language Models
Abraham Paul Elenjical, Vivek Hruday Kavuri, Vasudeva Varma · Feb 21, 2026 · Citations: 0

Pairwise Preference Human Eval

We introduce a psychologically grounded metacognitive framework that operationalizes Ann Brown's regulatory cycle (Planning, Monitoring, and Evaluation) as a structured prompting architecture, and study its integration within a lightweight…
Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback
Xiaoying Zhang, Yipeng Zhang, Hao Sun, Kaituo Feng, Chaochao Lu · Jun 3, 2025 · Citations: 0

Critique Edit Automatic Metrics

We show that plateaued RL models can successfully refine failed solutions when given natural language critiques.
D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models
Shunsuke Ubukata · Feb 25, 2026 · Citations: 0

Automatic Metrics

In this study, we propose Disciplined Chain-of-Thought (D-CoT), a novel framework that enforces a structured reasoning process using control tags -- such as <TEMP_LOW> for fact-checking and <TEMP_HIGH> for multi-perspective exploration --…
Cache What Lasts: Token Retention for Memory-Bounded KV Cache in LLMs
Ngoc Bui, Shubham Sharma, Simran Lamba, Saumitra Mishra, Rex Ying · Dec 3, 2025 · Citations: 0

Automatic Metrics

Across mathematical reasoning (GSM8K, MATH-500, AIME24), procedural generation (LongProc), conversational long-memory benchmarks (LongMemEval), and long-context understanding (LongBenchV2 and SCBench), TRIM-KV consistently outperforms…
SPARE: Single-Pass Annotation with Reference-Guided Evaluation for Automatic Process Supervision and Reward Modelling
Md Imbesat Hassan Rizvi, Xiaodan Zhu, Iryna Gurevych · Jun 18, 2025 · Citations: 0

Automatic Metrics

To address this, we introduce Single-Pass Annotation with Reference-Guided Evaluation (SPARE), a novel structured framework that enables efficient per-step annotation by jointly aligning solution steps to reference solutions and determine…
Confidence-Driven Multi-Scale Model Selection for Cost-Efficient Inference
Bo-Wei Chen, Chung-Chi Chen, An-Zi Yen · Feb 25, 2026 · Citations: 0

Automatic Metrics

Experiments on the Massive Multitask Language Understanding (MMLU) benchmark show that our approach achieves accuracy comparable to the largest model while reducing computational costs by 20\% to 40\%.
Long Grounded Thoughts: Synthesizing Visual Problems and Reasoning Chains at Scale
David Acuna, Chao-Han Huck Yang, Yuntian Deng, Jaehun Jung, Ximing Lu · Nov 7, 2025 · Citations: 0

Pairwise Preference

We introduce a framework able to synthesize vision-centric problems spanning diverse levels of complexity, and the resulting dataset with over 1M high-quality problems including: reasoning traces, preference data, and instruction prompts…
Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought
Siddharth Boppana, Annabel Ma, Max Loeffler, Raphael Sarfati, Eric Bigelow · Mar 5, 2026 · Citations: 0
$V_1$: Unifying Generation and Self-Verification for Parallel Reasoners
Harman Singh, Xiuyu Li, Kusha Sareen, Monishwaran Maheswaran, Sijun Tan · Mar 4, 2026 · Citations: 0
In-Context Environments Induce Evaluation-Awareness in Language Models
Maheep Chaudhary · Mar 4, 2026 · Citations: 0
Tool Verification for Test-Time Reinforcement Learning
Ruotong Liao, Nikolai Röhrich, Xiaohan Wang, Yuhui Zhang, Yasaman Samadzadeh · Mar 2, 2026 · Citations: 0
CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning
Xinyu Zhu, Yihao Feng, Yanchao Sun, Xianzhi Du, Pingzhi Li · Mar 1, 2026 · Citations: 0
Draft-Thinking: Learning Efficient Reasoning in Long Chain-of-Thought LLMs
Jie Cao, Tianwei Lin, Zhenxuan Fan, Bo Yuan, Ziyuan Zhao · Feb 28, 2026 · Citations: 0
MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes
Changsheng Zhao, Ernie Chang, Zechun Liu, Chia-Jung Chang, Wei Wen · Sep 29, 2025 · Citations: 0
Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding
Shijing Hu, Jingyang Li, Zhihui Lu, Pan Zhou · Sep 26, 2025 · Citations: 0
Latent Self-Consistency for Reliable Majority-Set Selection in Short- and Long-Answer Reasoning
Jungsuk Oh, Jay-Yoon Lee · Aug 25, 2025 · Citations: 0

Related Benchmark Hubs

Reasoning & Math Suite Benchmark Papers In CS.CL Reasoning & Math Suite Benchmark Papers In CS.AI CS.CL Human Feedback And Eval Papers CS.AI Human Feedback And Eval Papers AIME Benchmark Papers AIME In CS.CL Papers WebArena Ecosystem Benchmark Papers (15) SWE-Bench Ecosystem Benchmark Papers (10) WebArena Ecosystem Benchmark Papers In CS.CL (10) Reasoning & Math Suite Benchmark Papers In CS.CL (15) Arena/Judge Suite Benchmark Papers (10) Reasoning & Math Suite Benchmark Papers In CS.AI (13)

Need human evaluators for your AI research? Scale annotation with expert AI Trainers.

Post a Job Get a Quote