HFEPX Benchmark Hub
AIME or BFCL or MMLU Benchmark Papers
Updated from current HFEPX corpus (2026-10-29). This page tracks 60 papers reporting AIME or BFCL or MMLU benchmark evidence, with protocol and metric context for comparison.
← Back to explorer
Researcher Quick Triage
Use this page for benchmark-matched method comparisons and eval protocol selection. Quality band: High .
High-Signal Coverage
100.0%
60 / 60 sampled papers are not low-signal flagged.
Replication-Ready Set
40
Papers with explicit benchmark + metric + eval mode fields.
Quality Controls
3.3%
2 papers report calibration/adjudication/IAA controls.
- 60 papers explicitly name benchmark datasets in the sampled set.
- 42 papers report at least one metric term in metadata extraction.
- Start with the ranked shortlist below before reading all papers.
Primary action: Start with the top 2 benchmark-matched papers, then compare evaluation modes in the protocol matrix.
Why This Matters (Expanded)
Why This Matters For Eval Research
- Use this page to compare AIME or BFCL or MMLU papers by evaluation mode, metric, and evidence quality before reusing reported results.
Protocol Notes (Expanded)
Protocol Takeaways
- AIME or BFCL or MMLU papers are often paired with automatic_metrics, simulation_env.
Benchmark Interpretation
- MMLU: 32 papers
- AIME: 18 papers
- GSM8K: 12 papers
- BFCL: 11 papers
Metric Interpretation
- accuracy: 21 papers
- cost: 11 papers
- latency: 8 papers
- precision: 4 papers
Start Here (Benchmark-Matched First 6)
Ranked by protocol completeness so you can quickly find papers suitable for comparison studies.
- Evaluating and Benchmarking the System One Model Jev
Sep 29, 2026 · Citations: 0 · Score: 8.0
Eval: Automatic Metrics · Metrics: Accuracy
- Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeable
Sep 29, 2026 · Citations: 0 · Score: 8.0
Eval: Automatic Metrics · Metrics: Accuracy
- Decomposing Wrong-Consensus Agreement in LLM Self-Consistency
Aug 19, 2026 · Citations: 0 · Score: 8.0
Eval: Automatic Metrics · Metrics: Accuracy
- Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning
Aug 21, 2026 · Citations: 0 · Score: 8.0
Eval: Automatic Metrics · Metrics: Accuracy
- Skill or Skip? Learning Selective Skill Invocation in Agentic Tasks via Dual-Granularity Preference Learning
May 30, 2026 · Citations: 0 · Score: 7.5
Eval: Simulation Env · Metrics: Precision
- Will Scaling Improve Social Simulation with LLMs?
Jul 2, 2026 · Citations: 0 · Score: 7.5
Eval: Automatic Metrics, Simulation Env · Metrics: Accuracy
Protocol Matrix (Top 10)
Compare protocol ingredients quickly before deep-reading full papers.
| Paper | Eval Modes | Human Feedback | Metrics | Quality Controls |
|---|---|---|---|---|
| Evaluating and Benchmarking the System One Model Jev Sep 29, 2026 | Automatic Metrics | Rubric Rating | Accuracy, F1 | Not reported |
| Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeable Sep 29, 2026 | Automatic Metrics | Pairwise Preference | Accuracy | Not reported |
| Decomposing Wrong-Consensus Agreement in LLM Self-Consistency Aug 19, 2026 | Automatic Metrics | Pairwise Preference | Accuracy | Not reported |
| Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning Aug 21, 2026 | Automatic Metrics | Demonstrations | Accuracy, Latency | Not reported |
| Skill or Skip? Learning Selective Skill Invocation in Agentic Tasks via Dual-Granularity Preference Learning May 30, 2026 | Simulation Env | Pairwise Preference | Precision, Task success | Not reported |
| Will Scaling Improve Social Simulation with LLMs? Jul 2, 2026 | Automatic Metrics, Simulation Env | Not reported | Accuracy | Calibration |
| Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning Jun 24, 2026 | Automatic Metrics | Pairwise Preference | Accuracy, Pass@64 | Not reported |
| What RL Brings to Reasoning: A Few Decisions the Base Model Would Not Make Oct 17, 2026 | Automatic Metrics | Not reported | Pass@1 | Not reported |
| EdgeRock: Strong Reasoning at Constant Memory Oct 7, 2026 | Automatic Metrics | Not reported | Pass@1, Throughput | Not reported |
| Penalty-Framed No-Valid-Option MCQA: Analyzing LLM Abstention under Invalid Choices Oct 6, 2026 | Automatic Metrics | Not reported | Accuracy, Cost | Not reported |
Researcher Workflow (Detailed)
Checklist
-
Gap: Human feedback
Human feedback is present in 11 of 60 papers.
-
Gap: Quality controls
Quality controls is present in 2 of 60 papers.
-
Strong: Benchmarks
Benchmarks is present in 60 of 60 papers.
-
Strong: Metrics
Metrics is present in 42 of 60 papers.
-
Gap: Known rater population
Known rater population is present in 8 of 60 papers.
-
Gap: Known annotation unit
Known annotation unit is present in 12 of 60 papers.
Strengths
- Benchmarks is present in 60 of 60 papers.
- Metrics is present in 42 of 60 papers.
- Agentic evaluation is present in 18 of 60 papers.
Known Gaps
- Human feedback is present in 11 of 60 papers.
- Quality controls is present in 2 of 60 papers.
- Known rater population is present in 8 of 60 papers.
Suggested Next Analyses
- Review the most recent AIME or BFCL or MMLU papers first, then compare reported metrics and quality-control context before treating results as comparable.
Recommended Queries
Known Limitations
- This synthetic persisted page is generated from extraction data because the cached benchmark payload was missing for either-aime-or-bfcl-or-mmlu.
Research Utility Snapshot (Detailed)
Evaluation Modes
- Automatic Metrics (38)
- Simulation Env (3)
- Llm As Judge (2)
Human Feedback Mix
- None (49)
- Pairwise Preference (8)
- Demonstrations (2)
- Red Team (1)
Top Benchmarks
- MMLU (32)
- AIME (18)
- GSM8K (12)
- BFCL (11)
Top Metrics
- Accuracy (21)
- Cost (11)
- Latency (8)
- Precision (4)
Top Papers On This Benchmark
- AtManRL: Towards Faithful Reasoning via Differentiable Attention Saliency
Max Henning Höth, Kristian Kersting, Björn Deiseroth, Letitia Parcalabescu · Apr 17, 2026 · Citations: 0
Automatic MetricsAcross three models (Qwen2.5-3B, Granite-3.3-2B, and Phi-4-mini) and four benchmarks (GSM8K, MMLU, ProofWriter, and QuestBench), AtManRL sits above this trade-off curve by 3.9-4.0pp in saliency, measured as answer changes under mistake…
- Not All Flips Are Conformity: Decomposing Stance Convergence in Multi-Agent LLM Debate
Xiqi Hao, Zengqing Wu, Yu-Xuan Qiu, Chuan Xiao, Ruiqi Xu · May 30, 2026 · Citations: 0
Llm As JudgeAutomatic MetricsMulti-agent debate (MAD) among large language models (LLMs) is often judged by answer changes and convergence, yet these outcomes alone cannot establish that peers improved the answer.
- Toollery: Scaling LLM Agents to Thousands of Skills and Tools
Xiangxi Tian, Ran Guan · Sep 2, 2026 · Citations: 0
Automatic MetricsAs LLM agents are exposed to hundreds to tens of thousands of skills, tools, and API functions, full-library prompting becomes costly, slow, and less reliable: each added candidate increases prompt tokens and latency, while longer candidate…
- What RL Brings to Reasoning: A Few Decisions the Base Model Would Not Make
Siyu Zhang, Yuquan Lu, Yoshua Y. Li, Yifan Ding, Kun Zeng · Oct 17, 2026 · Citations: 0
Automatic MetricsAbstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- EdgeRock: Strong Reasoning at Constant Memory
Huishan Ji, Weiming Zhang, Qirui Ye, Jinchao Xu, Yike Guo · Oct 7, 2026 · Citations: 0
Automatic MetricsAbstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- Penalty-Framed No-Valid-Option MCQA: Analyzing LLM Abstention under Invalid Choices
Jinhyeok Kim, Hye-Young Jung · Oct 6, 2026 · Citations: 0
Automatic MetricsAbstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- PiERN: Token-Level Routing for Integrating High-Precision Computation and Reasoning
Jingyuan Fan, Purui Liu, Hengbo Xiao, Yuxuan Zheng, Jingzhao Zhang · Sep 17, 2025 · Citations: 0
Automatic MetricsResults show that PiERN achieves not only higher accuracy than directly finetuning LLMs but also significant improvements in response latency, token usage, GPU energy consumption, and experts routing accuracy compared with mainstream…
- Stabilizing Off-Policy Training for Long-Horizon LLM Agent via Turn-Level Importance Sampling and Clipping-Triggered Normalization
Chenliang Li, Adel Elmahdy, Alex Boyd, Zhongruo Wang, Siliang Zeng · Nov 25, 2025 · Citations: 0
To address these challenges, we propose SORL, Stabilizing Off-Policy Reinforcement Learning for Long-Horizon Agent Train- ing.
- Test-Time Adaptation of Reasoning Strategies with Bayesian Nonparametric Memory
Keshav Ramji, Tahira Naseem, Ramón Fernandez Astudillo · Oct 5, 2026 · Citations: 0
Automatic MetricsWe demonstrate that Bayesian Cheatsheet achieves clear performance gains relative to existing memory modules across reasoning benchmarks such as AIME'25, Omni-MATH, and PhysReason, even in the cold-start setting.
- Gaokerena: A Small Persian Medical Language Model Family
Mehrdad Ghassabi, Hamidreza Baradaran Kashani, Pedram Rostami, Sadra Hakim, Zahra Kazemi · Aug 2, 2026 · Citations: 0
Pairwise PreferenceRlaif Or Synthetic FeedbackThis training improved performance on a translated medical MMLU benchmark from 46.64% to 49.31%.
- HINT-SD: Targeted Hindsight Self-Distillation for Long-Horizon Agents
Woongyeong Yeo, Yumin Choi, Taekyung Ki, Sung Ju Hwang · May 18, 2026 · Citations: 0
Training long-horizon LLM agents with reinforcement learning is challenging because sparse outcome rewards reveal whether a task succeeds, but not which intermediate actions caused the outcome or how they should be corrected.
- AdviSD: Learning to Advise Frontier LLMs via Targeted Multi-Turn Self-Distillation
Rishabh Agrawal, Hejie Cui, Shasha Li, Shanchan Wu, Sercan Ö. Arık · Sep 29, 2026 · Citations: 0
Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- Time-Anchored Diffusion Language Models: Latent-Space Caching for Fast Generation
Joel Anto Paul, Litu Rout, Aditya Akella, Sanjay Shakkottai · Sep 29, 2026 · Citations: 0
Automatic MetricsApplied to DiffusionGemma-26B, TADM:Post-train improves throughput by approximately 49% to 79% on several math, code, and STEM benchmarks (GSM8K, AIME26, GPQA-Diamond, LiveCodeBench-v6, HumanEval, MMLU-Pro).
- Overcoming Scaling Limits in On-Policy Self-Distillation for LLM Reasoning
Md. Ismail Hossain, Humaira Kousar, Isidora Chara Tourni · Sep 29, 2026 · Citations: 0
Automatic MetricsWe separate these roles in a factorial analysis and find that scaffold correctness has a stronger effect on downstream accuracy than context correctness.
- Evaluating and Benchmarking the System One Model Jev
Tobias Deußer, Lorenz Sparrenberg, Rafet Sifa · Sep 29, 2026 · Citations: 0
Rubric Rating Automatic MetricsWe evaluate Jev (jev-1.13.0) zero-shot on 37 datasets spanning classification, routing, natural language inference, reading comprehension, commonsense reasoning, moderation, legal clause analysis and rubric scoring, with one frozen template…
- Authority Bias in Language Models: Source Deference and User Agreement Are Not Interchangeable
Abhinav Rajeev Kumar, Paras Chopra · Sep 29, 2026 · Citations: 0
Pairwise Preference Automatic MetricsIn three open-weight families, removing a fitted source direction lowers source compliance by 65-80 percentage points while removing a user or assistant direction has far smaller effects, and removing the user direction shows the reverse…
- Skill or Skip? Learning Selective Skill Invocation in Agentic Tasks via Dual-Granularity Preference Learning
Chishui Chen, Jiaye Lin, Te Sun, Yi Yang, Junxi Wang · May 30, 2026 · Citations: 0
Pairwise Preference Simulation EnvAgent skills are callable procedural modules that provide reusable knowledge and execution policies for complex agentic tasks.
- Decomposing Wrong-Consensus Agreement in LLM Self-Consistency
Lizhuo Zhang, Mengmeng Tang, Chenfeng Long, Xiaoyong Tang, Xiang Luo · Aug 19, 2026 · Citations: 0
Pairwise Preference Automatic MetricsThe mechanical reference is leak-free: each case's preference and accuracy are estimated from its other runs only.
- SPADE: Self-Play in Adaptive Synthetic Executable Environments
Bo Liu, Simon Yu, Yiding Jiang, Ao Qu, Andrew Zhao · Aug 19, 2026 · Citations: 0
For language agents, existing training environment pools (hand-curated, statically synthesized, or frozen-verifier) keep the goal distribution fixed as the learner scales.
- Breaking MCP with Function Hijacking Attacks: Novel Threats for Function Calling and Agentic Models
Yannis Belkhiter, Giulio Zizzo, Sergio Maffeis, Seshu Tirupathi, John D. Kelleher · Apr 22, 2026 · Citations: 0
Pairwise PreferenceRed Team Automatic MetricsThe growth of agentic AI has drawn significant attention to function calling Large Language Models (LLMs), which are designed to extend the capabilities of AI-powered system by invoking external functions.
- Auditing MCQA Benchmarks through Probability Landscapes
Minsoo Song, Chanjun Park · Aug 31, 2026 · Citations: 0
Pairwise PreferenceTo provide a scalable diagnostic approach, we propose a two-component probabilistic framework for auditing MCQA benchmarks using model output distributions.
- Kascade: A Practical Sparse Attention Method for Long-Context LLM Inference
Dhruv Deshmukh, Saurabh Goyal, Nipun Kwatra, Ramachandran Ramjee · Dec 18, 2025 · Citations: 0
Automatic MetricsKascade achieves up to 4.1x speedup in decode attention and 2.2x speedup in prefill attention over FlashAttention-3 baseline on H100 GPUs while closely matching dense attention accuracy on long-context benchmarks such as LongBench and…
- Tool Verification for Test-Time Reinforcement Learning
Ruotong Liao, Nikolai Röhrich, Xiaohan Wang, Yuhui Zhang, Yasaman Samadzadeh · Mar 2, 2026 · Citations: 0
Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- Reasoning or Rambling? Exploring the Effect of Thinking on Agent Persuasion
Haodong Zhao, Jidong Li, Zhaomin Wu, Tianjie Ju, Zhuosheng Zhang · Sep 25, 2025 · Citations: 0
Understanding persuasion is critical for the safety and reliability of multi-agent systems built on large language models (LLMs).
- Memory Augmentation Unlocks Efficient Chain-of-Thought Reasoning
Simeng Zhang, Yilong Chen, Wenyuan Zhang, Zhenyu Zhang, Yao Chen · Aug 21, 2026 · Citations: 0
Demonstrations Automatic MetricsBased on this principle, we propose Memory-Augmented Compression, a training-free framework that constructs reusable reasoning memories from historical traces and retrieves them as prefill-side scaffolds.
- RARE: Decoupling Representation Steering from Expert Routing in Mixture-of-Experts Language Models
Zhibo Zhang, Zhen Ouyang, Ling Shi, Kailong Wang · Aug 21, 2026 · Citations: 0
Automatic MetricsAbstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- Inject, Align, Recover: Staged Post-Training for Retrieval-Free Document Knowledge Internalization
Qian Kou, Xiaofeng Shi, Xiaosong Qiu, Hua Zhou · Aug 20, 2026 · Citations: 0
Automatic MetricsAbstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- Decoupled Contrastive Decoding via Expert-Aligned Drafting
Zhixuan Liu, Zhichen Dong, Yuanfu Wang, Chao Yang · Aug 13, 2026 · Citations: 0
Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- The Wording Effect: Quantifying Two-Way Drift in LLM Benchmark Performance
Shailja Thakur, Sungeun An, Chad DeLuca, Hima Patel · Aug 12, 2026 · Citations: 0
Automatic MetricsA benchmark score comes from a single phrasing of each problem.
- Beyond Single-Turn Confidence: Trajectory-Adapted Uncertainty Quantification for LLM Agents
Dylan Bouchard, Mohit Singh Chauhan · Aug 12, 2026 · Citations: 0
Automatic MetricsFor LLM agents, however, the unit of observation is an interactive trajectory, where the model can ask clarifying questions, call tools, update state, and make intermediate decisions whose errors propagate to the final outcome.
- Shorthand for Thought: Compressing LLM Reasoning via Entropy-Guided Supertokens
Zhenyu Zhao, Sander Land, Daniel M. Bikel, Waseem Alshikh · Apr 29, 2026 · Citations: 0
Automatic MetricsAcross three model families and five mathematical reasoning benchmarks, our approach shortens reasoning traces by 8.1% on average; under a TOST equivalence analysis at a +/- 2pp margin, accuracy is equivalent or inconclusive on 13/15 model…
- Will Scaling Improve Social Simulation with LLMs?
Caleb Ziems, William Held, Su Doga Karaca, David Grusky, Tatsunori Hashimoto · Jul 2, 2026 · Citations: 0
Automatic MetricsSimulation EnvWe use scaling laws to study the relationship between LLMs' compute scale, general capability benchmarks, and the fidelity of social simulation in three representative sub-domains: opinion modeling, behavioral simulation, and longitudinal…
- NITP: Next Implicit Token Prediction for LLM Pre-training
Xiangdong Zhang, Debing Zhang, Shaofeng Zhang, Xiaohan Qin, Yu Cheng · May 24, 2026 · Citations: 0
Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- PARTREP: Learning What to Repeat for Decoder-only LLMs
Andikawati P Widjaja, Yongjun Kim, Hyounghun Kim, Jaeho Lee · Jul 2, 2026 · Citations: 0
Automatic MetricsAcross eight benchmarks (including MMLU, GSM8K, and RULER) and three model families (Qwen2.5, Llama3.2, Gemma4), PartRep retains most of the gains of full repetition while using only 59.4\% of its KV cache and 79.0\% of its prefill FLOPs.
- On Compositional Learning Behaviours in Formal Mathematics
Kevin Yandoka Denamganaï · May 27, 2026 · Citations: 0
Automatic MetricsSelf-evolving scientific agents capable of conquering the hard tail of formal mathematics require Compositional Learning Behaviours (CLBs) -- the capacity to ground and recombine novel symbolic structures in context, beyond mere…
- Closing the Activation-Cone Blind Spot: Response-Time Probing and Unified Defense
Subhadip Mitra · Jun 28, 2026 · Citations: 0
Automatic MetricsInference-time safety methods for large language models have proliferated, yet no systematic comparison exists.
- Symbiotic-MoE: Unlocking the Synergy between Generation and Understanding
Xiangyue Liu, Zijian Zhang, Miles Yang, Zhao Zhong, Liefeng Bo · Apr 9, 2026 · Citations: 0
Abstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning
Jaeyong Ko, Pilsung Kang, Yukyung Lee · Jun 24, 2026 · Citations: 0
Pairwise Preference Automatic MetricsAcross seven models and three mathematical reasoning benchmarks (GSM1K, MATH500, AIME 2025), cliff tokens act as failure triggers; deleting the first cliff token and resampling recovers pass@64 to 1.0, while keeping it limits recovery to…
- SARA: Unlocking Multilingual Knowledge in Mixture-of-Experts via Semantically Anchored Routing Alignment
Tianyu Dong, Yangyang Liu, Jiang Zhou, Xinwei Wu, Xiaohu Zhao · Jun 24, 2026 · Citations: 0
We conduct experiments on 2 LLMs across 5 low-resource languages and 3 benchmarks.
- Holistic Data Scheduler for LLM Pre-training via Multi-Objective Reinforcement Learning
Chenhao Dang, Jing Ma, Mingjie Liao · Jun 23, 2026 · Citations: 0
Automatic MetricsOn The Pile benchmark, HDS reaches the final validation perplexity of the next best method with 44% fewer training iterations.
- RoPE-Aware Bit Allocation for KV-Cache Quantization
Fengfeng Liang, Yuechen Zhang, Jiaya Jia · Jun 23, 2026 · Citations: 0
Automatic MetricsAbstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- SIGMA: Search-Augmented On-Demand Knowledge Integration for Agentic Mathematical Reasoning
Ali Asgarov, Umid Suleymanov, Aadyant Khatri · Oct 31, 2025 · Citations: 0
Automatic MetricsWe introduce SIGMA (Search-Augmented On-Demand Knowledge Integration for AGentic Mathematical reAsoning), a unified framework that orchestrates specialized agents to independently reason, perform targeted searches, and synthesize findings…
- Where Does Social Reasoning Come From? Capability Provenance in Language Models
Glenn Matlin, Chandreyi Chakraborty, Saehee Eom, Mika Okamoto, Rayan Castilla · Jun 17, 2026 · Citations: 0
Training-data attribution measures how strongly each training document influences a model's predictions on a benchmark, but document-level scores are too noisy to identify which corpus regions support which capabilities, and prior work has…
- Resolution Diagnostics for Paired LLM Evaluation
Anany Kotawala · May 28, 2026 · Citations: 0
Pairwise PreferenceAcross two public LLM leaderboards, many displayed pairwise rankings do not meet a conventional paired-test resolution target under the actual paired evaluation design: 11 of 40 Open LLM Leaderboard v1 pairwise comparisons and 4 of 9…
- Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thought
Siddharth Boppana, Annabel Ma, Max Loeffler, Raphael Sarfati, Eric Bigelow · Mar 5, 2026 · Citations: 0
Automatic MetricsAbstract shows limited direct human-feedback or evaluation-protocol detail; use as adjacent methodological context.
- Latent Performance Profiling of Large Language Models
Tanmoy Chakraborty, Ayan Sengupta, Suparna Bhattacharya, Partha Pratim Chakrabarti, Amlan Chakrabarti · May 28, 2026 · Citations: 0
Automatic MetricsLarge language models (LLMs) frequently achieve impressive scores on standardized benchmarks, yet accuracy alone offers a limited view of their capabilities.
- Accelerated Test-Time Scaling with Model-Free Speculative Sampling
Woomin Song, Saket Dingliwal, Sai Muralidhar Jayanthi, Bhavana Ganesh, Jinwoo Shin · Jun 5, 2025 · Citations: 0
Automatic MetricsExtensive evaluations across multiple models and reasoning tasks (AIME-2024, GPQA-Diamond, and LiveCodeBench) demonstrate that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding while maintaining…
- Benchmarking EngGPT2-16B-A3B against Comparable Italian and International Open-source LLMs
Andrea Sassella, Andrea Chizzola, Tommaso Bianchi, Luca Alessandrelli, Mark James Carman · May 8, 2026 · Citations: 0
This report benchmarks the performance of ENGINEERING Ingegneria Informatica S.p.A.'s EngGPT2MoE-16B-A3B LLM, a 16B parameter Mixture of Experts (MoE) model with 3B active parameters.
- Tool Calling is Linearly Readable and Steerable in Language Models
Zekun Wu, Ze Wang, Seonglae Cho, Yufei Yang, Adriano Koshiyama · May 8, 2026 · Citations: 0
Automatic MetricsWhen a tool-calling agent picks the wrong tool, the failure is invisible until execution: the email gets sent, the meeting gets missed.
- SOD: Step-wise On-policy Distillation for Small Language Model Agents
Qiyong Zhong, Mao Zheng, Mingyang Song, Xin Lin, Jie Sun · May 8, 2026 · Citations: 0
To address this, we propose SOD, a step-wise on-policy distillation framework for small language model agents, which adaptively reweights distillation strength at each step based on step-level divergence.
- Mathematical Reasoning via Intervention-Based Time-Series Causal Discovery Using LLMs as Concept Mastery Simulators
Tsuyoshi Okita · May 8, 2026 · Citations: 0
Simulation EnvWith a 7B-parameter LLM whose weights are entirely frozen, CIKA achieves 69.7\% on the contamination-free Omni-MATH-Rule benchmark and 64.0\% overall, compared to 60.5\% for o1-mini, and 97.2\% on GSM8K, 46--50\% on AIME 2024--2026, and…
- Teaching Thinking Models to Reason with Tools: A Full-Pipeline Recipe for Tool-Integrated Reasoning
Qianjia Cheng, Yuchen Zhang, Zhilin Wang, Yuxin Zuo, Shunkai Zhang · May 7, 2026 · Citations: 0
Automatic MetricsParadoxically, we observe that tool-enabled evaluation can degrade reasoning performance even when the strong thinking models make almost no actual tool calls.
- Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarking
Solomon Messing · Apr 13, 2026 · Citations: 0
Demonstrations Llm As JudgeLLM evaluations drive which models get deployed, what safety standards get adopted, which research conclusions get published, and how projections of AI's labor-market impact get made.
- FIT to Forget: Robust Continual Unlearning for Large Language Models
Xiaoyu Xu, Minxin Du, Kun Fang, Yaxin Xiao, Zhicong Huang · Jan 29, 2026 · Citations: 0
Furthermore, to facilitate rigorous evaluation, we introduce PCH, a unified benchmark encompassing Personal, Copyrighted, and Harmful content, alongside two symmetric metrics, Forget Degree (F.D.) and Retain Utility (R.U.), to…
- Beyond Majority Voting: Towards Fine-grained and More Reliable Reward Signal for Test-Time Reinforcement Learning
Weiqin Wang, Yile Wang, Kehao Chen, Hui Huang · Dec 17, 2025 · Citations: 0
We conduct experiments across various models and benchmarks, experimental results show that SCOPE consistently outperforms recent baselines.
- Fragile Knowledge, Robust Instruction-Following: The Width Pruning Dichotomy in Llama-3.2
Pere Martra · Dec 27, 2025 · Citations: 0
Automatic MetricsWe evaluated seven expansion ratio configurations using comprehensive benchmarks assessing factual knowledge, mathematical reasoning, language comprehension, instruction-following, and truthfulness.
- UR$^2$: Unify RAG and Reasoning through Reinforcement Learning
Weitao Li, Boran Xiang, Xiaolong Wang, Zhinan Gou, Weizhi Ma · Aug 8, 2025 · Citations: 0
Experiments on open-domain QA, MMLU-Pro, medical, and mathematical reasoning tasks show that UR^2, built on Qwen-2.5-3/7B and LLaMA-3.1-8B, consistently outperforms existing RAG and RL baselines, and achieves performance comparable to…
- The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check
Qingyu Lu, Liang Ding, Kanjian Zhang, Jinxia Zhang, Dacheng Tao · Jan 19, 2026 · Citations: 0
Automatic MetricsIn this work, we present a comprehensive evaluation of dLLMs (e.g., LLaDA, Dream) across two distinct agentic paradigms: Embodied Agents (requiring long-horizon planning) and Tool-Calling Agents (requiring precise formatting).
- Cost-Effective Communication: An Auction-based Method for Language Agent Interaction
Yijia Fan, Jusheng Zhang, Kaitong Cai, Jing Yang, Chengpei Tang · Nov 17, 2025 · Citations: 0
Automatic MetricsTo address this, we introduce the Dynamic Auction-based Language Agent (DALA), a novel framework that treats communication bandwidth as a scarce and tradable resource.
- SkillX: Automatically Constructing Skill Knowledge Bases for Agents
Chenxi Wang, Zhuoyun Yu, Xin Xie, Wuguannan Yao, Runnan Fang · Apr 6, 2026 · Citations: 0
Automatic MetricsLearning from experience is critical for building capable large language model (LLM) agents, yet prevailing self-evolving paradigms remain inefficient: agents learn in isolation, repeatedly rediscover similar behaviors from limited…
Related Benchmark Hubs
BFCL or GPQA Benchmark Papers (23)
BFCL or HumanEval+ Benchmark Papers (21)
BFCL or MATH-500 Benchmark Papers (25)
GPQA or HumanEval+ Benchmark Papers (22)
GPQA or MATH-500 Benchmark Papers (25)
HumanEval+ or MATH-500 Benchmark Papers (24)
MATH-500 Benchmark Papers (15)
AIME or AlpacaEval or MMLU Benchmark Papers (60)
AIME or GSM8K or MMLU Benchmark Papers (72)
AIME or HotpotQA or MMLU Benchmark Papers (62)
AIME or HumanEval+ or MMLU Benchmark Papers (57)