HFEPX Metric Hub

Inference Cost In CS.CL Papers

Updated from current HFEPX corpus (Jun 30, 2026). 37 papers are grouped in this metric page.

Read Full Context

Updated from current HFEPX corpus (Jun 30, 2026). 37 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Llm As Judge. Most common rater population: Domain Experts. Common annotation unit: Trajectory. Frequently cited benchmark: BrowseComp. Common metric signal: cost. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Mar 31, 2026.

Papers: 37 Last published: Mar 31, 2026 Global RSS

When This Metric Page Is Useful

Useful for background comparison, but still validate benchmark and protocol details in the linked papers. Quality band: Medium .

Metric Coverage

35.1%

13 sampled papers include metric names.

Benchmark Anchoring

10.8%

Papers with explicit dataset/benchmark anchors for fair comparison.

Quality Controls

0.0%

0 papers report calibration/adjudication/IAA controls.

37 papers are not low-signal flagged in this sample.
Use the protocol matrix below to avoid comparing metrics across incompatible eval setups.

Recommended next step: Treat this as directional signal only; metric reporting is present but benchmark anchoring is still thin.

Main limitation: Benchmark coverage is still thin, so avoid treating this page as a definitive guide to the metric.

What This Metric Page Tells You

46.2% of papers report explicit human-feedback signals, led by critique/edit feedback.
automatic metrics appears in 32.4% of papers in this hub.
BrowseComp is a recurring benchmark anchor for cross-paper comparisons in this page.

Metric Notes (Expanded)

Metric-Driven Protocol Takeaways

Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Rater context is mostly domain experts, and annotation is commonly trajectory-level annotation; use this to scope replication staffing.
Pair this hub with a human_eval-heavy hub to validate judge-model calibration.

Metric Interpretation

cost is reported in 100% of hub papers (13/37); compare with a secondary metric before ranking methods.
inference cost is reported in 100% of hub papers (13/37); compare with a secondary metric before ranking methods.

Benchmark Context

BrowseComp appears in 7.7% of hub papers (1/37); use this cohort for benchmark-matched comparisons.
GAIA appears in 7.7% of hub papers (1/37); use this cohort for benchmark-matched comparisons.

Start Here (Metric-Reliable First 6)

Ranked for metric reporting completeness and comparability.

S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models
Apr 1, 2026 · Citations: 0 · Score: 7.0

Metrics: Pass@1, Cost · Eval: Automatic Metrics
Learning When to Sample: Confidence-Aware Self-Consistency for Efficient LLM Chain-of-Thought Reasoning
Mar 9, 2026 · Citations: 0 · Score: 7.0

Metrics: Accuracy, Cost · Eval: Automatic Metrics
GLiGuard: Schema-Conditioned Classification for LLM Safeguard
May 8, 2026 · Citations: 0 · Score: 7.0

Metrics: Accuracy, F1 · Eval: Automatic Metrics
$\texttt{YC-Bench}$: Benchmarking AI Agents for Long-Term Planning and Consistent Execution
Apr 1, 2026 · Citations: 0 · Score: 7.0

Metrics: Cost, Inference cost · Eval: Automatic Metrics
Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization
Feb 26, 2026 · Citations: 0 · Score: 7.0

Metrics: Accuracy, Latency · Eval: Automatic Metrics
Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE
Mar 31, 2026 · Citations: 0 · Score: 6.5

Metrics: Ndcg, Cost · Eval: Automatic Metrics

Metric Protocol Matrix (Top 10)

Compare metric, benchmark, and evaluation context side by side.

Paper	Metrics	Benchmarks	Eval Modes	Quality Controls
S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models Apr 1, 2026	Pass@1, Cost	MATH 500, GSM8K	Automatic Metrics	Not reported
Learning When to Sample: Confidence-Aware Self-Consistency for Efficient LLM Chain-of-Thought Reasoning Mar 9, 2026	Accuracy, Cost	MMLU	Automatic Metrics	Not reported
GLiGuard: Schema-Conditioned Classification for LLM Safeguard May 8, 2026	Accuracy, F1	Not reported	Automatic Metrics	Not reported
$\texttt{YC-Bench}$: Benchmarking AI Agents for Long-Term Planning and Consistent Execution Apr 1, 2026	Cost, Inference cost	Yc Bench	Automatic Metrics	Not reported
Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization Feb 26, 2026	Accuracy, Latency	GAIA, BrowseComp	Automatic Metrics	Not reported
Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE Mar 31, 2026	Ndcg, Cost	Not reported	Automatic Metrics	Not reported
QED-Nano: Teaching a Tiny Model to Prove Hard Theorems Apr 6, 2026	Cost, Inference cost	Not reported	Automatic Metrics	Not reported
CAMEL: Confidence-Gated Reflection for Reward Modeling Feb 24, 2026	Accuracy, Cost	Not reported	Automatic Metrics	Not reported
Distilling Feedback into Memory-as-a-Tool Jan 9, 2026	Cost, Inference cost	Not reported	Automatic Metrics	Not reported
Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation May 8, 2026	Perplexity, Cost	Not reported	Automatic Metrics	Not reported

How To Use This Page

Checklist

Strong: Papers with explicit human feedback

Coverage is strong (46.2% vs 45% target).
Gap: Papers reporting quality controls

Coverage is a replication risk (0% vs 30% target).
Moderate: Papers naming benchmarks/datasets

Coverage is usable but incomplete (30.8% vs 35% target).
Strong: Papers naming evaluation metrics

Coverage is strong (100% vs 35% target).
Gap: Papers with known rater population

Coverage is a replication risk (15.4% vs 35% target).
Strong: Papers with known annotation unit

Coverage is strong (53.8% vs 35% target).

Strengths

Strong human-feedback signal (46.2% of papers).
Agentic evaluation appears in 46.2% of papers.

Known Gaps

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (15.4% coverage).
LLM-as-judge appears without enough inter-annotator agreement reporting.

Suggested Next Analyses

Pair this hub with a human_eval-heavy hub to validate judge-model calibration.
Stratify by benchmark (BrowseComp vs GAIA) before comparing methods.
Track metric sensitivity by reporting both cost and inference cost.

Recommended Queries

LLM-as-Judge Protocols Benchmark Slice: BrowseComp Metric Slice: cost Recent High-Signal Papers

Known Limitations

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (15.4% coverage).
Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Coverage Snapshot

Top Metrics

Cost (13)
Inference cost (13)
Accuracy (5)
Latency (4)

Evaluation Modes

Automatic Metrics (12)
Llm As Judge (1)

Top Benchmarks

BrowseComp (1)
GAIA (1)
GSM8K (1)
HumanEval+ (1)

Agentic Mix

Long Horizon (6)
Web Browsing (1)

Top Papers Reporting This Metric

Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoE
Hejin Huang, Jusheng Zhang, Kaitong Cai, Jian Wang, Rong Pan · Mar 31, 2026 · Citations: 0

Automatic Metrics General

Preference-based alignment objectives have been widely adopted, from RLHF-style pairwise learning in large language models to emerging applications in recommender systems.
QED-Nano: Teaching a Tiny Model to Prove Hard Theorems
LM-Provers, Yuxiao Qu, Amrith Setlur, Jasper Dekoninck, Edward Beeching · Apr 6, 2026 · Citations: 0

Automatic Metrics MathCoding

To support further research on open mathematical reasoning, we release the full QED-Nano pipeline, including the QED-Nano and QED-Nano-SFT models, the FineProofs-SFT and FineProofs-RL datasets, and the training and evaluation code.
CAMEL: Confidence-Gated Reflection for Reward Modeling
Zirui Zhu, Hailun Xu, Yang Luo, Yong Liu, Kanchan Sarkar · Feb 24, 2026 · Citations: 0

Automatic Metrics General

Building on this insight, we propose CAMEL, a confidence-gated reflection framework that performs a lightweight single-token preference decision first and selectively invokes reflection only for low-confidence instances.
Distilling Feedback into Memory-as-a-Tool
Víctor Gallego · Jan 9, 2026 · Citations: 0

Automatic Metrics General

We propose a framework that amortizes the cost of inference-time reasoning by converting transient critiques into retrievable guidelines, through a file-based memory system and agent-controlled tool calls.
S0 Tuning: Zero-Overhead Adaptation of Hybrid Recurrent-Attention Models
Jack Young · Apr 1, 2026 · Citations: 0

Automatic Metrics MathCoding

Using roughly 48 execution-verified HumanEval training solutions, tuning a single initial state matrix per recurrent layer, with zero inference overhead, outperforms LoRA by +10.8 pp (p < 0.001) on HumanEval.
Learning When to Sample: Confidence-Aware Self-Consistency for Efficient LLM Chain-of-Thought Reasoning
Juming Xiong, Kevin Guo, Congning Ni, Chao Yan, Katherine Brown · Mar 9, 2026 · Citations: 0

Automatic Metrics Math

Recent self-consistency-based approaches further improve accuracy but require sampling and aggregating multiple reasoning trajectories, leading to substantial additional computational overhead.
ReasoningGuard: Safeguarding Large Reasoning Models with Inference-time Safety Aha Moments
Yuquan Wang, Mi Zhang, Yining Wang, Geng Hong, Mi Wen · Aug 6, 2025 · Citations: 0

General

It injects timely safety aha moments during the reasoning process to guide the model towards harmless yet helpful reasoning.
GLiGuard: Schema-Conditioned Classification for LLM Safeguard
Urchade Zaratiana, Mary Newhauser, George Hurn-Maloney, Ash Lewis · May 8, 2026 · Citations: 0

Automatic Metrics Coding

Ensuring safe, policy-compliant outputs from large language models requires real-time content moderation that can scale across multiple safety dimensions.
$\texttt{YC-Bench}$: Benchmarking AI Agents for Long-Term Planning and Consistent Execution
Muyu He, Adit Jain, Anand Kumar, Vincent Tu, Soumyadeep Bakshi · Apr 1, 2026 · Citations: 0

Automatic Metrics General

As LLM agents tackle increasingly complex tasks, a critical question is whether they can maintain strategic coherence over long horizons: planning under uncertainty, learning from delayed feedback, and adapting when early mistakes compound.
Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization
Qianben Chen, Tianrui Qin, King Zhu, Qiexiang Wang, Chengjun Yu · Feb 26, 2026 · Citations: 0

Automatic Metrics General

Recent deep research agents primarily improve performance by scaling reasoning depth, but this leads to high inference cost and latency in search-intensive scenarios.
Trajectory as the Teacher: Few-Step Discrete Flow Matching via Energy-Navigated Distillation
Amin Karimi Monsefi, Dominic Culver, Nikhil Bhendawade, Manuel R. Ciosici, Yizhe Zhang · May 8, 2026 · Citations: 0

Automatic Metrics General

Each training trajectory is built through a chain of blind stochastic jumps with no evaluation of sequence quality; a single bad decision at an early midpoint propagates through subsequent steps, yet the student must imitate the result.
Luna-2: Scalable Single-Token Evaluation with Small Language Models
Vatsal Goel, Rishon Dsouza, Nikhil Ega, Amey Ramesh Rambatla, Rob Friel · Feb 20, 2026 · Citations: 0

Llm As JudgeAutomatic Metrics General

We present Luna-2, a novel architecture that leverages decoder-only small language models (SLMs) into a deterministic evaluation model to reliably compute complex task-specific LLMAJ metrics (e.g.
TabAgent: A Framework for Replacing Agentic Generative Components with Tabular-Textual Classifiers
Ido Levy, Eilam Shapira, Yinon Goldshtein, Avi Yaeli, Nir Mashkif · Feb 18, 2026 · Citations: 0

Automatic Metrics General

We propose TabAgent, a framework for replacing generative decision components in closed-set selection tasks with a compact textual-tabular classifier trained on execution traces.
AdaptR1: Reinforcement Learning Based Adaptive Interleaved Thinking in Multi-hop Question Answering
Yuxin Wang, Jiahao Lu, Qifeng Wu, Shicheng Fang, Chuanyuan Tan · May 29, 2026 · Citations: 0
Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies
Yuxuan Ye, Raul Santos-Rodriguez, Edwin Simpson · May 28, 2026 · Citations: 0
Training Deliberative Monitors for Black-Box Scheming Detection
Aditya Sinha, Akshat Naik, Victor Gillioz, Simon Storf, Kilian Merkelbach · May 28, 2026 · Citations: 0
Rethinking Stepwise Model Routing: A Cost-Efficient Table Reasoning Perspective
Shenghao Ye, Yuxiang Wang, Yu Guo, Dong Jin, Shuangwu Chen · May 28, 2026 · Citations: 0
Prompt Compression in Diffusion Large Language Models: Evaluating LLMLingua-2 on LLaDA
Sterling Huang, Abigayle Brown, Jiyoo Noh, Jiakang Xu, Wantong Huo · May 18, 2026 · Citations: 0
FastOCR: Dynamic Visual Fixation via KV Cache Pruning for Efficient Document Parsing
Zihan Tang, Leqi Shen, Hui Chen, Ao Wang, Ben Wan · May 17, 2026 · Citations: 0
When Reviews Disagree: Fine-Grained Contradiction Analysis in Scientific Peer Reviews
Sandeep Kumar, Yash Kamdar, Abid Hossain, Bharti Kumari, Tanik Saikh · May 11, 2026 · Citations: 0
PaT: Planning-after-Trial for Efficient Test-Time Code Generation
Youngsik Yoon, Sungjae Lee, Seockbean Song, Siwei Wang, Wei Chen · May 8, 2026 · Citations: 0
A Few Good Clauses: Comparing LLMs vs Domain-Trained Small Language Models on Structured Contract Extraction
Nicole Lincoln, Nick Whitehouse, Jaron Mar, Rivindu Perera · May 7, 2026 · Citations: 0
ReaComp: Compiling LLM Reasoning into Symbolic Solvers for Efficient Program Synthesis
Atharva Naik, Yash Mathur, Prakam, Carolyn Rose, David Mortensen · May 6, 2026 · Citations: 0
RAG over Thinking Traces Can Improve Reasoning Tasks
Negar Arabzadeh, Wenjie Ma, Sewon Min, Matei Zaharia · May 5, 2026 · Citations: 0
Length Value Model: Scalable Value Pretraining for Token-Level Length Modeling
Zhen Zhang, Changyi Yang, Zijie Xia, Zhen Yang, Chengzhi Liu · Apr 29, 2026 · Citations: 0
Anchor-and-Resume Concession Under Dynamic Pricing for LLM-Augmented Freight Negotiation
Hoang Nguyen, Lu Wang, Marta Gaia Bras · Apr 22, 2026 · Citations: 0
Decoding Text Spans for Efficient and Accurate Named-Entity Recognition
Andrea Maracani, Savas Ozkan, Junyi Zhu, Sinan Mutlu, Mete Ozay · Apr 22, 2026 · Citations: 0
Stochasticity in Tokenisation Improves Robustness
Sophie Steger, Rui Li, Sofiane Ennadir, Anya Sims, Arno Solin · Apr 17, 2026 · Citations: 0
Route to Rome Attack: Directing LLM Routers to Expensive Models via Adversarial Suffix Optimization
Haochun Tang, Yuliang Yan, Jiahua Lu, Huaxiao Liu, Enyan Dai · Apr 16, 2026 · Citations: 0
DharmaOCR: Specialized Small Language Models for Structured OCR that outperform Open-Source and Commercial Baselines
Gabriel Pimenta de Freitas Cardoso, Caio Lucas da Silva Chacon, Jonas Felipe da Fonseca Oliveira, Paulo Henrique de Medeiros Araujo · Apr 15, 2026 · Citations: 0
Robust Explanations for User Trust in Enterprise NLP Systems
Guilin Zhang, Kai Zhao, Jeffrey Friedman, Xu Chu, Amine Anoun · Apr 13, 2026 · Citations: 0
Council Mode: A Heterogeneous Multi-Agent Consensus Framework for Reducing LLM Hallucination and Bias
Shuai Wu, Xue Li, Yanna Feng, Yufang Li, Zhijun Wang · Apr 3, 2026 · Citations: 0
Ensemble Self-Training for Unsupervised Machine Translation
Ido Aharon, Jonathan Shaki, Sarit Kraus · Mar 17, 2026 · Citations: 0
Thinking in Latents: Adaptive Anchor Refinement for Implicit Reasoning in LLMs
Disha Sheshanarayana, Rajat Subhra Pal, Manjira Sinha, Tirthankar Dasgupta · Mar 16, 2026 · Citations: 0
Adaptive Vision-Language Model Routing for Computer Use Agents
Xunzhuo Liu, Bowei He, Xue Liu, Andy Luo, Haichen Zhang · Mar 13, 2026 · Citations: 0
Reliability-Aware Adaptive Self-Consistency for Efficient Sampling in LLM Reasoning
Junseok Kim, Nakyeong Yang, Kyungmin Min, Kyomin Jung · Jan 6, 2026 · Citations: 0
RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards
Zhilin Wang, Jiaqi Zeng, Olivier Delalleau, Ellie Evans, Daniel Egert · Sep 25, 2025 · Citations: 0

Related Metrics And Hubs

Inference Cost Metric Papers CS.CL Human Feedback And Eval Papers Inference Cost In CS.AI Papers Inference Cost + Automatic Metrics Metric Papers CS.CL Papers (Last 90 Days) HFEPX Quarterly Archive: 2026-Q2 Coherence + General Metric Papers (16) Helpfulness In CS.CL Papers (17) Helpfulness Metric Papers (20) Relevance + Automatic Metrics Metric Papers (22) Relevance + General Metric Papers (12) Relevance In CS.CL Papers (68) Relevance In CS.LG Papers (21) Relevance Metric Papers (96) Throughput In CS.LG Papers (26) Inference Cost In CS.AI Papers (18)