HFEPX Metric Hub

Throughput In CS.LG Papers

Updated from current HFEPX corpus (Jun 30, 2026). 33 papers are grouped in this metric page.

Read Full Context

Updated from current HFEPX corpus (Jun 30, 2026). 33 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Llm As Judge. Frequently cited benchmark: ATE-Bench. Common metric signal: throughput. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Apr 7, 2026.

Papers: 33 Last published: Apr 7, 2026 Global RSS

When This Metric Page Is Useful

Context-only for now. This page is not strong enough to justify metric decisions on its own. Quality band: Developing .

Metric Coverage

12.1%

4 sampled papers include metric names.

Benchmark Anchoring

6.1%

Papers with explicit dataset/benchmark anchors for fair comparison.

Quality Controls

0.0%

0 papers report calibration/adjudication/IAA controls.

33 papers are not low-signal flagged in this sample.
Use the protocol matrix below to avoid comparing metrics across incompatible eval setups.

Recommended next step: Treat this as directional signal only; metric reporting is present but benchmark anchoring is still thin.

Main limitation: Benchmark coverage is still thin, so avoid treating this page as a definitive guide to the metric.

What This Metric Page Tells You

automatic metrics appears in 12.1% of papers in this hub.
ATE-Bench is a recurring benchmark anchor for cross-paper comparisons in this page.
long-horizon tasks appears in 6.1% of papers, indicating agentic evaluation demand.

Metric Notes (Expanded)

Metric-Driven Protocol Takeaways

Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Pair this hub with a human_eval-heavy hub to validate judge-model calibration.
Stratify by benchmark (ATE-Bench vs GSM8K) before comparing methods.

Metric Interpretation

throughput is reported in 100% of hub papers (33/33); compare with a secondary metric before ranking methods.
accuracy is reported in 39.4% of hub papers (13/33); compare with a secondary metric before ranking methods.

Benchmark Context

ATE-Bench appears in 3% of hub papers (1/33); use this cohort for benchmark-matched comparisons.
GSM8K appears in 3% of hub papers (1/33); use this cohort for benchmark-matched comparisons.

Start Here (Metric-Reliable First 6)

Ranked for metric reporting completeness and comparability.

Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding
Jun 23, 2026 · Citations: 0 · Score: 7.5

Metrics: Accuracy, Latency · Eval: Automatic Metrics
Weakly Supervised Distillation of Hallucination Signals into Transformer Representations
Apr 7, 2026 · Citations: 0 · Score: 7.0

Metrics: F1, Latency · Eval: Llm As Judge, Automatic Metrics
Luna-2: Scalable Single-Token Evaluation with Small Language Models
Feb 20, 2026 · Citations: 0 · Score: 5.5

Metrics: Accuracy, Latency · Eval: Llm As Judge, Automatic Metrics
Learning When to Attend: Conditional Memory Access for Long-Context LLMs
Mar 18, 2026 · Citations: 0 · Score: 5.5

Metrics: Throughput, Context length · Eval: Automatic Metrics
EntMTP: Accelerating LLM Inference with Entropy Guided Multi Token Prediction
Jun 25, 2026 · Citations: 0 · Score: 0.0

Metrics: Not reported · Eval: Not reported
PithTrain: A Compact and Agent-Native MoE Training System
May 29, 2026 · Citations: 0 · Score: 0.0

Metrics: Not reported · Eval: Not reported

Metric Protocol Matrix (Top 10)

Compare metric, benchmark, and evaluation context side by side.

Paper	Metrics	Benchmarks	Eval Modes	Quality Controls
Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding Jun 23, 2026	Accuracy, Latency	LongBench	Automatic Metrics	Not reported
Weakly Supervised Distillation of Hallucination Signals into Transformer Representations Apr 7, 2026	F1, Latency	SQuAD	Llm As Judge, Automatic Metrics	Not reported
Luna-2: Scalable Single-Token Evaluation with Small Language Models Feb 20, 2026	Accuracy, Latency	Not reported	Llm As Judge, Automatic Metrics	Not reported
Learning When to Attend: Conditional Memory Access for Long-Context LLMs Mar 18, 2026	Throughput, Context length	Not reported	Automatic Metrics	Not reported
EntMTP: Accelerating LLM Inference with Entropy Guided Multi Token Prediction Jun 25, 2026	Not reported	Not reported	Not reported	Not reported
PithTrain: A Compact and Agent-Native MoE Training System May 29, 2026	Not reported	Not reported	Not reported	Not reported
SIREM: Speech-Informed MRI Reconstruction with Learned Sampling May 18, 2026	Not reported	Not reported	Not reported	Not reported
PPI-Net connects molecular protein interactions to functional processes in disease May 8, 2026	Not reported	Not reported	Not reported	Not reported
TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning Apr 30, 2026	Not reported	Not reported	Not reported	Not reported
Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding Apr 29, 2026	Not reported	Not reported	Not reported	Not reported

How To Use This Page

Checklist

Gap: Papers with explicit human feedback

Coverage is a replication risk (0% vs 45% target).
Gap: Papers reporting quality controls

Coverage is a replication risk (0% vs 30% target).
Gap: Papers naming benchmarks/datasets

Coverage is a replication risk (18.2% vs 35% target).
Strong: Papers naming evaluation metrics

Coverage is strong (100% vs 35% target).
Gap: Papers with known rater population

Coverage is a replication risk (0% vs 35% target).
Gap: Papers with known annotation unit

Coverage is a replication risk (0% vs 35% target).

Strengths

This hub still surfaces a concentrated paper set for protocol triage and replication planning.

Known Gaps

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (0% coverage).
Annotation unit is under-specified (0% coverage).

Suggested Next Analyses

Pair this hub with a human_eval-heavy hub to validate judge-model calibration.
Stratify by benchmark (ATE-Bench vs GSM8K) before comparing methods.
Track metric sensitivity by reporting both throughput and accuracy.

Recommended Queries

LLM-as-Judge Protocols Benchmark Slice: ATE-Bench Metric Slice: throughput Recent High-Signal Papers

Known Limitations

Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
Rater population is under-specified (0% coverage).
Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Coverage Snapshot

Top Metrics

Throughput (33)
Accuracy (13)
Cost (10)
Latency (9)

Evaluation Modes

Automatic Metrics (4)
Llm As Judge (2)

Top Benchmarks

ATE Bench (1)
GSM8K (1)
HumanEval+ (1)
HumanoidBench (1)

Agentic Mix

Long Horizon (2)

Top Papers Reporting This Metric

Weakly Supervised Distillation of Hallucination Signals into Transformer Representations
Shoaib Sadiq Salehmohamed, Jinal Prashant Thakkar, Hansika Aredla, Shaik Mohammed Omar, Shalmali Ayachit · Apr 7, 2026 · Citations: 0

Llm As JudgeAutomatic Metrics General

We introduce a weak supervision framework that combines three complementary grounding signals: substring matching, sentence embedding similarity, and an LLM as a judge verdict to label generated responses as grounded or hallucinated without…
Dustin: Draft-Augmented Sparse Verification for Efficient Long-Context Generation with Speculative Decoding
WenHung Lee, Jian-Jia Chen, Xiaolin Lin, Pei-Shuo Wang, Chi-Chih Chang · Jun 23, 2026 · Citations: 0

Automatic Metrics General

Evaluations on PG-19 and LongBench with Qwen2.5-72B demonstrate that Dustin achieves a 27.85x speedup in self-attention and a 9.17x end-to-end decoding speedup at a 32k sequence length, all with negligible accuracy degradation.
Luna-2: Scalable Single-Token Evaluation with Small Language Models
Vatsal Goel, Rishon Dsouza, Nikhil Ega, Amey Ramesh Rambatla, Rob Friel · Feb 20, 2026 · Citations: 0

Llm As JudgeAutomatic Metrics General

We present Luna-2, a novel architecture that leverages decoder-only small language models (SLMs) into a deterministic evaluation model to reliably compute complex task-specific LLMAJ metrics (e.g.
Learning When to Attend: Conditional Memory Access for Long-Context LLMs
Sakshi Choudhary, Aditya Chattopadhyay, Luca Zancato, Elvis Nunez, Matthew Trager · Mar 18, 2026 · Citations: 0

Automatic Metrics General

Based on this, we propose L2A (Learning To Attend), a layer that enables conditional (token-wise) long-range memory access by deciding when to invoke global attention.
EntMTP: Accelerating LLM Inference with Entropy Guided Multi Token Prediction
Carrie Chen · Jun 25, 2026 · Citations: 0
PithTrain: A Compact and Agent-Native MoE Training System
Ruihang Lai, Hao Kang, Haozhan Tang, Akaash R. Parthasarathy, Zichun Yu · May 29, 2026 · Citations: 0
SIREM: Speech-Informed MRI Reconstruction with Learned Sampling
Md Hasan, Nyvenn Castro, Daiqi Liu, Lukas Mulzer, Jana Hutter · May 18, 2026 · Citations: 0
PPI-Net connects molecular protein interactions to functional processes in disease
Kyle Higgins, Guadalupe Gonzalez, Dennis Veselkov, Ivan Laponogov, Kirill Veselkov · May 8, 2026 · Citations: 0
TwinGate: Stateful Defense against Decompositional Jailbreaks in Untraceable Traffic via Asymmetric Contrastive Learning
Bowen Sun, Chaozhuo Li, Yaodong Yang, Yiwei Wang, Chaowei Xiao · Apr 30, 2026 · Citations: 0
Accelerating RL Post-Training Rollouts via System-Integrated Speculative Decoding
Hayate Iso, Tiyasa Mitra, Sudipta Mondal, Rasoul Shafipour, Venmugil Elango · Apr 29, 2026 · Citations: 0
Faster LLM Inference via Sequential Monte Carlo
Yahya Emara, Mauricio Barba da Costa, Chi-Chih Chang, Cameron Freer, Tim Vieira · Apr 17, 2026 · Citations: 0
Calibrated Speculative Decoding: Frequency-Guided Candidate Selection for Efficient Inference
Xuwen Zhou, Fangxin Liu, Chao Wang, Xiao Zheng, Hao Zheng · Apr 15, 2026 · Citations: 0
Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning
NVIDIA, :, Aakshita Chandiramani, Aaron Blakeman, Abdullahi Olaoye · Apr 14, 2026 · Citations: 0
Scalable Neural Decoders for Practical Fault-Tolerant Quantum Computation
Andi Gu, J. Pablo Bonilla Ataides, Mikhail D. Lukin, Susanne F. Yelin · Apr 9, 2026 · Citations: 0
GRASS: Gradient-based Adaptive Layer-wise Importance Sampling for Memory-efficient Large Language Model Fine-tuning
Kaiyuan Tian, Yu Tang, Gongqingjian Jiang, Baihui Liu, Yifu Gao · Apr 9, 2026 · Citations: 0
FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control
Donghu Kim, Youngdo Lee, Minho Park, Kinam Kim, I Made Aswin Nahendra · Apr 6, 2026 · Citations: 0
Benchmarking Multi-Agent LLM Architectures for Financial Document Processing: A Comparative Study of Orchestration Patterns, Cost-Accuracy Tradeoffs and Production Scaling Strategies
Siddhant Kulkarni, Yukta Kulkarni · Mar 24, 2026 · Citations: 0
MKA: Memory-Keyed Attention for Efficient Long-Context Reasoning
Dong Liu, Yanxuan Yu, Ben Lengerich, Ying Nian Wu · Mar 21, 2026 · Citations: 0
Cost-Efficient Multimodal LLM Inference via Cross-Tier GPU Heterogeneity
Donglin Yu · Mar 13, 2026 · Citations: 0
FastDSAC: Unlocking the Potential of Maximum Entropy RL in High-Dimensional Humanoid Control
Jun Xue, Junze Wang, Xinming Zhang, Shanze Wang, Yanjun Chen · Mar 13, 2026 · Citations: 0
Proof-Carrying Materials: Falsifiable Safety Certificates for Machine-Learned Interatomic Potentials
Abhinaba Basu, Pavan Chakraborty · Mar 12, 2026 · Citations: 0
Automatic Generation of High-Performance RL Environments
Seth Karten, Rahul Dev Appapogu, Chi Jin · Mar 12, 2026 · Citations: 0
Slow-Fast Inference: Training-Free Inference Acceleration via Within-Sentence Support Stability
Xingyu Xie, Zhaochen Yu, Yue Liao, Tao Wang, Kim-Chuan Toh · Mar 12, 2026 · Citations: 0
Balancing Coverage and Draft Latency in Vocabulary Trimming for Faster Speculative Decoding
Ofir Ben Shoham · Mar 5, 2026 · Citations: 0
Data Driven Optimization of GPU efficiency for Distributed LLM Adapter Serving
Ferran Agullo, Joan Oliveras, Chen Wang, Alberto Gutierrez-Torre, Olivier Tardieu · Feb 27, 2026 · Citations: 0
Accordion-Thinking: Self-Regulated Step Summaries for Efficient and Readable LLM Reasoning
Zhicheng Yang, Zhijiang Guo, Yinya Huang, Yongxin Wang, Wenlei Shi · Feb 3, 2026 · Citations: 0
Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speed
Yonggan Fu, Lexington Whalen, Zhifan Ye, Xin Dong, Shizhe Diao · Dec 16, 2025 · Citations: 0
Toward Closed-loop Molecular Discovery via Language Model, Property Alignment and Strategic Search
Junkai Ji, Zhangfan Yang, Dong Xu, Ruibin Bai, Jianqiang Li · Dec 10, 2025 · Citations: 0
Periodic Asynchrony: An On-Policy Approach for Accelerating LLM Reinforcement Learning
Jian Lu · Nov 24, 2025 · Citations: 0
AccelOpt: A Self-Improving LLM Agentic System for AI Accelerator Kernel Optimization
Genghan Zhang, Shaowei Zhu, Anjiang Wei, Zhenyu Song, Allen Nie · Nov 19, 2025 · Citations: 0
The Polar Express: Optimal Matrix Sign Methods and Their Application to the Muon Algorithm
Noah Amsel, David Persson, Christopher Musco, Robert M. Gower · May 22, 2025 · Citations: 0
Unicorn: A Universal and Collaborative Reinforcement Learning Approach Towards Generalizable Network-Wide Traffic Signal Control
Yifeng Zhang, Yilin Liu, Ping Gong, Peizhuo Li, Mingfeng Fan · Mar 14, 2025 · Citations: 0
BatchLLM: Optimizing Large Batched LLM Inference with Global Prefix Sharing and Throughput-oriented Token Batching
Zhen Zheng, Xin Ji, Taosong Fang, Fanghao Zhou, Chuanjie Liu · Nov 29, 2024 · Citations: 0

Related Metrics And Hubs

Throughput Metric Papers CS.LG Human Feedback And Eval Papers Throughput In CS.AI Papers Throughput In CS.CL Papers CS.AI Human Feedback And Eval Papers CS.CL Human Feedback And Eval Papers Coherence + General Metric Papers (16) Helpfulness In CS.CL Papers (17) Helpfulness Metric Papers (20) Relevance + Automatic Metrics Metric Papers (22) Relevance + General Metric Papers (12) Relevance In CS.CL Papers (68) Relevance In CS.LG Papers (21) Relevance Metric Papers (96) Inference Cost In CS.AI Papers (18) Throughput In CS.CL Papers (35)