Latency Metric Papers

Research Utility Snapshot

Top Papers Reporting This Metric

• LiCQA : A Lightweight Complex Question Answering System

  Sourav Saha, Dwaipayan Roy, Mandar Mitra · Feb 25, 2026

  Automatic Metrics General

  The results of our experiments show that LiCQA significantly outperforms these two state-of-the-art systems on benchmark data with noteworthy reduction in latency.

• SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents

  Patrick Tser Jern Kon, Archana Pradeep, Ang Chen, Alexander P. Ellis, Warren Hunt · Feb 25, 2026

  Automatic Metrics Coding

  Our approach combines supervised fine-tuning on expert-augmented trajectories with agentic reinforcement learning that explicitly discourages degenerative looping and unproductive expert collaboration.

• Both Ends Count! Just How Good are LLM Agents at "Text-to-Big SQL"?

  Germán T. Eizaguirre, Lars Tissen, Marc Sánchez-Artigas · Feb 25, 2026

  Automatic Metrics Multilingual

  Text-to-SQL and Big Data are both extensively benchmarked fields, yet there is limited research that evaluates them jointly.

• Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

  Anas Barakat, Souradip Chakraborty, Khushbu Pahwa, Amrit Singh Bedi · Feb 24, 2026

  Automatic Metrics MathCoding

  Pass@k is a widely used performance metric for verifiable large language model tasks, including mathematical reasoning, code generation, and short-answer reasoning.

• Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning

  Sanket Badhe, Deep Shah · Feb 24, 2026

  Automatic Metrics Law

  These expressive instructions render the decision-making process transparent, allowing for full human verification of logic, making this approach ideal for regulated industries such as law, finance, and content moderation, as well as high-v

• HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders

  Kun Yuan, Junyu Bi, Daixuan Cheng, Changfa Wu, Shuwen Xiao · Feb 24, 2026

  Automatic Metrics Coding

  Modern recommender systems leverage ultra-long user behavior sequences to capture dynamic preferences, but end-to-end modeling is infeasible in production due to latency and memory constraints.

• Generative Pseudo-Labeling for Pre-Ranking with LLMs

  Junyu Bi, Xinting Niu, Daixuan Cheng, Kun Yuan, Tao Wang · Feb 24, 2026

  Automatic Metrics General

  Pre-ranking is a critical stage in industrial recommendation systems, tasked with efficiently scoring thousands of recalled items for downstream ranking.

• HELP: HyperNode Expansion and Logical Path-Guided Evidence Localization for Accurate and Efficient GraphRAG

  Yuqi Huang, Ning Liao, Kai Yang, Anning Hu, Shengchao Hu · Feb 24, 2026

  Automatic Metrics General

  Extensive experiments demonstrate that HELP achieves competitive performance across multiple simple and multi-hop QA benchmarks and up to a 28.8$\times$ speedup over leading Graph-based RAG baselines.

• CHESS: Context-aware Hierarchical Efficient Semantic Selection for Long-Context LLM Inference

  Chao Fei, Guozhong Li, Chenxi Liu, Panos Kalnis · Feb 24, 2026

  Automatic Metrics Coding

  Extensive evaluations demonstrate that CHESS surpasses Full-KV quality using only \textbf{1\%} of the KV cache, delivers low-latency stable inference with up to \textbf{4.56$\times$} higher throughput, and consistently outperforms other str

• Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

  Arindam Khaled · Feb 23, 2026

  Automatic Metrics Math

  In this work, we propose "Pyramid MoA", a hierarchical Mixture-of-Agents architecture that uses a lightweight Router to dynamically escalate queries only when necessary.

• Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations

  Dongming Jiang, Yi Li, Songtao Wei, Jinxin Yang, Ayushi Kishore · Feb 22, 2026

  Automatic Metrics General

  Agentic memory systems enable large language model (LLM) agents to maintain state across long interactions, supporting long-horizon reasoning and personalization beyond fixed context windows.

• Luna-2: Scalable Single-Token Evaluation with Small Language Models

  Vatsal Goel, Rishon Dsouza, Nikhil Ega, Amey Ramesh Rambatla, Rob Friel · Feb 20, 2026

  Automatic Metrics Coding

  Real-time guardrails require evaluation that is accurate, cheap, and fast - yet today's default, LLM-as-a-judge (LLMAJ), is slow, expensive, and operationally non-deterministic due to multi-token generation.

• TabAgent: A Framework for Replacing Agentic Generative Components with Tabular-Textual Classifiers

  Ido Levy, Eilam Shapira, Yinon Goldshtein, Avi Yaeli, Nir Mashkif · Feb 18, 2026

  Automatic Metrics General

  Agentic systems, AI architectures that autonomously execute multi-step workflows to achieve complex goals, are often built using repeated large language model (LLM) calls for closed-set decision tasks such as routing, shortlisting, gating,

• Discrete Stochastic Localization for Non-autoregressive Generation

  Yunshu Wu, Jiayi Cheng, Partha Thakuria, Rob Brekelmans, Evangelos E. Papalexakis · Feb 18, 2026

  Automatic Metrics General

  On OpenWebText, \textsc{DSL} fine-tuning yields large MAUVE gains at low step budgets, surpassing the MDLM+ReMDM baseline with \(\sim\)4$\times$ fewer denoiser evaluations, and matches autoregressive quality at high budgets.

• Overthinking Loops in Agents: A Structural Risk via MCP Tools

  Yohan Lee, Jisoo Jang, Seoyeon Choi, Sangyeop Kim, Seungtaek Choi · Feb 16, 2026

  Automatic Metrics General

  Tool-using LLM agents increasingly coordinate real workloads by selecting and chaining third-party tools based on text-visible metadata such as tool names, descriptions, and return messages.

• Query as Anchor: Scenario-Adaptive User Representation via Large Language Model

  Jiahao Yuan, Yike Xu, Jinyong Wen, Baokun Wang, Ziyi Gao · Feb 16, 2026

  Automatic Metrics Coding

  Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment.

• From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design

  Sha Li, Stefano Petrangeli, Yu Shen, Xiang Chen · Feb 14, 2026

  Simulation Env Coding

  We introduce LaySPA, a reinforcement learning framework that equips large language models (LLMs) with explicit and interpretable spatial reasoning for content-aware graphic layout design.

• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception

  Lai Wei, Liangbo He, Jun Lan, Lingzhong Dong, Yutong Cai · Feb 12, 2026

  Automatic Metrics Coding

  To address this, we propose Region-to-Image Distillation, which transforms zooming from an inference-time tool into a training-time primitive, thereby internalizing the benefits of agentic zooming into a single forward pass of an MLLM.

• Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

  Ailin Huang, Ang Li, Aobo Kong, Bin Wang, Binxing Jiao · Feb 11, 2026

  Simulation Env MathCoding

  We introduce Step 3.5 Flash, a sparse Mixture-of-Experts (MoE) model that bridges frontier-level agentic intelligence and computational efficiency.

• AceGRPO: Adaptive Curriculum Enhanced Group Relative Policy Optimization for Autonomous Machine Learning Engineering

  Yuzhu Cai, Zexi Liu, Xinyu Zhu, Cheng Wang, Siheng Chen · Feb 8, 2026

  Automatic Metrics Coding

  Autonomous Machine Learning Engineering (MLE) requires agents to perform sustained, iterative optimization over long horizons.

• Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts

  Wenhao Li, Daohai Yu, Gen Luo, Yuxin Zhang, Fei Chao · Feb 2, 2026

  Automatic Metrics Coding

  Training Large Language Models (LLMs) on long contexts is severely constrained by prohibitive GPU memory overhead, not training time.

• INSURE-Dial: A Phase-Aware Conversational Dataset & Benchmark for Compliance Verification and Phase Detection

  Shubham Kulkarni, Alexander Lyzhov, Preetam Joshi, Shiva Chaitanya · Jan 28, 2026

  Automatic Metrics Medicine

  We introduce INSURE-Dial, to our knowledge the first public benchmark for developing and assessing compliance-aware voice agents for phase-aware call auditing with span-based compliance verification.

• Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring

  Dongxu Zhang, Yiding Sun, Cheng Tan, Wenbiao Yan, Ning Yang · Jan 20, 2026

  Automatic Metrics General

  While Chain-of-Thought (CoT) reasoning significantly enhances the performance of Multimodal Large Language Models (MLLMs), its autoregressive nature incurs prohibitive latency constraints.

• Fast-ThinkAct: Efficient Vision-Language-Action Reasoning via Verbalizable Latent Planning

  Chi-Pin Huang, Yunze Man, Zhiding Yu, Min-Hung Chen, Jan Kautz · Jan 14, 2026

  Simulation Env General

  Fast-ThinkAct learns to reason efficiently with latent CoTs by distilling from a teacher, driven by a preference-guided objective to align manipulation trajectories that transfers both linguistic and visual planning capabilities for embodie

• Towards Efficient Agents: A Co-Design of Inference Architecture and System

  Weizhe Lin, Hui-Ling Zhen, Shuai Yang, Xian Wang, Renxi Liu · Dec 20, 2025

  Automatic Metrics General

  The rapid development of large language model (LLM)-based agents has unlocked new possibilities for autonomous multi-turn reasoning and tool-augmented decision-making.

• CDLM: Consistency Diffusion Language Models For Faster Sampling

  Minseo Kim, Chenfeng Xu, Coleman Hooper, Harman Singh, Ben Athiwaratkun · Nov 24, 2025

  Automatic Metrics MathCoding

  The full training and evaluation code is available at https://github.com/SqueezeAILab/CDLM.

• OckBench: Measuring the Efficiency of LLM Reasoning

  Zheng Du, Hao Kang, Song Han, Tushar Krishna, Ligeng Zhu · Nov 7, 2025

  Automatic Metrics Coding

  Yet current benchmarks emphasize accuracy and output quality, neglecting a critical dimension: efficiency of token usage.

• FrugalPrompt: Reducing Contextual Overhead in Large Language Models via Token Attribution

  Syed Rifat Raiyan, Md Farhan Ishmam, Abdullah Al Imran, Mohammad Ali Moni · Oct 18, 2025

  Automatic Metrics General

  Human communication heavily relies on laconism and inferential pragmatics, allowing listeners to successfully reconstruct rich meaning from sparse, telegraphic speech.

• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction

  Jing Xiong, Qiujiang Chen, Fanghua Ye, Zhongwei Wan, Chuanyang Zheng · Sep 18, 2025

  Automatic Metrics MathCoding

  Large language models (LLMs) benefit from test-time scaling but are often hampered by high inference latency.

• SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML

  Ismail Lamaakal, Chaymae Yahyati, Khalid El Makkaoui, Ibrahim Ouahbi, Yassine Maleh · Aug 18, 2025

  Automatic Metrics Coding

  Reliable uncertainty estimation is a key missing piece for on-device monitoring in TinyML: microcontrollers must detect failures, distribution shift, or accuracy drops under strict flash/latency budgets, yet common uncertainty approaches (d

• On the Inference (In-)Security of Vertical Federated Learning: Efficient Auditing against Inference Tampering Attack

  Chung-ju Huang, Ziqi Zhang, Yinggui Wang, Binghui Wang, Tao Wei · Jul 3, 2025

  Automatic MetricsSimulation Env General

  Vertical Federated Learning (VFL) is an emerging distributed learning paradigm for cross-silo collaboration without accessing participants' data.

• $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts

  Mert Cemri, Nived Rajaraman, Rishabh Tiwari, Xiaoxuan Liu, Kurt Keutzer · Jun 15, 2025

  Automatic Metrics Math

  Scaling test-time compute has driven the recent advances in the reasoning capabilities of large language models (LLMs), typically by allocating additional computation for more thorough exploration.

• vCache: Verified Semantic Prompt Caching

  Luis Gaspar Schroeder, Aditya Desai, Alejandro Cuadron, Kyle Chu, Shu Liu · Feb 6, 2025

  Automatic Metrics General

  We release the vCache implementation and four benchmarks to support future research.

• Intermittent Semi-Working Mask: A New Masking Paradigm for LLMs

  HaoYuan Hu, Mingcong Lu, Di Luo, XinYa Wu, Jiangcai Zhu · Aug 1, 2024

  Automatic Metrics Math

  Across extensive evaluations, ISM outperforms causal baselines not only on multi-turn dialogue, but also on context-intensive tasks like mathematical reasoning.

Other Metric Hubs