Latency In CS.CL Papers

Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 39 papers for Latency In CS.CL Papers. Dominant protocol signals include automatic metrics, simulation environments, with frequent benchmark focus on BrowseComp, GSM8K and metric focus on latency, accuracy. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.

Top Papers Reporting This Metric

• Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems

  Siyuan Liu, Jiahui Xu, Feng Jiang, Kuang Wang, Zefeng Zhao · Feb 26, 2026

  Automatic Metrics General

  Achieving human-like responsiveness is a critical yet challenging goal for cascaded spoken dialogue systems.

• Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?

  Pengxiang Li, Dilxat Muhtar, Lu Yin, Tianlong Chen, Shiwei Liu · Feb 26, 2026

  Automatic Metrics MathMedicine

  Across math reasoning benchmarks, NAP yields stronger performance under parallel decoding than DLMs trained on standard long CoT data, with gains growing as parallelism increases.

• InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models

  Sayed Mohammadreza Tayaranian Hosseini, Amir Ardakani, Warren J. Gross · Feb 26, 2026

  Automatic Metrics MathCoding

  Our evaluation experiments on Llama models shows that InnerQ maintains a few-shot GSM8K performance comparable to non-quantized KV caches and surpasses prior KV cache quantization methods.

• Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

  Roy Miles, Aysim Toker, Andreea-Maria Oncescu, Songcen Xu, Jiankang Deng · Feb 26, 2026

  Automatic Metrics MathCoding

  This modular pipeline separates exploration (diffusion) from evaluation and solution synthesis, avoiding monolithic unified hybrids while preserving broad search.

• Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization

  Qianben Chen, Tianrui Qin, King Zhu, Qiexiang Wang, Chengjun Yu · Feb 26, 2026

  Automatic Metrics General

  Recent deep research agents primarily improve performance by scaling reasoning depth, but this leads to high inference cost and latency in search-intensive scenarios.

• Vectorizing the Trie: Efficient Constrained Decoding for LLM-based Generative Retrieval on Accelerators

  Zhengyang Su, Isay Katsman, Yueqi Wang, Ruining He, Lukasz Heldt · Feb 26, 2026

  Automatic Metrics Coding

  In addition, evaluation on academic benchmarks demonstrates that STATIC can considerably improve cold-start performance for generative retrieval.

• Ruyi2 Technical Report

  Huan Song, Shuyu Tian, Junyi Hao, Minxiu Xu, Hongjun An · Feb 26, 2026

  Automatic Metrics General

  Large Language Models (LLMs) face significant challenges regarding deployment costs and latency, necessitating adaptive computing strategies.

• LiCQA : A Lightweight Complex Question Answering System

  Sourav Saha, Dwaipayan Roy, Mandar Mitra · Feb 25, 2026

  Automatic Metrics General

  The results of our experiments show that LiCQA significantly outperforms these two state-of-the-art systems on benchmark data with noteworthy reduction in latency.

• SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents

  Patrick Tser Jern Kon, Archana Pradeep, Ang Chen, Alexander P. Ellis, Warren Hunt · Feb 25, 2026

  Automatic Metrics Coding

  Our approach combines supervised fine-tuning on expert-augmented trajectories with agentic reinforcement learning that explicitly discourages degenerative looping and unproductive expert collaboration.

• Both Ends Count! Just How Good are LLM Agents at "Text-to-Big SQL"?

  Germán T. Eizaguirre, Lars Tissen, Marc Sánchez-Artigas · Feb 25, 2026

  Automatic Metrics Multilingual

  Text-to-SQL and Big Data are both extensively benchmarked fields, yet there is limited research that evaluates them jointly.

• Prompt-Level Distillation: A Non-Parametric Alternative to Model Fine-Tuning for Efficient Reasoning

  Sanket Badhe, Deep Shah · Feb 24, 2026

  Automatic Metrics Law

  These expressive instructions render the decision-making process transparent, allowing for full human verification of logic, making this approach ideal for regulated industries such as law, finance, and content moderation, as well as high-v

• HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders

  Kun Yuan, Junyu Bi, Daixuan Cheng, Changfa Wu, Shuwen Xiao · Feb 24, 2026

  Automatic Metrics Coding

  Modern recommender systems leverage ultra-long user behavior sequences to capture dynamic preferences, but end-to-end modeling is infeasible in production due to latency and memory constraints.

• Generative Pseudo-Labeling for Pre-Ranking with LLMs

  Junyu Bi, Xinting Niu, Daixuan Cheng, Kun Yuan, Tao Wang · Feb 24, 2026

  Automatic Metrics General

  Pre-ranking is a critical stage in industrial recommendation systems, tasked with efficiently scoring thousands of recalled items for downstream ranking.

• Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

  Arindam Khaled · Feb 23, 2026

  Automatic Metrics Math

  In this work, we propose "Pyramid MoA", a hierarchical Mixture-of-Agents architecture that uses a lightweight Router to dynamically escalate queries only when necessary.

• Anatomy of Agentic Memory: Taxonomy and Empirical Analysis of Evaluation and System Limitations

  Dongming Jiang, Yi Li, Songtao Wei, Jinxin Yang, Ayushi Kishore · Feb 22, 2026

  Automatic Metrics General

  Agentic memory systems enable large language model (LLM) agents to maintain state across long interactions, supporting long-horizon reasoning and personalization beyond fixed context windows.

• Luna-2: Scalable Single-Token Evaluation with Small Language Models

  Vatsal Goel, Rishon Dsouza, Nikhil Ega, Amey Ramesh Rambatla, Rob Friel · Feb 20, 2026

  Automatic Metrics Coding

  Real-time guardrails require evaluation that is accurate, cheap, and fast - yet today's default, LLM-as-a-judge (LLMAJ), is slow, expensive, and operationally non-deterministic due to multi-token generation.

• TabAgent: A Framework for Replacing Agentic Generative Components with Tabular-Textual Classifiers

  Ido Levy, Eilam Shapira, Yinon Goldshtein, Avi Yaeli, Nir Mashkif · Feb 18, 2026

  Automatic Metrics General

  Agentic systems, AI architectures that autonomously execute multi-step workflows to achieve complex goals, are often built using repeated large language model (LLM) calls for closed-set decision tasks such as routing, shortlisting, gating,

• Discrete Stochastic Localization for Non-autoregressive Generation

  Yunshu Wu, Jiayi Cheng, Partha Thakuria, Rob Brekelmans, Evangelos E. Papalexakis · Feb 18, 2026

  Automatic Metrics General

  On OpenWebText, \textsc{DSL} fine-tuning yields large MAUVE gains at low step budgets, surpassing the MDLM+ReMDM baseline with \(\sim\)4$\times$ fewer denoiser evaluations, and matches autoregressive quality at high budgets.

• Overthinking Loops in Agents: A Structural Risk via MCP Tools

  Yohan Lee, Jisoo Jang, Seoyeon Choi, Sangyeop Kim, Seungtaek Choi · Feb 16, 2026

  Automatic Metrics General

  Tool-using LLM agents increasingly coordinate real workloads by selecting and chaining third-party tools based on text-visible metadata such as tool names, descriptions, and return messages.

• Query as Anchor: Scenario-Adaptive User Representation via Large Language Model

  Jiahao Yuan, Yike Xu, Jinyong Wen, Baokun Wang, Ziyi Gao · Feb 16, 2026

  Automatic Metrics Coding

  Evaluations on 10 Alipay industrial benchmarks show consistent SOTA performance, strong scalability, and efficient deployment.

• From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design

  Sha Li, Stefano Petrangeli, Yu Shen, Xiang Chen · Feb 14, 2026

  Simulation Env Coding

  We introduce LaySPA, a reinforcement learning framework that equips large language models (LLMs) with explicit and interpretable spatial reasoning for content-aware graphic layout design.

• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception

  Lai Wei, Liangbo He, Jun Lan, Lingzhong Dong, Yutong Cai · Feb 12, 2026

  Automatic Metrics Coding

  To address this, we propose Region-to-Image Distillation, which transforms zooming from an inference-time tool into a training-time primitive, thereby internalizing the benefits of agentic zooming into a single forward pass of an MLLM.

• Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

  Ailin Huang, Ang Li, Aobo Kong, Bin Wang, Binxing Jiao · Feb 11, 2026

  Simulation Env MathCoding

  We introduce Step 3.5 Flash, a sparse Mixture-of-Experts (MoE) model that bridges frontier-level agentic intelligence and computational efficiency.

• Out of the Memory Barrier: A Highly Memory Efficient Training System for LLMs with Million-Token Contexts

  Wenhao Li, Daohai Yu, Gen Luo, Yuxin Zhang, Fei Chao · Feb 2, 2026

  Automatic Metrics Coding

  Training Large Language Models (LLMs) on long contexts is severely constrained by prohibitive GPU memory overhead, not training time.

• INSURE-Dial: A Phase-Aware Conversational Dataset & Benchmark for Compliance Verification and Phase Detection

  Shubham Kulkarni, Alexander Lyzhov, Preetam Joshi, Shiva Chaitanya · Jan 28, 2026

  Automatic Metrics Medicine

  We introduce INSURE-Dial, to our knowledge the first public benchmark for developing and assessing compliance-aware voice agents for phase-aware call auditing with span-based compliance verification.

• Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring

  Dongxu Zhang, Yiding Sun, Cheng Tan, Wenbiao Yan, Ning Yang · Jan 20, 2026

  Automatic Metrics General

  While Chain-of-Thought (CoT) reasoning significantly enhances the performance of Multimodal Large Language Models (MLLMs), its autoregressive nature incurs prohibitive latency constraints.

• Towards Efficient Agents: A Co-Design of Inference Architecture and System

  Weizhe Lin, Hui-Ling Zhen, Shuai Yang, Xian Wang, Renxi Liu · Dec 20, 2025

  Automatic Metrics General

  The rapid development of large language model (LLM)-based agents has unlocked new possibilities for autonomous multi-turn reasoning and tool-augmented decision-making.

• CDLM: Consistency Diffusion Language Models For Faster Sampling

  Minseo Kim, Chenfeng Xu, Coleman Hooper, Harman Singh, Ben Athiwaratkun · Nov 24, 2025

  Automatic Metrics MathCoding

  The full training and evaluation code is available at https://github.com/SqueezeAILab/CDLM.

• Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

  Jon Saad-Falcon, Avanika Narayan, Hakki Orhun Akengin, J. Wes Griffin, Herumb Shandilya · Nov 11, 2025

  Automatic Metrics General

  Large language model (LLM) queries are predominantly processed by frontier models in centralized cloud infrastructure.

• OckBench: Measuring the Efficiency of LLM Reasoning

  Zheng Du, Hao Kang, Song Han, Tushar Krishna, Ligeng Zhu · Nov 7, 2025

  Automatic Metrics Coding

  Yet current benchmarks emphasize accuracy and output quality, neglecting a critical dimension: efficiency of token usage.

• Inference-Cost-Aware Dynamic Tree Construction for Efficient Inference in Large Language Models

  Yinrong Hong, Zhiquan Tan, Kai Hu · Oct 30, 2025

  Automatic Metrics Coding

  Large Language Models (LLMs) face significant inference latency challenges stemming from their autoregressive design and large size.

• RELOOP: Recursive Retrieval with Multi-Hop Reasoner and Planners for Heterogeneous QA

  Ruiyi Yang, Hao Xue, Imran Razzak, Hakim Hacid, Flora D. Salim · Oct 23, 2025

  Automatic Metrics General

  A Head Agent provides guidance that leads retrieval, while an Iteration Agent selects and expands HSeq via structure-respecting actions (e.g., parent/child hops, table row/column neighbors, KG relations); Finally the head agent composes can

• FrugalPrompt: Reducing Contextual Overhead in Large Language Models via Token Attribution

  Syed Rifat Raiyan, Md Farhan Ishmam, Abdullah Al Imran, Mohammad Ali Moni · Oct 18, 2025

  Automatic Metrics General

  Human communication heavily relies on laconism and inferential pragmatics, allowing listeners to successfully reconstruct rich meaning from sparse, telegraphic speech.

• ATTS: Asynchronous Test-Time Scaling via Conformal Prediction

  Jing Xiong, Qiujiang Chen, Fanghua Ye, Zhongwei Wan, Chuanyang Zheng · Sep 18, 2025

  Automatic Metrics MathCoding

  Large language models (LLMs) benefit from test-time scaling but are often hampered by high inference latency.

• SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML

  Ismail Lamaakal, Chaymae Yahyati, Khalid El Makkaoui, Ibrahim Ouahbi, Yassine Maleh · Aug 18, 2025

  Automatic Metrics Coding

  Reliable uncertainty estimation is a key missing piece for on-device monitoring in TinyML: microcontrollers must detect failures, distribution shift, or accuracy drops under strict flash/latency budgets, yet common uncertainty approaches (d

• $\texttt{SPECS}$: Faster Test-Time Scaling through Speculative Drafts

  Mert Cemri, Nived Rajaraman, Rishabh Tiwari, Xiaoxuan Liu, Kurt Keutzer · Jun 15, 2025

  Automatic Metrics Math

  Scaling test-time compute has driven the recent advances in the reasoning capabilities of large language models (LLMs), typically by allocating additional computation for more thorough exploration.

• vCache: Verified Semantic Prompt Caching

  Luis Gaspar Schroeder, Aditya Desai, Alejandro Cuadron, Kyle Chu, Shu Liu · Feb 6, 2025

  Automatic Metrics General

  We release the vCache implementation and four benchmarks to support future research.

• Intermittent Semi-Working Mask: A New Masking Paradigm for LLMs

  HaoYuan Hu, Mingcong Lu, Di Luo, XinYa Wu, Jiangcai Zhu · Aug 1, 2024

  Automatic Metrics Math

  Across extensive evaluations, ISM outperforms causal baselines not only on multi-turn dialogue, but also on context-intensive tasks like mathematical reasoning.

• Not All Attention is Needed: Parameter and Computation Efficient Transfer Learning for Multi-modal Large Language Models

  Qiong Wu, Weihao Ye, Yiyi Zhou, Xiaoshuai Sun, Rongrong Ji · Mar 22, 2024

  Automatic Metrics General

  To validate EAS, we apply it to a recently proposed MLLM called LaVIN and a classic VL pre-trained model called METER, and conduct extensive experiments on a set of benchmarks.

Other Metric Hubs