Skip to content
← Back to explorer

Metric Hub

Latency + Math Metric Papers

Updated from current HFEPX corpus (Feb 27, 2026). 10 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Simulation Env. Most common rater population: Domain Experts. Common annotation unit: Trajectory. Frequent quality control: Calibration. Frequently cited benchmark: GSM8K. Common metric signal: latency. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 26, 2026.

Papers: 10 Last published: Feb 26, 2026 Global RSS

Research Narrative

Grounded narrative Model: deterministic-grounded Source: persisted

Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 10 papers for Latency + Math Metric Papers. Dominant protocol signals include automatic metrics, simulation environments, with frequent benchmark focus on GSM8K, MATH and metric focus on latency, accuracy. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.

Why This Matters For Eval Research

Protocol Takeaways

Benchmark Interpretation

  • GSM8K appears in 20% of hub papers (2/10); use this cohort for benchmark-matched comparisons.
  • MATH appears in 20% of hub papers (2/10); use this cohort for benchmark-matched comparisons.

Metric Interpretation

  • latency is reported in 100% of hub papers (10/10); compare with a secondary metric before ranking methods.
  • accuracy is reported in 60% of hub papers (6/10); compare with a secondary metric before ranking methods.

Abstract Evidence Highlights

Direct snippets from paper abstracts to ground protocol and benchmark interpretation.

Protocol Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?

Human-eval abstract signal: Diffusion Language Models (DLMs) are often advertised as enabling parallel token generation, yet practical fast DLMs frequently converge to left-to-right, autoregressive (AR)-like decoding dynamics.

Protocol InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models

Human-eval abstract signal: Reducing the hardware footprint of large language models (LLMs) during decoding is critical for efficient long-sequence generation.

Benchmark InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models

GSM8K benchmark signal: Our evaluation experiments on Llama models shows that InnerQ maintains a few-shot GSM8K performance comparable to non-quantized KV caches and surpasses prior KV cache quantization methods.

Benchmark Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

GSM8K benchmark signal: On the GSM8K benchmark, our system achieves 93.0% accuracy, effectively matching the Oracle baseline (98.0%) while reducing compute costs by 61%.

Metric InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models

latency metric signal: We introduce InnerQ, a hardware-aware KV-cache quantization scheme that lowers decode latency without sacrificing accuracy.

Quality Control Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

rater calibration quality-control signal: By leveraging semantic agreement and confidence calibration among an ensemble of small models, our Router identifies "hard" problems with high precision.

Quality Control ATTS: Asynchronous Test-Time Scaling via Conformal Prediction

rater calibration quality-control signal: It enables asynchronous inference through online calibration and proposes an ordinal classification algorithm that supports a three-stage rejection sampling pipeline, scaling along both the sequential and parallel axes.

Protocol Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

Protocol abstract signal: Reasoning with large language models often benefits from generating multiple chains-of-thought, but existing aggregation strategies are typically trajectory-level (e.g., selecting the best trace or voting on the final answer), discarding useful intermediate work from...

Researcher Checklist

  • Close gap on Papers with explicit human feedback. Coverage is a replication risk (10% vs 45% target).
  • Tighten coverage on Papers reporting quality controls. Coverage is usable but incomplete (20% vs 30% target).
  • Maintain strength on Papers naming benchmarks/datasets. Coverage is strong (60% vs 35% target).
  • Maintain strength on Papers naming evaluation metrics. Coverage is strong (100% vs 35% target).
  • Close gap on Papers with known rater population. Coverage is a replication risk (10% vs 35% target).
  • Close gap on Papers with known annotation unit. Coverage is a replication risk (10% vs 35% target).

Papers with explicit human feedback

Coverage is a replication risk (10% vs 45% target).

Papers reporting quality controls

Coverage is usable but incomplete (20% vs 30% target).

Papers naming benchmarks/datasets

Coverage is strong (60% vs 35% target).

Papers naming evaluation metrics

Coverage is strong (100% vs 35% target).

Papers with known rater population

Coverage is a replication risk (10% vs 35% target).

Papers with known annotation unit

Coverage is a replication risk (10% vs 35% target).

Suggested Reading Order

  1. 1. Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding?

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  2. 2. InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  3. 3. Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  4. 4. Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

    Adds automatic metrics for broader coverage within this hub.

  5. 5. Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

    Adds automatic metrics for broader coverage within this hub.

  6. 6. Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

    Adds simulation environments with pairwise preferences for broader coverage within this hub.

  7. 7. CDLM: Consistency Diffusion Language Models For Faster Sampling

    Adds automatic metrics for broader coverage within this hub.

  8. 8. ATTS: Asynchronous Test-Time Scaling via Conformal Prediction

    Adds automatic metrics for broader coverage within this hub.

Known Limitations

  • Rater population is under-specified (10% coverage).
  • Annotation unit is under-specified (10% coverage).
  • Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Links

automatic_metrics vs simulation_env

both=0, left_only=9, right_only=1

0 papers use both Automatic Metrics and Simulation Env.

Benchmark Brief

GSM8K

Coverage: 2 papers (20%)

2 papers (20%) mention GSM8K.

Examples: InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models , Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

Benchmark Brief

MATH

Coverage: 2 papers (20%)

2 papers (20%) mention MATH.

Examples: Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters , CDLM: Consistency Diffusion Language Models For Faster Sampling

Benchmark Brief

AIME

Coverage: 1 papers (10%)

1 papers (10%) mention AIME.

Examples: ATTS: Asynchronous Test-Time Scaling via Conformal Prediction

Metric Brief

latency

Coverage: 10 papers (100%)

10 papers (100%) mention latency.

Examples: Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding? , InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models , Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

Metric Brief

accuracy

Coverage: 6 papers (60%)

6 papers (60%) mention accuracy.

Examples: InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models , Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching , Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

Metric Brief

cost

Coverage: 5 papers (50%)

5 papers (50%) mention cost.

Examples: Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching , Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training , Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

Most Cited In This Hub

Fast path to methods with the strongest citation traction in this scope.

Papers: Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding? , InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models , Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

Most Recent

Fast path to latest protocol changes and newly published evaluation setups.

Papers: Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding? , InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models , Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

Best Protocol Detail

Papers with explicit rater/unit metadata and quality-control signals for reproducibility.

Papers: Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding? , InnerQ: Hardware-aware Tuning-free Quantization of KV Cache for Large Language Models , Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

Top Papers Reporting This Metric

Other Metric Hubs

Agreement Metric Papers (25) Agreement In CS.CL Papers (23) Cost Metric Papers (94) Accuracy In CS.LG Papers (82) Cost In CS.CL Papers (81) Accuracy + Coding Metric Papers (73) Accuracy In CS.AI Papers (142) Accuracy + Math Metric Papers (35) Calibration Metric Papers (17) Accuracy In CS.CL Papers (213) Accuracy Metric Papers (245) Cost In CS.AI Papers (53) Recall Metric Papers (36) Recall In CS.CL Papers (29) Accuracy + General Metric Papers (117) Latency Metric Papers (45) F1 Metric Papers (38) F1 In CS.CL Papers (34) Latency In CS.CL Papers (39) Accuracy + Automatic Metrics Metric Papers (245) Cost + Coding Metric Papers (32) Calibration + Automatic Metrics Metric Papers (16) Cost + Automatic Metrics Metric Papers (82) Precision Metric Papers (34)