Skip to content
← Back to explorer

Metric Hub

Cost + Math Metric Papers

Updated from current HFEPX corpus (Feb 27, 2026). 12 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Simulation Env. Most common rater population: Domain Experts. Common annotation unit: Trajectory. Frequent quality control: Calibration. Frequently cited benchmark: GSM8K. Common metric signal: cost. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 26, 2026.

Papers: 12 Last published: Feb 26, 2026 Global RSS

Research Narrative

Grounded narrative Model: deterministic-grounded Source: persisted

Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 12 papers for Cost + Math Metric Papers. Dominant protocol signals include automatic metrics, simulation environments, with frequent benchmark focus on GSM8K, MMLU and metric focus on cost, latency. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.

Why This Matters For Eval Research

Protocol Takeaways

Benchmark Interpretation

  • GSM8K appears in 16.7% of hub papers (2/12); use this cohort for benchmark-matched comparisons.
  • MMLU appears in 16.7% of hub papers (2/12); use this cohort for benchmark-matched comparisons.

Metric Interpretation

  • cost is reported in 100% of hub papers (12/12); compare with a secondary metric before ranking methods.
  • latency is reported in 41.7% of hub papers (5/12); compare with a secondary metric before ranking methods.

Abstract Evidence Highlights

Direct snippets from paper abstracts to ground protocol and benchmark interpretation.

Protocol Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

Human-eval abstract signal: Reasoning with large language models often benefits from generating multiple chains-of-thought, but existing aggregation strategies are typically trajectory-level (e.g., selecting the best trace or voting on the final answer), discarding useful intermediate work from...

Protocol Sparsity Induction for Accurate Post-Training Pruning of Large Language Models

Human-eval abstract signal: Large language models have demonstrated capabilities in text generation, while their increasing parameter scales present challenges in computational and memory efficiency.

Benchmark Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

GSM8K benchmark signal: On the GSM8K benchmark, our system achieves 93.0% accuracy, effectively matching the Oracle baseline (98.0%) while reducing compute costs by 61%.

Benchmark Diffusion Language Models Know the Answer Before Decoding

GSM8K benchmark signal: For example, on GSM8K and MMLU, up to 97% and 99% of instances, respectively, can be decoded correctly using only half of the refinement steps.

Metric Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

cost metric signal: Using low-confidence diffusion sampling with parallel, independent rollouts, our training-free framework improves average accuracy by up to 23.8% across six math and coding tasks.

Quality Control Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

rater calibration quality-control signal: By leveraging semantic agreement and confidence calibration among an ensemble of small models, our Router identifies "hard" problems with high precision.

Protocol Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

Protocol abstract signal: Pass@k is a widely used performance metric for verifiable large language model tasks, including mathematical reasoning, code generation, and short-answer reasoning.

Protocol KNIGHT: Knowledge Graph-Driven Multiple-Choice Question Generation with Adaptive Hardness Calibration

Protocol abstract signal: With the rise of large language models (LLMs), they have become instrumental in applications such as Retrieval-Augmented Generation (RAG).

Researcher Checklist

  • Close gap on Papers with explicit human feedback. Coverage is a replication risk (8.3% vs 45% target).
  • Close gap on Papers reporting quality controls. Coverage is a replication risk (16.7% vs 30% target).
  • Tighten coverage on Papers naming benchmarks/datasets. Coverage is usable but incomplete (33.3% vs 35% target).
  • Maintain strength on Papers naming evaluation metrics. Coverage is strong (100% vs 35% target).
  • Close gap on Papers with known rater population. Coverage is a replication risk (8.3% vs 35% target).
  • Tighten coverage on Papers with known annotation unit. Coverage is usable but incomplete (25% vs 35% target).

Papers with explicit human feedback

Coverage is a replication risk (8.3% vs 45% target).

Papers reporting quality controls

Coverage is a replication risk (16.7% vs 30% target).

Papers naming benchmarks/datasets

Coverage is usable but incomplete (33.3% vs 35% target).

Papers naming evaluation metrics

Coverage is strong (100% vs 35% target).

Papers with known rater population

Coverage is a replication risk (8.3% vs 35% target).

Papers with known annotation unit

Coverage is usable but incomplete (25% vs 35% target).

Suggested Reading Order

  1. 1. Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  2. 2. Sparsity Induction for Accurate Post-Training Pruning of Large Language Models

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  3. 3. Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  4. 4. KNIGHT: Knowledge Graph-Driven Multiple-Choice Question Generation with Adaptive Hardness Calibration

    Adds automatic metrics for broader coverage within this hub.

  5. 5. Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

    Adds automatic metrics for broader coverage within this hub.

  6. 6. Do LLMs and VLMs Share Neurons for Inference? Evidence and Mechanisms of Cross-Modal Transfer

    Adds automatic metrics for broader coverage within this hub.

  7. 7. Watermarking LLM Agent Trajectories

    Adds automatic metrics for broader coverage within this hub.

  8. 8. Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

    Adds simulation environments with pairwise preferences for broader coverage within this hub.

Known Limitations

  • Only 16.7% of papers report quality controls; prioritize calibration/adjudication evidence.
  • Rater population is under-specified (8.3% coverage).
  • Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Links

automatic_metrics vs simulation_env

both=0, left_only=10, right_only=2

0 papers use both Automatic Metrics and Simulation Env.

Benchmark Brief

GSM8K

Coverage: 2 papers (16.7%)

2 papers (16.7%) mention GSM8K.

Examples: Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference , Diffusion Language Models Know the Answer Before Decoding

Benchmark Brief

MMLU

Coverage: 2 papers (16.7%)

2 papers (16.7%) mention MMLU.

Examples: KNIGHT: Knowledge Graph-Driven Multiple-Choice Question Generation with Adaptive Hardness Calibration , Diffusion Language Models Know the Answer Before Decoding

Benchmark Brief

BrowseComp

Coverage: 1 papers (8.3%)

1 papers (8.3%) mention BrowseComp.

Examples: Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters

Metric Brief

cost

Coverage: 12 papers (100%)

12 papers (100%) mention cost.

Examples: Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching , Sparsity Induction for Accurate Post-Training Pruning of Large Language Models , Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

Metric Brief

latency

Coverage: 5 papers (41.7%)

5 papers (41.7%) mention latency.

Examples: Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching , Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training , Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

Metric Brief

accuracy

Coverage: 2 papers (16.7%)

2 papers (16.7%) mention accuracy.

Examples: Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching , Pyramid MoA: A Probabilistic Framework for Cost-Optimized Anytime Inference

Most Cited In This Hub

Fast path to methods with the strongest citation traction in this scope.

Papers: Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching , Sparsity Induction for Accurate Post-Training Pruning of Large Language Models , Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

Most Recent

Fast path to latest protocol changes and newly published evaluation setups.

Papers: Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching , Sparsity Induction for Accurate Post-Training Pruning of Large Language Models , Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

Best Protocol Detail

Papers with explicit rater/unit metadata and quality-control signals for reproducibility.

Papers: Test-Time Scaling with Diffusion Language Models via Reward-Guided Stitching , Sparsity Induction for Accurate Post-Training Pruning of Large Language Models , Why Pass@k Optimization Can Degrade Pass@1: Prompt Interference in LLM Post-training

Top Papers Reporting This Metric

Other Metric Hubs

Agreement Metric Papers (25) Agreement In CS.CL Papers (23) Cost Metric Papers (94) Accuracy In CS.LG Papers (82) Cost In CS.CL Papers (81) Accuracy + Coding Metric Papers (73) Accuracy In CS.AI Papers (142) Accuracy + Math Metric Papers (35) Calibration Metric Papers (17) Accuracy In CS.CL Papers (213) Accuracy Metric Papers (245) Cost In CS.AI Papers (53) Recall Metric Papers (36) Recall In CS.CL Papers (29) Accuracy + General Metric Papers (117) Latency Metric Papers (45) F1 Metric Papers (38) F1 In CS.CL Papers (34) Latency In CS.CL Papers (39) Accuracy + Automatic Metrics Metric Papers (245) Cost + Coding Metric Papers (32) Calibration + Automatic Metrics Metric Papers (16) Cost + Automatic Metrics Metric Papers (82) Precision Metric Papers (34)