Skip to content
← Back to explorer

Metric Hub

Coherence + Automatic Metrics Metric Papers

Updated from current HFEPX corpus (Feb 27, 2026). 15 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Llm As Judge. Most common rater population: Domain Experts. Common annotation unit: Multi Dim Rubric. Frequently cited benchmark: Retrieval. Common metric signal: coherence. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 26, 2026.

Papers: 15 Last published: Feb 26, 2026 Global RSS

Research Narrative

Grounded narrative Model: deterministic-grounded Source: persisted

Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 15 papers for Coherence + Automatic Metrics Metric Papers. Dominant protocol signals include automatic metrics, LLM-as-judge, with frequent benchmark focus on Retrieval, LongBench and metric focus on coherence, accuracy. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.

Why This Matters For Eval Research

Protocol Takeaways

Benchmark Interpretation

  • Retrieval appears in 53.3% of hub papers (8/15); use this cohort for benchmark-matched comparisons.
  • LongBench appears in 13.3% of hub papers (2/15); use this cohort for benchmark-matched comparisons.

Metric Interpretation

  • coherence is reported in 100% of hub papers (15/15); compare with a secondary metric before ranking methods.
  • accuracy is reported in 20% of hub papers (3/15); compare with a secondary metric before ranking methods.

Researcher Checklist

  • Close gap on Papers with explicit human feedback. Coverage is a replication risk (13.3% vs 45% target).
  • Close gap on Papers reporting quality controls. Coverage is a replication risk (0% vs 30% target).
  • Maintain strength on Papers naming benchmarks/datasets. Coverage is strong (66.7% vs 35% target).
  • Maintain strength on Papers naming evaluation metrics. Coverage is strong (100% vs 35% target).
  • Close gap on Papers with known rater population. Coverage is a replication risk (6.7% vs 35% target).
  • Close gap on Papers with known annotation unit. Coverage is a replication risk (20% vs 35% target).

Papers with explicit human feedback

Coverage is a replication risk (13.3% vs 45% target).

Papers reporting quality controls

Coverage is a replication risk (0% vs 30% target).

Papers naming benchmarks/datasets

Coverage is strong (66.7% vs 35% target).

Papers naming evaluation metrics

Coverage is strong (100% vs 35% target).

Papers with known rater population

Coverage is a replication risk (6.7% vs 35% target).

Papers with known annotation unit

Coverage is a replication risk (20% vs 35% target).

Suggested Reading Order

  1. 1. Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  2. 2. Stop-Think-AutoRegress: Language Modeling with Latent Diffusion Planning

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  3. 3. KGHaluBench: A Knowledge Graph-Based Hallucination Benchmark for Evaluating the Breadth and Depth of LLM Knowledge

    Start here for detailed protocol reporting, including rater and quality-control evidence.

  4. 4. Facet-Level Persona Control by Trait-Activated Routing with Contrastive SAE for Role-Playing LLMs

    Adds automatic metrics for broader coverage within this hub.

  5. 5. Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering

    Adds automatic metrics for broader coverage within this hub.

  6. 6. Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions

    Adds automatic metrics for broader coverage within this hub.

  7. 7. AIDG: Evaluating Asymmetry Between Information Extraction and Containment in Multi-Turn Dialogue

    Adds automatic metrics for broader coverage within this hub.

  8. 8. Reinforced Fast Weights with Next-Sequence Prediction

    Adds automatic metrics for broader coverage within this hub.

Known Limitations

  • Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
  • Rater population is under-specified (6.7% coverage).
  • Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Links

llm_as_judge vs automatic_metrics

both=1, left_only=0, right_only=14

1 papers use both Llm As Judge and Automatic Metrics.

Benchmark Brief

LongBench

Coverage: 2 papers (13.3%)

2 papers (13.3%) mention LongBench.

Examples: Reinforced Fast Weights with Next-Sequence Prediction , Document Reconstruction Unlocks Scalable Long-Context RLVR

Benchmark Brief

Financebench

Coverage: 1 papers (6.7%)

1 papers (6.7%) mention Financebench.

Examples: Decomposing Retrieval Failures in RAG for Long-Document Financial Question Answering

Top Papers Reporting This Metric

Other Metric Hubs