Metric Hub
Updated from current HFEPX corpus (Feb 27, 2026). 10 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics. Common annotation unit: Pairwise. Frequent quality control: Calibration. Common metric signal: calibration. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 22, 2026.
Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 10 papers for Calibration + General Metric Papers. Dominant protocol signals include automatic metrics, with frequent benchmark focus on multiple benchmark families and metric focus on calibration, accuracy. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.
10% of papers report explicit human-feedback signals, led by pairwise preferences.
Evidence: Who can we trust? LLM-as-a-jury for Comparative Assessment , Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Discrete Stochastic Localization for Non-autoregressive Generation , PMG: Parameterized Motion Generator for Human-like Locomotion Control
automatic metrics appears in 100% of papers in this hub.
Evidence: Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Who can we trust? LLM-as-a-jury for Comparative Assessment , Discrete Stochastic Localization for Non-autoregressive Generation , PMG: Parameterized Motion Generator for Human-like Locomotion Control
long-horizon tasks appears in 10% of papers, indicating agentic evaluation demand.
Evidence: PMG: Parameterized Motion Generator for Human-like Locomotion Control , Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Who can we trust? LLM-as-a-jury for Comparative Assessment , Discrete Stochastic Localization for Non-autoregressive Generation
Most common quality-control signal is rater calibration (100% of papers).
Evidence: Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Who can we trust? LLM-as-a-jury for Comparative Assessment , Discrete Stochastic Localization for Non-autoregressive Generation , PMG: Parameterized Motion Generator for Human-like Locomotion Control
Rater context is mostly unspecified rater pools, and annotation is commonly pairwise annotation; use this to scope replication staffing.
Evidence: Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Who can we trust? LLM-as-a-jury for Comparative Assessment , Discrete Stochastic Localization for Non-autoregressive Generation , PMG: Parameterized Motion Generator for Human-like Locomotion Control
Track metric sensitivity by reporting both calibration and accuracy.
Evidence: Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Who can we trust? LLM-as-a-jury for Comparative Assessment , Discrete Stochastic Localization for Non-autoregressive Generation , PMG: Parameterized Motion Generator for Human-like Locomotion Control
Papers with explicit human feedback
Coverage is a replication risk (10% vs 45% target).
Papers reporting quality controls
Coverage is strong (100% vs 30% target).
Papers naming benchmarks/datasets
Coverage is a replication risk (0% vs 35% target).
Papers naming evaluation metrics
Coverage is strong (100% vs 35% target).
Papers with known rater population
Coverage is a replication risk (0% vs 35% target).
Papers with known annotation unit
Coverage is a replication risk (20% vs 35% target).
Start here for detailed protocol reporting, including rater and quality-control evidence.
2. Who can we trust? LLM-as-a-jury for Comparative Assessment
Start here for detailed protocol reporting, including rater and quality-control evidence.
3. Discrete Stochastic Localization for Non-autoregressive Generation
Start here for detailed protocol reporting, including rater and quality-control evidence.
4. PMG: Parameterized Motion Generator for Human-like Locomotion Control
Adds automatic metrics for broader coverage within this hub.
5. WISE: Web Information Satire and Fakeness Evaluation
Adds automatic metrics for broader coverage within this hub.
Adds automatic metrics for broader coverage within this hub.
7. LogiPart: Local Large Language Models for Data Exploration at Scale with Logical Partitioning
Adds automatic metrics for broader coverage within this hub.
8. CoSpaDi: Compressing LLMs via Calibration-Guided Sparse Dictionary Learning
Adds automatic metrics for broader coverage within this hub.
Metric Brief
calibration
Coverage: 10 papers (100%)
10 papers (100%) mention calibration.
Examples: Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Who can we trust? LLM-as-a-jury for Comparative Assessment , Discrete Stochastic Localization for Non-autoregressive Generation
Metric Brief
accuracy
Coverage: 4 papers (40%)
4 papers (40%) mention accuracy.
Examples: Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , WISE: Web Information Satire and Fakeness Evaluation , LogiPart: Local Large Language Models for Data Exploration at Scale with Logical Partitioning
Metric Brief
auc
Coverage: 1 papers (10%)
1 papers (10%) mention auc.
Examples: WISE: Web Information Satire and Fakeness Evaluation
Most Cited In This Hub
Fast path to methods with the strongest citation traction in this scope.
Papers: Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Who can we trust? LLM-as-a-jury for Comparative Assessment , Discrete Stochastic Localization for Non-autoregressive Generation
Most Recent
Fast path to latest protocol changes and newly published evaluation setups.
Papers: Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Who can we trust? LLM-as-a-jury for Comparative Assessment , Discrete Stochastic Localization for Non-autoregressive Generation
Best Protocol Detail
Papers with explicit rater/unit metadata and quality-control signals for reproducibility.
Papers: Adaptive Data Augmentation with Multi-armed Bandit: Sample-Efficient Embedding Calibration for Implicit Pattern Recognition , Who can we trust? LLM-as-a-jury for Comparative Assessment , Discrete Stochastic Localization for Non-autoregressive Generation
Minxue Tang, Yangyang Yu, Aolin Ding, Maziyar Baran Pouyan, Taha Belkhouja Yujia Bao · Feb 22, 2026
Recognizing implicit visual and textual patterns is essential in many real-world applications of modern AI.
Mengjie Qian, Guangzhi Sun, Mark J. F. Gales, Kate M. Knill · Feb 18, 2026
Large language models (LLMs) are increasingly applied as automatic evaluators for natural language generation assessment often using pairwise comparative judgements.
Yunshu Wu, Jiayi Cheng, Partha Thakuria, Rob Brekelmans, Evangelos E. Papalexakis · Feb 18, 2026
On OpenWebText, \textsc{DSL} fine-tuning yields large MAUVE gains at low step budgets, surpassing the MDLM+ReMDM baseline with \(\sim\)4$\times$ fewer denoiser evaluations, and matches autoregressive quality at high budgets.
Chenxi Han, Yuheng Min, Zihao Huang, Ao Hong, Hang Liu · Feb 13, 2026
Recent advances in data-driven reinforcement learning and motion tracking have substantially improved humanoid locomotion, yet critical practical challenges remain.
Gaurab Chhetri, Subasish Das, Tausif Islam Chowdhury · Dec 30, 2025
This study develops WISE (Web Information Satire and Fakeness Evaluation) framework which benchmarks eight lightweight transformer models alongside two baseline models on a balanced dataset of 20,000 samples from Fakeddit, annotated as eith
Antonio Martínez-Ibarra, Aurora González-Vidal, Adrián Cánovas-Rodríguez, Antonio F. Skarmeta · Oct 10, 2025
The Mar Menor, Europe's largest hypersaline coastal lagoon, located in southeastern Spain, has undergone severe eutrophication crises, with devastating impacts on biodiversity and water quality.
Tiago Fernandes Tavares · Sep 26, 2025
A qualitative audit by an independent LLM-as-a-judge confirms the discovery of meaningful functional axes, such as policy intent, that thematic ground-truth labels fail to capture.
Denis Makhov, Dmitriy Shopkhoev, Magauiya Zhussip, Ammar Ali, Stamatios Lefkimmiatis · Sep 26, 2025
Post-training compression of large language models (LLMs) often relies on low-rank weight approximations that represent each column of the weight matrix in a shared low-dimensional subspace.
Dmitriy Shopkhoev, Ammar Ali, Magauiya Zhussip, Valentin Malykh, Stamatios Lefkimmiatis · May 5, 2025
Applied to several large language models (LLMs), ReplaceMe achieves up to 25\% pruning while retaining approximately 90\% of the original model's performance on open benchmarks - without any training or healing steps, resulting in minimal c
Qing Lyu, Kumar Shridhar, Chaitanya Malaviya, Li Zhang, Yanai Elazar · Feb 21, 2024
We perform an extensive evaluation across various open and closed-source models on nine reasoning datasets.
