Metric Hub
Updated from current HFEPX corpus (Feb 27, 2026). 10 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Simulation Env. Frequently cited benchmark: Retrieval. Common metric signal: coherence. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 20, 2026.
Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 10 papers for Coherence In CS.AI Papers. Dominant protocol signals include automatic metrics, simulation environments, with frequent benchmark focus on Retrieval, Mle-Bench and metric focus on coherence, accuracy. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.
20% of papers report explicit human-feedback signals, led by critique/edit feedback.
Evidence: From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design , Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting , How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?
automatic metrics appears in 60% of papers in this hub.
Evidence: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting , KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification , Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces
Retrieval is a recurring benchmark anchor for cross-paper comparisons in this page.
Evidence: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces , Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting
Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Evidence: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting , From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design , How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?
Stratify by benchmark (Retrieval vs Mle-Bench) before comparing methods.
Evidence: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting , From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design , How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?
Track metric sensitivity by reporting both coherence and accuracy.
Evidence: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting , From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design , How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?
Papers with explicit human feedback
Coverage is a replication risk (20% vs 45% target).
Papers reporting quality controls
Coverage is a replication risk (0% vs 30% target).
Papers naming benchmarks/datasets
Coverage is strong (60% vs 35% target).
Papers naming evaluation metrics
Coverage is strong (100% vs 35% target).
Papers with known rater population
Coverage is a replication risk (0% vs 35% target).
Papers with known annotation unit
Coverage is a replication risk (0% vs 35% target).
1. Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions
Start here for detailed protocol reporting, including rater and quality-control evidence.
2. Protecting Language Models Against Unauthorized Distillation through Trace Rewriting
Start here for detailed protocol reporting, including rater and quality-control evidence.
Start here for detailed protocol reporting, including rater and quality-control evidence.
4. How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?
Adds simulation environments for broader coverage within this hub.
5. Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering
Adds simulation environments for broader coverage within this hub.
6. KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
Adds automatic metrics for broader coverage within this hub.
7. Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces
Adds automatic metrics for broader coverage within this hub.
8. Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs
Adds automatic metrics for broader coverage within this hub.
automatic_metrics vs simulation_env
both=0, left_only=6, right_only=4
0 papers use both Automatic Metrics and Simulation Env.
Benchmark Brief
Retrieval
Coverage: 4 papers (40%)
4 papers (40%) mention Retrieval.
Examples: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Beyond Fact Retrieval: Episodic Memory for RAG with Generative Semantic Workspaces , Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs
Benchmark Brief
Mle-Bench
Coverage: 1 papers (10%)
1 papers (10%) mention Mle-Bench.
Examples: Toward Ultra-Long-Horizon Agentic Science: Cognitive Accumulation for Machine Learning Engineering
Benchmark Brief
sp-abcbench
Coverage: 1 papers (10%)
1 papers (10%) mention sp-abcbench.
Examples: How Well Can LLM Agents Simulate End-User Security and Privacy Attitudes and Behaviors?
Metric Brief
coherence
Coverage: 10 papers (100%)
10 papers (100%) mention coherence.
Examples: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting , From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design
Metric Brief
accuracy
Coverage: 1 papers (10%)
1 papers (10%) mention accuracy.
Examples: KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
Metric Brief
cost
Coverage: 1 papers (10%)
1 papers (10%) mention cost.
Examples: Protecting Language Models Against Unauthorized Distillation through Trace Rewriting
Most Cited In This Hub
Fast path to methods with the strongest citation traction in this scope.
Papers: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting , From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design
Most Recent
Fast path to latest protocol changes and newly published evaluation setups.
Papers: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting , From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design
Best Protocol Detail
Papers with explicit rater/unit metadata and quality-control signals for reproducibility.
Papers: Improving Neural Topic Modeling with Semantically-Grounded Soft Label Distributions , Protecting Language Models Against Unauthorized Distillation through Trace Rewriting , From Pixels to Policies: Reinforcing Spatial Reasoning in Language Models for Content-Aware Layout Design
Raymond Li, Amirhossein Abaskohi, Chuyuan Li, Gabriel Murray, Giuseppe Carenini · Feb 20, 2026
Traditional neural topic models are typically optimized by reconstructing the document's Bag-of-Words (BoW) representations, overlooking contextual information and struggling with data sparsity.
Xinhang Ma, William Yeoh, Ning Zhang, Yevgeniy Vorobeychik · Feb 16, 2026
Knowledge distillation is a widely adopted technique for transferring capabilities from LLMs to smaller, more efficient student models.
Sha Li, Stefano Petrangeli, Yu Shen, Xiang Chen · Feb 14, 2026
We introduce LaySPA, a reinforcement learning framework that equips large language models (LLMs) with explicit and interpretable spatial reasoning for content-aware graphic layout design.
Yuxuan Li, Leyang Li, Hao-Ping Lee, Sauvik Das · Feb 6, 2026
A growing body of research assumes that large language model (LLM) agents can serve as proxies for how people form attitudes toward and behave in response to security and privacy (S&P) threats.
Xinyu Zhu, Yuzhu Cai, Zexi Liu, Bingyang Zheng, Cheng Wang · Jan 15, 2026
The advancement of artificial intelligence toward agentic science is currently bottlenecked by the challenge of ultra-long-horizon autonomy, the ability to sustain strategic coherence and iterative correction over experimental cycles spanni
Erfan Nourbakhsh, Nasrin Sanjari, Ali Nourbakhsh · Dec 9, 2025
Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide, with optical coherence tomography (OCT) serving as a cornerstone for early detection and managemen
Shreyas Rajesh, Pavan Holur, Chenda Duan, David Chong, Vwani Roychowdhury · Nov 10, 2025
On the Episodic Memory Benchmark (EpBench) \cite{huet_episodic_2025} comprising corpora ranging from 100k to 1M tokens in length, GSW outperforms existing RAG based baselines by up to \textbf{20\%}.
Mohammad Tavakoli, Alireza Salemi, Carrie Ye, Mohamed Abdalla, Hamed Zamani · Oct 31, 2025
Evaluating the abilities of large language models (LLMs) for tasks that require long-term memory and thus long-context reasoning, for example in conversational settings, is hampered by the existing benchmarks, which often lack narrative coh
Yubo Li, Xiaobin Shen, Xinyu Yao, Xueying Ding, Yidi Miao · Apr 7, 2025
We organize existing benchmarks and datasets into coherent categories reflecting the evolving landscape of multi-turn dialogue evaluation, and review a broad spectrum of enhancement methodologies, including model-centric strategies (in-cont
Sara Fish, Julia Shephard, Minkai Li, Ran I. Shorrer, Yannai A. Gonczarowski · Mar 24, 2025
We develop evaluation methods for measuring the economic decision-making capabilities and tendencies of LLMs.
