Metric Hub
Updated from current HFEPX corpus (Feb 27, 2026). 10 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Simulation Env. Common annotation unit: Ranking. Frequently cited benchmark: Pii-Bench. Common metric signal: relevance. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 26, 2026.
Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 10 papers for Relevance + General Metric Papers. Dominant protocol signals include automatic metrics, simulation environments, with frequent benchmark focus on Pii-Bench, Retrieval and metric focus on relevance, accuracy. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.
automatic metrics appears in 80% of papers in this hub.
Evidence: When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , How Retrieved Context Shapes Internal Representations in RAG , ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting , Testimole-Conversational: A 30-Billion-Word Italian Discussion Board Corpus (1996-2024) for Language Modeling and Sociolinguistic Research
Pii-Bench is a recurring benchmark anchor for cross-paper comparisons in this page.
Evidence: PII-Bench: Evaluating Query-Aware Privacy Protection Systems , CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery , When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
relevance is a repeated reporting metric here, enabling more consistent cross-paper score interpretation.
Evidence: PII-Bench: Evaluating Query-Aware Privacy Protection Systems , CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery , When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Evidence: CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery , When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs , How Retrieved Context Shapes Internal Representations in RAG
Rater context is mostly unspecified rater pools, and annotation is commonly ranking annotation; use this to scope replication staffing.
Evidence: CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery , When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs , How Retrieved Context Shapes Internal Representations in RAG
Stratify by benchmark (Pii-Bench vs Retrieval) before comparing methods.
Evidence: CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery , When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs , How Retrieved Context Shapes Internal Representations in RAG
Direct snippets from paper abstracts to ground protocol and benchmark interpretation.
Human-eval abstract signal: Large language models (LLMs) have created new opportunities to enhance the efficiency of scholarly activities; however, challenges persist in the ethical deployment of AI assistance, including (1) the trustworthiness of AI-generated content, (2) preservation...
Human-eval abstract signal: Visual spatial reasoning (VSR) remains challenging for modern vision-language models (VLMs), despite advances in multimodal architectures.
relevance metric signal: Targeted single spatial cues outperform multi-context aggregation, excessive or weakly relevant commonsense knowledge degrades performance, and CoT prompting improves accuracy only when spatial grounding is sufficiently precise.
Protocol abstract signal: The integration of external data services (e.g., Model Context Protocol, MCP) has made large language model-based agents increasingly powerful for complex task execution.
Protocol abstract signal: Retrieval-augmented generation (RAG) enhances large language models (LLMs) by conditioning generation on retrieved external documents, but the effect of retrieved context is often non-trivial.
Protocol abstract signal: The strong capabilities of recent Large Language Models (LLMs) have made them highly effective for zero-shot re-ranking task.
Protocol abstract signal: We present "Testimole-conversational" a massive collection of discussion boards messages in the Italian language.
Protocol abstract signal: Search relevance plays a central role in web e-commerce.
Papers with explicit human feedback
Coverage is a replication risk (0% vs 45% target).
Papers reporting quality controls
Coverage is a replication risk (0% vs 30% target).
Papers naming benchmarks/datasets
Coverage is a replication risk (20% vs 35% target).
Papers naming evaluation metrics
Coverage is strong (100% vs 35% target).
Papers with known rater population
Coverage is a replication risk (0% vs 35% target).
Papers with known annotation unit
Coverage is usable but incomplete (30% vs 35% target).
1. CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery
Start here for detailed protocol reporting, including rater and quality-control evidence.
Start here for detailed protocol reporting, including rater and quality-control evidence.
3. AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
Start here for detailed protocol reporting, including rater and quality-control evidence.
4. How Retrieved Context Shapes Internal Representations in RAG
Adds automatic metrics for broader coverage within this hub.
5. ReAttn: Improving Attention-based Re-ranking via Attention Re-weighting
Adds automatic metrics for broader coverage within this hub.
Adds automatic metrics for broader coverage within this hub.
Adds automatic metrics for broader coverage within this hub.
8. On the Existence and Behavior of Secondary Attention Sinks
Adds automatic metrics for broader coverage within this hub.
automatic_metrics vs simulation_env
both=0, left_only=8, right_only=2
0 papers use both Automatic Metrics and Simulation Env.
Benchmark Brief
Pii-Bench
Coverage: 1 papers (10%)
1 papers (10%) mention Pii-Bench.
Examples: PII-Bench: Evaluating Query-Aware Privacy Protection Systems
Benchmark Brief
Retrieval
Coverage: 1 papers (10%)
1 papers (10%) mention Retrieval.
Examples: How Retrieved Context Shapes Internal Representations in RAG
Metric Brief
relevance
Coverage: 10 papers (100%)
10 papers (100%) mention relevance.
Examples: CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery , When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
Metric Brief
accuracy
Coverage: 1 papers (10%)
1 papers (10%) mention accuracy.
Metric Brief
jailbreak success rate
Coverage: 1 papers (10%)
1 papers (10%) mention jailbreak success rate.
Examples: AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
Most Cited In This Hub
Fast path to methods with the strongest citation traction in this scope.
Papers: CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery , When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
Most Recent
Fast path to latest protocol changes and newly published evaluation setups.
Papers: CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery , When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
Best Protocol Detail
Papers with explicit rater/unit metadata and quality-control signals for reproducibility.
Papers: CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery , When More Is Less: A Systematic Analysis of Spatial and Commonsense Information for Visual Spatial Reasoning , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
Mengze Hong, Di Jiang, Chen Jason Zhang, Zichang Guo, Yawen Li · Feb 26, 2026
In this work, we present CiteLLM, a specialized agentic platform designed to enable trustworthy reference discovery for grounding author-drafted claims and statements.
Muku Akasaka, Soyeon Caren Han · Feb 25, 2026
In this paper, we conduct a hypothesis-driven analysis of information injection for VSR across three representative VLMs and two public benchmarks.
Che Wang, Jiaming Zhang, Ziqi Zhang, Zijie Wang, Yinghui Wang · Feb 24, 2026
The integration of external data services (e.g., Model Context Protocol, MCP) has made large language model-based agents increasingly powerful for complex task execution.
Samuel Yeh, Sharon Li · Feb 23, 2026
Retrieval-augmented generation (RAG) enhances large language models (LLMs) by conditioning generation on retrieved external documents, but the effect of retrieved context is often non-trivial.
Yuxing Tian, Fengran Mo, Weixu Zhang, Yiyan Qi, Jian-Yun Nie · Feb 23, 2026
The strong capabilities of recent Large Language Models (LLMs) have made them highly effective for zero-shot re-ranking task.
Matteo Rinaldi, Rossella Varvara, Viviana Patti · Feb 16, 2026
We present "Testimole-conversational" a massive collection of discussion boards messages in the Italian language.
Chenji Lu, Zhuo Chen, Hui Zhao, Zhenyi Wang, Pengjie Wang · Dec 31, 2025
While large language models (LLMs) have shown significant results on relevance task, existing benchmarks lack sufficient complexity for comprehensive model assessment, resulting in an absence of standardized relevance evaluation metrics acr
Jeffrey T. H. Wong, Cheng Zhang, Louis Mahon, Wayne Luk, Anton Isopoussu · Dec 22, 2025
Attention sinks are tokens, often the beginning-of-sequence (BOS) token, that receive disproportionately high attention despite limited semantic relevance.
Mingdai Yang, Nurendra Choudhary, Jiangshu Du, Edward W. Huang, Philip S. Yu · Oct 7, 2025
Recent agent-based recommendation frameworks aim to simulate user behaviors by incorporating memory mechanisms and prompting strategies, but they struggle with hallucinating non-existent items and full-catalog ranking.
Hao Shen, Zhouhong Gu, Haokai Hong, Weili Han · Feb 25, 2025
To address this challenge, we propose a query-unrelated PII masking strategy and introduce PII-Bench, the first comprehensive evaluation framework for assessing privacy protection systems.
