Metric Hub
Updated from current HFEPX corpus (Feb 27, 2026). 10 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Simulation Env. Frequently cited benchmark: APPS. Common metric signal: success rate. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 26, 2026.
Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 10 papers for Success Rate In CS.CL Papers. Dominant protocol signals include automatic metrics, simulation environments, with frequent benchmark focus on APPS, Retrieval and metric focus on success rate, jailbreak success rate. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.
40% of papers report explicit human-feedback signals, led by red-team protocols.
Evidence: MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs , What Matters For Safety Alignment? , Reasoning Up the Instruction Ladder for Controllable Language Models , When Style Breaks Safety: Defending LLMs Against Superficial Style Alignment
automatic metrics appears in 90% of papers in this hub.
Evidence: Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Uncovering Context Reliance in Unstructured Knowledge Editing , MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs
APPS is a recurring benchmark anchor for cross-paper comparisons in this page.
Evidence: The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Execution , Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Uncovering Context Reliance in Unstructured Knowledge Editing
Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Evidence: Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Uncovering Context Reliance in Unstructured Knowledge Editing , MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs
Stratify by benchmark (APPS vs Retrieval) before comparing methods.
Evidence: Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Uncovering Context Reliance in Unstructured Knowledge Editing , MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs
Track metric sensitivity by reporting both success rate and jailbreak success rate.
Evidence: Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Uncovering Context Reliance in Unstructured Knowledge Editing , MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs
Papers with explicit human feedback
Coverage is usable but incomplete (40% vs 45% target).
Papers reporting quality controls
Coverage is a replication risk (0% vs 30% target).
Papers naming benchmarks/datasets
Coverage is a replication risk (20% vs 35% target).
Papers naming evaluation metrics
Coverage is strong (100% vs 35% target).
Papers with known rater population
Coverage is a replication risk (0% vs 35% target).
Papers with known annotation unit
Coverage is a replication risk (0% vs 35% target).
1. Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies
Start here for detailed protocol reporting, including rater and quality-control evidence.
Start here for detailed protocol reporting, including rater and quality-control evidence.
3. Uncovering Context Reliance in Unstructured Knowledge Editing
Start here for detailed protocol reporting, including rater and quality-control evidence.
4. MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs
Adds automatic metrics with red-team protocols for broader coverage within this hub.
5. Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning
Adds automatic metrics for broader coverage within this hub.
6. What Matters For Safety Alignment?
Adds automatic metrics with red-team protocols for broader coverage within this hub.
7. Reasoning Up the Instruction Ladder for Controllable Language Models
Adds automatic metrics with red-team protocols for broader coverage within this hub.
Adds simulation environments for broader coverage within this hub.
automatic_metrics vs simulation_env
both=0, left_only=9, right_only=1
0 papers use both Automatic Metrics and Simulation Env.
Benchmark Brief
APPS
Coverage: 1 papers (10%)
1 papers (10%) mention APPS.
Benchmark Brief
Retrieval
Coverage: 1 papers (10%)
1 papers (10%) mention Retrieval.
Metric Brief
success rate
Coverage: 10 papers (100%)
10 papers (100%) mention success rate.
Examples: Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Uncovering Context Reliance in Unstructured Knowledge Editing
Metric Brief
jailbreak success rate
Coverage: 6 papers (60%)
6 papers (60%) mention jailbreak success rate.
Examples: Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs , Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning
Metric Brief
cost
Coverage: 2 papers (20%)
2 papers (20%) mention cost.
Examples: Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning
Most Cited In This Hub
Fast path to methods with the strongest citation traction in this scope.
Papers: Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Uncovering Context Reliance in Unstructured Knowledge Editing
Most Recent
Fast path to latest protocol changes and newly published evaluation setups.
Papers: Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Uncovering Context Reliance in Unstructured Knowledge Editing
Best Protocol Detail
Papers with explicit rater/unit metadata and quality-control signals for reproducibility.
Papers: Enhancing Persuasive Dialogue Agents by Synthesizing Cross-Disciplinary Communication Strategies , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Uncovering Context Reliance in Unstructured Knowledge Editing
Shinnosuke Nozue, Yuto Nakano, Yotaro Watanabe, Meguru Takasaki, Shoji Moriya · Feb 26, 2026
Current approaches to developing persuasive dialogue agents often rely on a limited set of predefined persuasive strategies that fail to capture the complexity of real-world interactions.
Inderjeet Singh, Vikas Pahuja, Aishvariya Priya Rathina Sabapathy, Chiara Picardi, Amit Giloni · Feb 24, 2026
Current stateless defences for multimodal agentic RAG fail to detect adversarial strategies that distribute malicious semantics across retrieval, planning, and generation components.
Zisheng Zhou, Mengqi Zhang, Shiguang Wu, Xiaotian Ye, Chi Zhang · Feb 22, 2026
Evaluations show that COIN reduces Context Reliance by 45.2% and outperforms strong baselines by 23.6% in editing success rate, highlighting the vital role of mitigating Context Reliance for robust editing.
Chun Yan Ryan Kan, Tommy Tran, Vedant Yadav, Ava Cai, Kevin Zhu · Feb 21, 2026
Defending LLMs against adversarial jailbreak attacks remains an open challenge.
Jyotin Goel, Souvik Maji, Pratik Mazumder · Feb 19, 2026
Instruction-following language models are trained to be helpful and safe, yet their safety behavior can deteriorate under benign fine-tuning and worsen under adversarial updates.
Xing Li, Hui-Ling Zhen, Lihao Yin, Xianzhi Yu, Zhenhua Dong · Jan 7, 2026
This paper presents a comprehensive empirical study on the safety alignment capabilities.
Zishuo Zheng, Vidhisha Balachandran, Chan Young Park, Faeze Brahman, Sachin Kumar · Oct 30, 2025
Our finetuned models achieve consistent improvements on instruction following and instruction hierarchy benchmarks, achieving roughly a 20% improvement on the IHEval conflict setup.
Junlong Li, Wenshuo Zhao, Jian Zhao, Weihao Zeng, Haoze Wu · Oct 29, 2025
Real-world language agents must handle complex, multi-step workflows across diverse Apps.
Linxin Song, Yutong Dai, Viraj Prabhu, Jieyu Zhang, Taiwei Shi · Aug 5, 2025
Autonomous agents that operate computers via Graphical User Interfaces (GUIs) often struggle with efficiency and reliability on complex, long-horizon tasks.
Yuxin Xiao, Sana Tonekaboni, Walter Gerych, Vinith Suriyakumar, Marzyeh Ghassemi · Jun 9, 2025
In this work, we seek to understand whether style patterns compromise LLM safety, how superficial style alignment increases model vulnerability, and how best to mitigate these risks during alignment.
