Metric Hub
Updated from current HFEPX corpus (Feb 27, 2026). 13 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Simulation Env. Common annotation unit: Trajectory. Frequently cited benchmark: DROP. Common metric signal: jailbreak success rate. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 26, 2026.
Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 13 papers for Jailbreak Success Rate In CS.AI Papers. Dominant protocol signals include automatic metrics, simulation environments, with frequent benchmark focus on DROP, Retrieval and metric focus on jailbreak success rate, success rate. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.
30.8% of papers report explicit human-feedback signals, led by red-team protocols.
Evidence: MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs , What Matters For Safety Alignment? , Reasoning Up the Instruction Ladder for Controllable Language Models , When Style Breaks Safety: Defending LLMs Against Superficial Style Alignment
automatic metrics appears in 76.9% of papers in this hub.
Evidence: Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration , Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG
DROP is a recurring benchmark anchor for cross-paper comparisons in this page.
Evidence: Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition
Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Evidence: Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition , Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
Rater context is mostly unspecified rater pools, and annotation is commonly trajectory-level annotation; use this to scope replication staffing.
Evidence: Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition , Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
Stratify by benchmark (DROP vs Retrieval) before comparing methods.
Evidence: Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition , Mitigating Structural Noise in Low-Resource S2TT: An Optimized Cascaded Nepali-English Pipeline with Punctuation Restoration
Papers with explicit human feedback
Coverage is usable but incomplete (30.8% vs 45% target).
Papers reporting quality controls
Coverage is a replication risk (0% vs 30% target).
Papers naming benchmarks/datasets
Coverage is a replication risk (15.4% vs 35% target).
Papers naming evaluation metrics
Coverage is strong (100% vs 35% target).
Papers with known rater population
Coverage is a replication risk (0% vs 35% target).
Papers with known annotation unit
Coverage is a replication risk (7.7% vs 35% target).
Start here for detailed protocol reporting, including rater and quality-control evidence.
2. Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing
Start here for detailed protocol reporting, including rater and quality-control evidence.
Start here for detailed protocol reporting, including rater and quality-control evidence.
Adds automatic metrics for broader coverage within this hub.
Adds automatic metrics for broader coverage within this hub.
6. Training-Free Intelligibility-Guided Observation Addition for Noisy ASR
Adds simulation environments for broader coverage within this hub.
7. AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs
Adds simulation environments for broader coverage within this hub.
8. ICON: Indirect Prompt Injection Defense for Agents based on Inference-Time Correction
Adds automatic metrics for broader coverage within this hub.
automatic_metrics vs simulation_env
both=0, left_only=10, right_only=3
0 papers use both Automatic Metrics and Simulation Env.
Benchmark Brief
DROP
Coverage: 1 papers (7.7%)
1 papers (7.7%) mention DROP.
Benchmark Brief
Retrieval
Coverage: 1 papers (7.7%)
1 papers (7.7%) mention Retrieval.
Metric Brief
jailbreak success rate
Coverage: 13 papers (100%)
13 papers (100%) mention jailbreak success rate.
Examples: Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition
Metric Brief
success rate
Coverage: 6 papers (46.2%)
6 papers (46.2%) mention success rate.
Examples: Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , AdapTools: Adaptive Tool-based Indirect Prompt Injection Attacks on Agentic LLMs , MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs
Metric Brief
accuracy
Coverage: 2 papers (15.4%)
2 papers (15.4%) mention accuracy.
Examples: Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , The Cascade Equivalence Hypothesis: When Do Speech LLMs Behave Like ASR$\rightarrow$LLM Pipelines?
Most Cited In This Hub
Fast path to methods with the strongest citation traction in this scope.
Papers: Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition
Most Recent
Fast path to latest protocol changes and newly published evaluation setups.
Papers: Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition
Best Protocol Detail
Papers with explicit rater/unit metadata and quality-control signals for reproducibility.
Papers: Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition
Sanjid Hasan, Risalat Labib, A H M Fuad, Bayazid Hasan · Feb 26, 2026
Ultimately, this work outlines a highly optimized dual pipeline achieving a $\sim$0.019 Real-Time Factor (RTF), establishing a practical, empirically backed benchmark for low-resource, long-form speech processing.
An-Ci Peng, Kuan-Tang Huang, Tien-Hong Lo, Hung-Shin Lee, Hsin-Min Wang · Feb 26, 2026
Taiwanese Hakka is a low-resource, endangered language that poses significant challenges for automatic speech recognition (ASR), including high dialectal variability and the presence of two distinct writing systems (Hanzi and Pinyin).
Cheng-Yeh Yang, Chien-Chun Wang, Li-Wei Chen, Hung-Shin Lee, Hsin-Min Wang · Feb 25, 2026
Low-resource automatic speech recognition (ASR) continues to pose significant challenges, primarily due to the limited availability of transcribed data for numerous languages.
Tangsang Chongbang, Pranesh Pyara Shrestha, Amrit Sarki, Anku Jaiswal · Feb 25, 2026
We first establish highly proficient ASR and NMT components: a Wav2Vec2-XLS-R-300m model achieved a state-of-the-art 2.72% CER on OpenSLR-54, and a multi-stage fine-tuned MarianMT model reached a 28.32 BLEU score on the FLORES-200 benchmark
Inderjeet Singh, Vikas Pahuja, Aishvariya Priya Rathina Sabapathy, Chiara Picardi, Amit Giloni · Feb 24, 2026
Current stateless defences for multimodal agentic RAG fail to detect adversarial strategies that distribute malicious semantics across retrieval, planning, and generation components.
Haoyang Li, Changsong Liu, Wei Rao, Hao Shi, Sakriani Sakti · Feb 24, 2026
Automatic speech recognition (ASR) degrades severely in noisy environments.
Che Wang, Jiaming Zhang, Ziqi Zhang, Zijie Wang, Yinghui Wang · Feb 24, 2026
The integration of external data services (e.g., Model Context Protocol, MCP) has made large language model-based agents increasingly powerful for complex task execution.
Che Wang, Fuyao Zhang, Jiaming Zhang, Ziqi Zhang, Yinghui Wang · Feb 24, 2026
Large Language Model (LLM) agents are susceptible to Indirect Prompt Injection (IPI) attacks, where malicious instructions in retrieved content hijack the agent's execution.
Chun Yan Ryan Kan, Tommy Tran, Vedant Yadav, Ava Cai, Kevin Zhu · Feb 21, 2026
Defending LLMs against adversarial jailbreak attacks remains an open challenge.
Jayadev Billa · Feb 19, 2026
Current speech LLMs largely perform implicit ASR: on tasks solvable from a transcript, they are behaviorally and mechanistically equivalent to simple Whisper$\to$LLM cascades.
Xing Li, Hui-Ling Zhen, Lihao Yin, Xianzhi Yu, Zhenhua Dong · Jan 7, 2026
This paper presents a comprehensive empirical study on the safety alignment capabilities.
Zishuo Zheng, Vidhisha Balachandran, Chan Young Park, Faeze Brahman, Sachin Kumar · Oct 30, 2025
Our finetuned models achieve consistent improvements on instruction following and instruction hierarchy benchmarks, achieving roughly a 20% improvement on the IHEval conflict setup.
Yuxin Xiao, Sana Tonekaboni, Walter Gerych, Vinith Suriyakumar, Marzyeh Ghassemi · Jun 9, 2025
In this work, we seek to understand whether style patterns compromise LLM safety, how superficial style alignment increases model vulnerability, and how best to mitigate these risks during alignment.
