Metric Hub
Updated from current HFEPX corpus (Feb 27, 2026). 18 papers are grouped in this metric page. Common evaluation modes: Automatic Metrics, Simulation Env. Frequently cited benchmark: Retrieval. Common metric signal: jailbreak success rate. Use this page to compare protocol setup, judge behavior, and labeling design decisions before running new eval experiments. Newest paper in this set is from Feb 26, 2026.
Updated from current HFEPX corpus (Feb 27, 2026). This page tracks 18 papers for Jailbreak Success Rate In CS.CL Papers. Dominant protocol signals include automatic metrics, simulation environments, with frequent benchmark focus on Retrieval, DROP and metric focus on jailbreak success rate, success rate. Use the grounded sections below to prioritize reproducible protocol choices, benchmark-matched comparisons, and judge-vs-human evaluation checks.
27.8% of papers report explicit human-feedback signals, led by red-team protocols.
Evidence: MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs , What Matters For Safety Alignment? , Reasoning Up the Instruction Ladder for Controllable Language Models , When Style Breaks Safety: Defending LLMs Against Superficial Style Alignment
automatic metrics appears in 88.9% of papers in this hub.
Evidence: Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , Robust Long-Form Bangla Speech Processing: Automatic Speech Recognition and Speaker Diarization
Retrieval is a recurring benchmark anchor for cross-paper comparisons in this page.
Evidence: Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Cross-lingual Matryoshka Representation Learning across Speech and Text , Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment
Quality-control reporting is sparse in this slice; prioritize papers with explicit calibration or adjudication steps.
Evidence: Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition
Stratify by benchmark (Retrieval vs DROP) before comparing methods.
Evidence: Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition
Track metric sensitivity by reporting both jailbreak success rate and success rate.
Evidence: Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition
Papers with explicit human feedback
Coverage is usable but incomplete (27.8% vs 45% target).
Papers reporting quality controls
Coverage is a replication risk (0% vs 30% target).
Papers naming benchmarks/datasets
Coverage is usable but incomplete (22.2% vs 35% target).
Papers naming evaluation metrics
Coverage is strong (100% vs 35% target).
Papers with known rater population
Coverage is a replication risk (0% vs 35% target).
Papers with known annotation unit
Coverage is a replication risk (0% vs 35% target).
1. Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems
Start here for detailed protocol reporting, including rater and quality-control evidence.
Start here for detailed protocol reporting, including rater and quality-control evidence.
3. Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing
Start here for detailed protocol reporting, including rater and quality-control evidence.
Adds simulation environments for broader coverage within this hub.
5. Robust Long-Form Bangla Speech Processing: Automatic Speech Recognition and Speaker Diarization
Adds automatic metrics for broader coverage within this hub.
Adds automatic metrics for broader coverage within this hub.
Adds automatic metrics for broader coverage within this hub.
8. Cross-lingual Matryoshka Representation Learning across Speech and Text
Adds automatic metrics for broader coverage within this hub.
automatic_metrics vs simulation_env
both=0, left_only=16, right_only=2
0 papers use both Automatic Metrics and Simulation Env.
Benchmark Brief
Retrieval
Coverage: 2 papers (11.1%)
2 papers (11.1%) mention Retrieval.
Examples: Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , Cross-lingual Matryoshka Representation Learning across Speech and Text
Benchmark Brief
DROP
Coverage: 1 papers (5.6%)
1 papers (5.6%) mention DROP.
Benchmark Brief
Rtc-Bench
Coverage: 1 papers (5.6%)
1 papers (5.6%) mention Rtc-Bench.
Examples: RedTeamCUA: Realistic Adversarial Testing of Computer-Use Agents in Hybrid Web-OS Environments
Metric Brief
jailbreak success rate
Coverage: 18 papers (100%)
18 papers (100%) mention jailbreak success rate.
Examples: Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing
Metric Brief
success rate
Coverage: 6 papers (33.3%)
6 papers (33.3%) mention success rate.
Examples: Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG , MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs , Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning
Metric Brief
error rate
Coverage: 4 papers (22.2%)
4 papers (22.2%) mention error rate.
Examples: Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing , TG-ASR: Translation-Guided Learning with Parallel Gated Cross Attention for Low-Resource Automatic Speech Recognition , Robust Long-Form Bangla Speech Processing: Automatic Speech Recognition and Speaker Diarization
Most Cited In This Hub
Fast path to methods with the strongest citation traction in this scope.
Papers: Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing
Most Recent
Fast path to latest protocol changes and newly published evaluation setups.
Papers: Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing
Best Protocol Detail
Papers with explicit rater/unit metadata and quality-control signals for reproducibility.
Papers: Discourse-Aware Dual-Track Streaming Response for Low-Latency Spoken Dialogue Systems , Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment , Efficient Dialect-Aware Modeling and Conditioning for Low-Resource Taiwanese Hakka Speech Processing
Siyuan Liu, Jiahui Xu, Feng Jiang, Kuang Wang, Zefeng Zhao · Feb 26, 2026
Achieving human-like responsiveness is a critical yet challenging goal for cascaded spoken dialogue systems.
Sanjid Hasan, Risalat Labib, A H M Fuad, Bayazid Hasan · Feb 26, 2026
Ultimately, this work outlines a highly optimized dual pipeline achieving a $\sim$0.019 Real-Time Factor (RTF), establishing a practical, empirically backed benchmark for low-resource, long-form speech processing.
An-Ci Peng, Kuan-Tang Huang, Tien-Hong Lo, Hung-Shin Lee, Hsin-Min Wang · Feb 26, 2026
Taiwanese Hakka is a low-resource, endangered language that poses significant challenges for automatic speech recognition (ASR), including high dialectal variability and the presence of two distinct writing systems (Hanzi and Pinyin).
Cheng-Yeh Yang, Chien-Chun Wang, Li-Wei Chen, Hung-Shin Lee, Hsin-Min Wang · Feb 25, 2026
Low-resource automatic speech recognition (ASR) continues to pose significant challenges, primarily due to the limited availability of transcribed data for numerous languages.
MD. Sagor Chowdhury, Adiba Fairooz Chowdhury · Feb 25, 2026
We describe our end-to-end system for Bengali long-form speech recognition (ASR) and speaker diarization submitted to the DL Sprint 4.0 competition on Kaggle.
Tangsang Chongbang, Pranesh Pyara Shrestha, Amrit Sarki, Anku Jaiswal · Feb 25, 2026
We first establish highly proficient ASR and NMT components: a Wav2Vec2-XLS-R-300m model achieved a state-of-the-art 2.72% CER on OpenSLR-54, and a multi-stage fine-tuned MarianMT model reached a 28.32 BLEU score on the FLORES-200 benchmark
Inderjeet Singh, Vikas Pahuja, Aishvariya Priya Rathina Sabapathy, Chiara Picardi, Amit Giloni · Feb 24, 2026
Current stateless defences for multimodal agentic RAG fail to detect adversarial strategies that distribute malicious semantics across retrieval, planning, and generation components.
Yaya Sy, Dioula Doucouré, Christophe Cerisara, Irina Illina · Feb 23, 2026
We introduce large-scale data curation pipelines and new benchmarks, compare modeling strategies, and show that modality fusion within a frozen text Matryoshka model performs best.
Yonathan Ron, Shiri Gilboa, Tammuz Dubnov · Feb 21, 2026
We introduce Whisper: Courtside Edition, a novel multi-agent large language model (LLM) pipeline that enhances Whisper transcriptions without retraining.
Chun Yan Ryan Kan, Tommy Tran, Vedant Yadav, Ava Cai, Kevin Zhu · Feb 21, 2026
Defending LLMs against adversarial jailbreak attacks remains an open challenge.
Zefang Liu, Chenyang Zhu, Sangwoo Cho, Shi-Xiong Zhang · Feb 21, 2026
Experimental results across diverse benchmarks demonstrate that ReHear effectively mitigates error propagation, consistently outperforming both supervised and pseudo-labeling baselines.
Jayadev Billa · Feb 19, 2026
Current speech LLMs largely perform implicit ASR: on tasks solvable from a transcript, they are behaviorally and mechanistically equivalent to simple Whisper$\to$LLM cascades.
Jyotin Goel, Souvik Maji, Pratik Mazumder · Feb 19, 2026
Instruction-following language models are trained to be helpful and safe, yet their safety behavior can deteriorate under benign fine-tuning and worsen under adversarial updates.
Haoshen Wang, Xueli Zhong, Bingbing Lin, Jia Huang, Xingduo Pan · Feb 9, 2026
Dysarthric speech exhibits high variability and limited labeled data, posing major challenges for both automatic speech recognition (ASR) and assistive speech technologies.
Xing Li, Hui-Ling Zhen, Lihao Yin, Xianzhi Yu, Zhenhua Dong · Jan 7, 2026
This paper presents a comprehensive empirical study on the safety alignment capabilities.
Zishuo Zheng, Vidhisha Balachandran, Chan Young Park, Faeze Brahman, Sachin Kumar · Oct 30, 2025
Our finetuned models achieve consistent improvements on instruction following and instruction hierarchy benchmarks, achieving roughly a 20% improvement on the IHEval conflict setup.
Yuxin Xiao, Sana Tonekaboni, Walter Gerych, Vinith Suriyakumar, Marzyeh Ghassemi · Jun 9, 2025
In this work, we seek to understand whether style patterns compromise LLM safety, how superficial style alignment increases model vulnerability, and how best to mitigate these risks during alignment.
Zeyi Liao, Jaylen Jones, Linxi Jiang, Yuting Ning, Eric Fosler-Lussier · May 28, 2025
Computer-use agents (CUAs) promise to automate complex tasks across operating systems (OS) and the web, but remain vulnerable to indirect prompt injection.
