Large language models (LLMs) show promise in generating supportive responses for mental health queries, but improving their usefulness, empathy, and safety often requires substantial compute, expert input, and labeled data.
LLUMI consists of two complementary components: a generation model (GM), which drafts supportive responses to mental health queries, and an improvement model (IM), which revises an initial human-crafted response.
Feng Han, Zhixiong Zhang, Zheming Liang, Yibin Wang, Jiaqi Wang · May 28, 2026 · Citations: 0
Pairwise PreferenceGeneral
Such data bias leads VLMs to exhibit distinct preferences for information acquisition across different modalities.
Extensive experiments across 13 diverse multimodal benchmarks demonstrate that LoMo significantly improves overall multimodal reasoning and yields deeper cross-modal fusion.
Amrita Mazumdar, Seonwook Park, Rajarshi Roy, Nikhil Srihari, Shengze Wang, Yuhao Zhou · May 28, 2026 · Citations: 0
Rubric RatingGeneral
In this work, we present VideoFDB, the first benchmark to evaluate full-duplex audio-visual-to-audio-visual (AV2AV) conversational agents.
Across open- and closed-source vision-speech agents, we find systematic failure modes: captioning collapse and visual-stream ignorance, and we show that current systems exploit vision for explicit visual question answering but not for the…
Bo-Han Feng, Yu-Hsuan Li Liang, Chien-Feng Liu, You-Hsuan Chang, Yun-Nung Chen · May 28, 2026 · Citations: 0
Red TeamAutomatic MetricsGeneral
Existing work studies these risks under heterogeneous threat models and evaluation protocols, making it difficult to compare attack practicality or defense utility.
This paper provides a unified taxonomy and a controlled empirical evaluation of LALM jailbreak attacks and defenses.
Zhenghao Herbert Zhou, R. Thomas McCoy, Robert Frank · May 28, 2026 · Citations: 0
Pairwise PreferenceGeneral
We show that verb bias is causally represented in steering vectors extracted from large language models: counterfactual edits to verb bias systematically shift downstream structural preferences.
Pairwise PreferenceLlm As JudgeAutomatic MetricsGeneral
The consequences are stark: the panel's actual accuracy falls 8-22 percentage points short of what independent voting would achieve, and the best single judge matches or outperforms the full panel across all conditions.
Neither adding more judges nor using smarter aggregation algorithms helps -- established methods close at most 11% of this gap, even with access to the correct answers.
ZhiShu Jiang, Haibo Liu, Xin Shen, Guanqiang QI, Chenxi Miao, Weikang Li · May 28, 2026 · Citations: 0
Pairwise PreferenceGeneral
Long-term conversational agents are expected to remember past interactions, but memory is useful only when the right evidence is recalled for the right user.
Existing memory-augmented LLM agents have made progress in building compact memory banks, yet retrieval is still often driven by query-centered similarity or fixed ranking rules, leaving user-conditioned relevance underexplored.
Ye Yuan, Rui Song, Weien Li, Zeyu Li, Haochen Liu, Xiangyu Kong · May 26, 2026 · Citations: 0
Automatic MetricsGeneral
However, most environments are scored only by game outcomes such as win rates and largely remain to text-only interaction, making it difficult to tell whether an agent's language is actually grounded in what it perceived and did, or to…
To address this gap, we introduce QUACK, an open-source environment and evaluation framework for auditing the grounding of agent language in multimodal social reasoning.
We investigate whether acoustic emotion recognition models can serve as proxies for the Pathos dimension in political speech analysis, as operationalised by the TRUST multi-agent large language model (LLM) pipeline.
We further demonstrate, via a systematic quality evaluation of the Berlin Database of Emotional Speech (EMO-DB) using Gemini in an open-ended annotation paradigm, that standard SER benchmark corpora suffer from acted speech, cultural bias,…
Yejin Cho, Katrin Erk · May 21, 2026 · Citations: 0
Pairwise PreferenceAutomatic MetricsGeneral
Our contributions are three-fold: (1) a structured representation framework for situated lexical meaning; (2) COCA-Scenes, a dataset of 520 usage instances across 26 keywords for distinct scene identification; and (3) empirical evidence…
Jingyi Kang, Junyu Lu, Bo Xu, Hongbo Wang, Linlin zong, Roy Ka-Wei Lee · May 21, 2026 · Citations: 0
Red TeamHuman EvalGeneral
We introduce Chinese Implicit Toxicity Attack (CITA), a controlled red-team evaluation and defense-data generation framework, not a deployable evasion tool.
On CITA-generated evaluation samples, the seven tested detectors exhibit substantial missed-detection risks, reaching an average ASR of 69.48%; human evaluation further confirms preserved harmfulness and increased implicitness/evasiveness.
Training tool-calling agents requires large-scale trajectory data with verifiable labels, yet existing approaches either synthesize environments that diverge from real API behavior or generate tasks without ground-truth outcomes for…
To address environment drift in live APIs, we construct a retrieval-augmented simulator that caches all exploration results and replays them during training and evaluation, enabling fully offline and reproducible RL.
Simiao Liu, Fang Liu, Peiding Wang, Taichuan Li, Yinghao Zhu, Xiaoli Lian · May 17, 2026 · Citations: 0
DemonstrationsGeneral
We present ContraFix, an agentic AVR framework that constructs such evidence through contrastive runtime analysis.
A semantic audit of benchmark-validated SEC-Bench patches shows that 58.2% of ContraFix's patches are semantically correct, compared with 31.3% for the strongest baseline, indicating that the proposed framework improves semantic correctness…
Qinwu Xu, Zhuoheng Li, Jessie Salas · May 13, 2026 · Citations: 0
Pairwise PreferenceGeneral
Selecting a final checkpoint for multimodal large language models (MLLMs) is challenging when late-stage candidates are closely matched and downstream evaluation signals are noisy.
Small observed differences can be comparable to variability introduced by finite evaluation samples, LLM-based judges, and ambiguous multimodal evidence, while validation loss may not identify the checkpoint preferred by downstream…
Oscar Gilg, Pierre Beckmann, Daniel Paleka, Patrick Butlin · May 13, 2026 · Citations: 0
Pairwise PreferenceGeneral
Large language models (LLMs) can be said to have preferences: they reliably pick certain tasks and outputs over others, and preferences shaped by post-training and system prompts appear to shape much of their behaviour.
But models can also adopt different personas which have radically different preferences.
