Helpful to a Fault: Measuring Illicit Assistance in Multi-Turn, Multilingual LLM Agents

Nivya Talokar, Ayush K Tarun, Murari Mandal, Maksym Andriushchenko, Antoine Bosselut · Feb 18, 2026 · Citations: 0

Open arXiv Find Implementation RSS feed Shortlist (0)

How to use this page

Coverage: Stale

Use this page to decide whether the paper is strong enough to influence an eval design. If the signals below are thin, treat it as background context and compare it against the stronger hub pages before making protocol choices.

Paper metadata checked

Feb 19, 2026, 10:44 AM

Stale

Protocol signals checked

Feb 19, 2026, 10:44 AM

Stale

Signal strength

Low

Model confidence 0.45

Abstract

LLM-based agents execute real-world workflows via tools and memory. These affordances enable ill-intended adversaries to also use these agents to carry out complex misuse scenarios. Existing agent misuse benchmarks largely test single-prompt instructions, leaving a gap in measuring how agents end up helping with harmful or illegal tasks over multiple turns. We introduce STING (Sequential Testing of Illicit N-step Goal execution), an automated red-teaming framework that constructs a step-by-step illicit plan grounded in a benign persona and iteratively probes a target agent with adaptive follow-ups, using judge agents to track phase completion. We further introduce an analysis framework that models multi-turn red-teaming as a time-to-first-jailbreak random variable, enabling analysis tools like discovery curves, hazard-ratio attribution by attack language, and a new metric: Restricted Mean Jailbreak Discovery. Across AgentHarm scenarios, STING yields substantially higher illicit-task completion than single-turn prompting and chat-oriented multi-turn baselines adapted to tool-using agents. In multilingual evaluations across six non-English settings, we find that attack success and illicit-task completion do not consistently increase in lower-resource languages, diverging from common chatbot findings. Overall, STING provides a practical way to evaluate and stress-test agent misuse in realistic deployment settings, where interactions are inherently multi-turn and often multilingual.

Use caution before copying this protocol

Use this page for context, then validate protocol choices against stronger HFEPX references before implementation decisions.

Extraction confidence is 0.45 (below strong-reference threshold).
No explicit evaluation mode was extracted from available metadata.
No benchmark/dataset or metric anchors were extracted.

Human Eval Hub LLM-as-Judge Hub Pairwise Preference Hub

HFEPX Relevance Assessment

This paper is adjacent to HFEPX scope and is best used for background context, not as a primary protocol reference.

Best use

Background context only

Use if you need

Background context only.

Main weakness

Extraction confidence is 0.45 (below strong-reference threshold).

Trust level

Low

Eval-Fit Score

40/100 • Low

Treat as adjacent context, not a core eval-method reference.

Human Feedback Signal

Detected

Evaluation Signal

Weak / implicit signal

HFEPX Fit

Adjacent candidate

Extraction confidence: Low

If you are doing eval pipeline work, start here:

Human Eval Hub LLM-as-Judge Hub Pairwise Preference Hub Tool-Use Eval Hub

What We Could Reliably Extract

Each protocol field below shows whether the signal looked explicit, partial, or missing in the available metadata. Use this to judge what is safe to trust directly and what still needs full-paper validation.

Human Feedback Types

partial

Red Team

Confidence: Low Source: Persisted extraction evidenced

Directly usable for protocol triage.