General + Red Team (Last 30 Days)

Researcher Checklist

• Strong: Papers with explicit human feedback

  Coverage is strong (100% vs 45% target).

• Gap: Papers reporting quality controls

  Coverage is a replication risk (0% vs 30% target).

• Gap: Papers naming benchmarks/datasets

  Coverage is a replication risk (0% vs 35% target).

• Moderate: Papers naming evaluation metrics

  Coverage is usable but incomplete (30% vs 35% target).

• Gap: Papers with known rater population

  Coverage is a replication risk (10% vs 35% target).

• Gap: Papers with known annotation unit

  Coverage is a replication risk (10% vs 35% target).

Strengths

• Strong human-feedback signal (100% of papers).

Known Gaps

• Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
• Rater population is under-specified (10% coverage).
• Annotation unit is under-specified (10% coverage).

Suggested Next Analyses

• Track metric sensitivity by reporting both jailbreak success rate and success rate.

Recommended Queries

This section is intentionally expanded only when needed; use “Start Here” above for a faster pass.

Suggested Reading Order

1. TAO-Attack: Toward Advanced Optimization-Based Jailbreak Attacks for Large Language Models

   Start here for detailed protocol reporting and quality-control evidence. Signals: red-team protocols. Abstract: Large language models (LLMs) have achieved remarkable success across diverse applications but remain vulnerable to.

2. MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models

   Start here for detailed protocol reporting and quality-control evidence. Signals: automatic metrics + red-team protocols. Focus: success rate. Abstract: We present MUSE (Multimodal Unified Safety Evaluation), an open-source,.

3. Alignment-Weighted DPO: A principled reasoning approach to improve safety alignment

   Start here for detailed protocol reporting and quality-control evidence. Signals: pairwise preferences. Abstract: Recent advances in alignment techniques such as Supervised Fine-Tuning (SFT), Reinforcement Learning from Human Feedback.

4. MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs

   Include a human-eval paper to calibrate against judge-based evaluation settings. Signals: automatic metrics + red-team protocols. Focus: success rate. Abstract: Defending LLMs against adversarial jailbreak attacks remains an.

5. FENCE: A Financial and Multimodal Jailbreak Detection Dataset

   Adds automatic metrics with red-team protocols for broader protocol coverage within this hub. Signals: automatic metrics + red-team protocols. Focus: accuracy. Abstract: Jailbreaking poses a significant risk to.

6. SibylSense: Adaptive Rubric Learning via Memory Tuning and Adversarial Probing

   Adds evaluation protocol evidence with rubric ratings for broader protocol coverage within this hub. Signals: rubric ratings. Abstract: Rubrics provide structured, interpretable supervision, but scaling rubric construction is.

7. A Systematic Review of Algorithmic Red Teaming Methodologies for Assurance and Security of AI Applications

   Adds automatic metrics with red-team protocols for broader protocol coverage within this hub. Signals: automatic metrics + red-team protocols. Abstract: Cybersecurity threats are becoming increasingly sophisticated, making traditional.

8. Intent Laundering: AI Safety Datasets Are Not What They Seem

   Adds evaluation protocol evidence with red-team protocols for broader protocol coverage within this hub. Signals: red-team protocols. Abstract: We systematically evaluate the quality of widely used AI safety.

Known Limitations

• Only 0% of papers report quality controls; prioritize calibration/adjudication evidence.
• Rater population is under-specified (10% coverage).
• Narrative synthesis is grounded in metadata and abstracts only; full-paper implementation details are not parsed.

Research Utility Snapshot