Learning to Detect Language Model Training Data via Active Reconstruction
Junjie Oscar Yin, John X. Morris, Vitaly Shmatikov, Sewon Min, Hannaneh Hajishirzi
Abstract
Detecting LLM training data is generally framed as a membership inference attack (MIA) problem. However, conventional MIAs operate passively on fixed model weights, using log-likelihoods or text generations. In this work, we introduce \textbf{Active Data Reconstruction Attack} (ADRA), a family of MIA that actively induces a model to reconstruct a given text through training. We hypothesize that training data are \tex...
Results & Benchmarks
| Task | Dataset | Metric | Value |
|---|---|---|---|
| Representation learning | AUROC (Top-5 Mean) | Embedding | 76.9 |
| Reinforcement learning | AIME | Eff. | 64 |
Hardware Requirements
- Expect multi-day setup/compute for meaningful reproduction based on current guidance.
Best Implementation
Maintained implementation evidence is not confirmed for this paper yet.
Use the Implementation Status and Reproduction Path sections below for the current action plan.
Reproduction Path
Follow this baseline workflow to decide if this paper is worth immediate prototyping.
- 1
No maintained paper-verified implementation was found; start with the closest related repositories below.
- 2
Compare repo methods against the paper equations/algorithm before trusting metrics.
- 3
Create a minimal baseline implementation from the paper and use adjacent repos as references.
- 4
Prioritize reproducing the core method first: Reinforcement learning.
Related Implementations
These are not paper-verified. Use them as reference points when no direct implementation is available.
Matches contextual method/domain keyword: reinforcement learning
Additional Implementations
No additional verified repositories beyond the primary recommendation.
Hugging Face Artifacts
No trustworthy direct or curated related Hugging Face artifacts were found yet.
Continue with targeted Hugging Face searches: