Embedding Inversion via Conditional Masked Diffusion Language Models

Research Summary

Contribution Summary

• We frame embedding inversion as conditional masked diffusion, recovering all tokens in parallel through iterative denoising rather than sequential autoregressive generation.
• A masked diffusion language model is conditioned on the target embedding via adaptive layer normalization, requiring only 8 forward passes with no access to the target encoder at inference time.
• On 32-token sequences across three embedding models, the method achieves token recovery through parallel denoising without requiring encoder access, iterative correction, or architecture-specific alignment.

Related Papers

• When Stereotypes GTG: The Impact of Predictive Text Suggestions on Gender Bias in Human-AI Co-Writing Embedding Neighbor
• TAROT: Test-driven and Capability-adaptive Curriculum Reinforcement Fine-tuning for Code Generation with Large Language Models Embedding Neighbor
• Why Diffusion Language Models Struggle with Truly Parallel (Non-Autoregressive) Decoding? Embedding Neighbor
• PoSh: Using Scene Graphs To Guide LLMs-as-a-Judge For Detailed Image Descriptions Embedding Neighbor
• Duel-Evolve: Reward-Free Test-Time Scaling via LLM Self-Preferences Embedding Neighbor
• Large Language Models and Impossible Language Acquisition: "False Promise" or an Overturn of our Current Perspective towards AI Embedding Neighbor
• Make It Hard to Hear, Easy to Learn: Long-Form Bengali ASR and Speaker Diarization via Extreme Augmentation and Perfect Alignment Embedding Neighbor
• Personalized Prediction of Perceived Message Effectiveness Using Large Language Model Based Digital Twins Embedding Neighbor
• Reconstructing KV Caches with Cross-layer Fusion For Enhanced Transformers Embedding Neighbor
• Inner Speech as Behavior Guides: Steerable Imitation of Diverse Behaviors for Human-AI coordination Tag Overlap
• Simple Yet Effective: Extracting Private Data Across Clients in Federated Fine-Tuning of Large Language Models Tag Overlap
• EARL: Entropy-Aware RL Alignment of LLMs for Reliable RTL Code Generation Tag Overlap