Stop-Think-AutoRegress: Language Modeling with Latent Diffusion Planning

Research Summary

Contribution Summary

• The Stop-Think-AutoRegress Language Diffusion Model (STAR-LDM) integrates latent diffusion planning with autoregressive generation.
• Unlike conventional autoregressive language models limited to token-by-token decisions, STAR-LDM incorporates a "thinking" phase that pauses generation to refine a semantic plan through diffusion before continuing.
• This enables global planning in continuous space prior to committing to discrete tokens.

Why It Matters For Eval

• Evaluations show STAR-LDM significantly outperforms similar-sized models on language understanding benchmarks and achieves $>70\%$ win rates in LLM-as-judge comparisons for narrative coherence and commonsense reasoning.

Related Papers

• Can Multimodal LLMs Perform Time Series Anomaly Detection? Embedding Neighbor
• Communication-Inspired Tokenization for Structured Image Representations Embedding Neighbor
• Flatter Tokens are More Valuable for Speculative Draft Model Training Embedding Neighbor
• Using the Path of Least Resistance to Explain Deep Networks Embedding Neighbor
• Repurposing Synthetic Data for Fine-grained Search Agent Supervision Embedding Neighbor
• Alignment-Weighted DPO: A principled reasoning approach to improve safety alignment Embedding Neighbor
• FigEx2: Visual-Conditioned Panel Detection and Captioning for Scientific Compound Figures Embedding Neighbor
• Towards Expectation Detection in Language: A Case Study on Treatment Expectations in Reddit Embedding Neighbor
• CUICurate: A GraphRAG-based Framework for Automated Clinical Concept Curation for NLP applications Embedding Neighbor
• Overton Pluralistic Reinforcement Learning for Large Language Models Tag Overlap
• The Headless Firm: How AI Reshapes Enterprise Boundaries Tag Overlap
• Measuring AI Ability to Complete Long Software Tasks Tag Overlap