CDLM: Consistency Diffusion Language Models For Faster Sampling

Research Summary

Contribution Summary

• Diffusion Language Models (DLMs) offer a promising parallel generation paradigm but suffer from slow inference due to numerous refinement steps and the inability to use standard KV caching.
• We introduce CDLM (Consistency Diffusion Language Models), a training-based acceleration method that simultaneously tackles both bottlenecks.
• CDLM integrates consistency modeling to drastically reduce the number of required sampling steps by enabling multi-token finalization.

Why It Matters For Eval

• The full training and evaluation code is available at https://github.com/SqueezeAILab/CDLM.

Related Papers

• Should You Use Your Large Language Model to Explore or Exploit? Embedding Neighbor
• Following the Diagnostic Trace: Visual Cognition-guided Cooperative Network for Chest X-Ray Diagnosis Embedding Neighbor
• Causal Effect Estimation with Latent Textual Treatments Embedding Neighbor
• Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning Embedding Neighbor
• Explainable AI: Context-Aware Layer-Wise Integrated Gradients for Explaining Transformer Models Embedding Neighbor
• CodePDE: An Inference Framework for LLM-driven PDE Solver Generation Embedding Neighbor
• Latent-Augmented Discrete Diffusion Models Embedding Neighbor
• Anatomy of Capability Emergence: Scale-Invariant Representation Collapse and Top-Down Reorganization in Neural Networks Embedding Neighbor
• EconEvals: Benchmarks and Litmus Tests for Economic Decision-Making by LLM Agents Embedding Neighbor
• Esoteric Language Models: Bridging Autoregressive and Masked Diffusion LLMs Tag Overlap
• Recursive Concept Evolution for Compositional Reasoning in Large Language Models Tag Overlap
• RFEval: Benchmarking Reasoning Faithfulness under Counterfactual Reasoning Intervention in Large Reasoning Models Tag Overlap