d3LLM: Ultra-Fast Diffusion LLM using Pseudo-Trajectory Distillation 🚀
Model Description
d3LLM-LLaDA is an ultra-fast diffusion language model that achieves high generation speed while maintaining competitive performance. Built on the Dream architecture.
Key Features
- 🚀 High throughput: 5.0× faster than autoregressive models (Qwen-2.5-7B-it) on H100 GPU, 3.5× faster on A100 GPU. Achieves 288.73 tokens/s on H100 (vs 57.32 for AR baseline) on GSM8K-CoT Dataset.
- 📊 High AUP (Accuracy Under Parallelism) scores across benchmarks
- 🔧 Optimized for coding and math reasoning tasks
Usage
For detailed usage instructions, evaluation scripts, training datasets, and training code, please refer to the official GitHub repository and our blog:
- 👉 Code repo: https://github.com/hao-ai-lab/d3LLM
- 🌐 Blog: https://hao-ai-lab.github.io/blogs/text-diffusion/
- Downloads last month
- 826