The Mamba in the Llama: Distilling and Accelerating Hybrid Models
Results and benchmarks
The Mamba in the Llama: Distilling and Accelerating Hybrid Models is the primary contribution described in this paper.
| Task | Dataset | Metric | Value | Source |
|---|---|---|---|---|
| Distilling Accelerating Hybrid Models | MMLU | Perplexity | 6.20 | paper-derived |
| Distilling Accelerating Hybrid Models | MT-Bench | Perplexity | 46.67 | paper-derived |
| Distilling Accelerating Hybrid Models | ARC | Perplexity | 6.20 | paper-derived |
| Distilling Accelerating Hybrid Models | TruthfulQA | Perplexity | 47.98 | paper-derived |
| Distilling Accelerating Hybrid Models | HellaSwag | Perplexity | 46.67 | paper-derived |
Audit each benchmark finding before selecting an implementation path. Evidence refs map to the disclosure below.
Evidence graph: 3 refs, 3 links.
Utility signals: depth 90/100, grounding 85/100, status high.
Implementation
Best maintained implementation now
[NeurIPS 2024] Official Repository of The Mamba in the Llama: Distilling and Accelerating Hybrid Models
243 stars · 22 forks · Last push Oct 14, 2025 · Apache-2.0 license
- License
- CI
- Dependencies
- Docker
Official implementation from Papers with Code · Repository link is mentioned in the paper metadata · Strong overlap with paper title keywords
jxiw/mambainllama is the strongest maintained implementation based on ranking signals. License is declared (Apache-2.0). Dependency/environment manifests are present.
Open jxiw/mambainllama- No CI workflows detected
- Selected jxiw/mambainllama as the strongest maintained implementation for new work.
- Includes dependency/environment manifest signals.
- Repository activity is within the last 24 months.
- Official repository is preserved separately as historical context.
Compare implementation paths
Compare maintenance quality, reproducibility coverage, and evidence confidence before choosing a reproduction baseline.
- Maintenance
- Stale risk
- Confidence
- High
- Reproducibility
- Moderate
- Stars
- 243
- Last push
- Oct 14, 2025 (316d)
Official implementation from Papers with Code · Repository link is mentioned in the paper metadata
- No CI pipeline detected
- No tagged releases
- No Docker setup
- Maintenance
- Stale
- Confidence
- High
- Reproducibility
- Limited
- Stars
- 17
- Last push
- Nov 28, 2024 (636d)
Official implementation from Papers with Code · Repository link is mentioned in the paper metadata
- No push in 12+ months
- No CI pipeline detected
- No tagged releases
Reproduction readiness
Setup required
Dependencies pinned, manual setup needed
- jxiw/mambainllama has environment.yml but requires manual environment setup.
- Last push was 316 days ago, so expect possible dependency version conflicts.
- No Dockerfile, so you will set up the environment manually.
- No CI pipeline, so test coverage is unknown.
Quick start
git clone https://github.com/jxiw/mambainllama.git
conda env create -f environment.yml && conda activate <env-name>Hugging Face artifacts
No trustworthy direct or curated related Hugging Face artifacts were found yet. Use targeted searches to quickly locate candidate models, datasets, and demos.
Tip: start with models, then check datasets and spaces if you need evaluation data or demos.
Research context
Tasks
Distilling Accelerating Hybrid Models
Methods
None detected
Domains
None detected
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXJump to Paper2Code search queries derived from this paper's research context.
Data includes links from Papers with Code ( CC-BY-SA-4.0 ).