Puffin-World: Scaling a Unified Multimodal Model with Native 3D World States
Abstract
Domain fit: AI-adjacent · Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.
We propose Puffin-World, a unified multimodal architecture that integrates physical understanding, spatial simulation, and 3D world generation and reconstruction without relying on external offline modules. To reliably construct and interact with 3D worlds, our framework jointly models three native world states: physics (gravity field and latitude), geometry (depth), and appearance (image), together with a unified Omni-Camera representation that supports diverse tasks and flexible motions. Beyond modeling these states, we introduce a strategy for propagating physical dynamics across future frames. By grounding absolute camera properties in the real world, Puffin-World enables physically consistent and visually stable world generation. We further couple appearance and geometry within a single generative process, jointly synthesizing each future view and reconstructing its underlying geometry. This unified paradigm enables interleaved closed-loop applications requiring synergy across multiple tasks, including mimic and self-calibrated world exploration. To scale Puffin-World to complex scenarios, we construct Puffin-16M, comprising 15 million vision-language-camera triplets and 1 million trajectories featuring various and challenging motions. To foster further research in this area, we released the code, models, and datasets.
Results and benchmarks
We propose Puffin-World, a unified multimodal architecture that integrates physical understanding, spatial simulation, and 3D world generation and reconstruction without relying on external offline modules.
Benchmark evidence is limited
Evidence graph: 4 refs, 4 links.
Utility signals: depth 65/100, grounding 85/100, status medium.
Implementation
Best maintained implementation now
Puffin Series: Towards Unified Multimodal 3D World Models
516 stars · 22 forks · Last push Sep 4, 2026 · NOASSERTION license
- License
- CI
- Dependencies
- Docker
Matched via arXiv identifier search · Strong overlap with paper title keywords · Community adoption signal (516 stars)
KangLiao929/Puffin is the best available implementation candidate based on ranking signals, but recommendation confidence is not yet high. License is declared (NOASSERTION).
Open KangLiao929/Puffin- No CI workflows detected
- Dependency manifest is missing
- Selected KangLiao929/Puffin as the strongest maintained implementation for new work.
- Repository activity is within the last 24 months.
Reproduction readiness
Major work
No dependency manifest, manual reconstruction required
- KangLiao929/Puffin has no requirements.txt, environment.yml, pyproject.toml, or Dockerfile.
- You will need to reverse-engineer dependencies from import statements in the source code.
Hardware requirements
- Expect multi-day setup/compute for meaningful reproduction based on current guidance.
Validation caveat
Hugging Face artifacts
No direct paper-linked artifacts were found. Showing strongest curated related artifacts for faster exploration.
Datasets
- ACERobotics/Puffin-16M
4,514 downloads · 16 likes · Updated Sep 4, 2026
Spaces
No trustworthy spaces matches right now.
Search spaces on Hugging FaceResearch context
Tasks
Scientific computing
Methods
Transformer
Domains
Computer vision
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXJump to Paper2Code search queries derived from this paper's research context.