Skip to content
OpenTrain AIFor AI Companies

Puffin-World: Scaling a Unified Multimodal Model with Native 3D World States

Kang Liao, Yihang Luo, Xiao-Ming Wu, Linyi Jin, Size Wu +5 morePublished Sep 3, 2026
arXiv PDF
Researcher verdict
Starting point
Use as implementation starting point
Benchmark evidence
Missing
Not verified yet
Time to first repro
A few days
Plan setup time
Risk flags
2
Review before use

Abstract

Domain fit: AI-adjacent · Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.

We propose Puffin-World, a unified multimodal architecture that integrates physical understanding, spatial simulation, and 3D world generation and reconstruction without relying on external offline modules. To reliably construct and interact with 3D worlds, our framework jointly models three native world states: physics (gravity field and latitude), geometry (depth), and appearance (image), together with a unified Omni-Camera representation that supports diverse tasks and flexible motions. Beyond modeling these states, we introduce a strategy for propagating physical dynamics across future frames. By grounding absolute camera properties in the real world, Puffin-World enables physically consistent and visually stable world generation. We further couple appearance and geometry within a single generative process, jointly synthesizing each future view and reconstructing its underlying geometry. This unified paradigm enables interleaved closed-loop applications requiring synergy across multiple tasks, including mimic and self-calibrated world exploration. To scale Puffin-World to complex scenarios, we construct Puffin-16M, comprising 15 million vision-language-camera triplets and 1 million trajectories featuring various and challenging motions. To foster further research in this area, we released the code, models, and datasets.

Results and benchmarks

Freshness tier: hot
We propose Puffin-World, a unified multimodal architecture that integrates physical understanding, spatial simulation, and 3D world generation and reconstruction without relying on external offline modules.

Implementation

Best maintained implementation now

Recommended
Confidence: Medium
Reproducibility: Limited

Puffin Series: Towards Unified Multimodal 3D World Models

516 stars · 22 forks · Last push Sep 4, 2026 · NOASSERTION license

  • License
  • CI
  • Dependencies
  • Docker

Matched via arXiv identifier search · Strong overlap with paper title keywords · Community adoption signal (516 stars)

Why this implementation
Confidence: medium

KangLiao929/Puffin is the best available implementation candidate based on ranking signals, but recommendation confidence is not yet high. License is declared (NOASSERTION).

Open KangLiao929/Puffin
Reproduction risks
  • No CI workflows detected
  • Dependency manifest is missing
  • Selected KangLiao929/Puffin as the strongest maintained implementation for new work.
  • Repository activity is within the last 24 months.

Reproduction readiness

Time to first repro: days
Last checked: Oct 29, 2026

Major work

No dependency manifest, manual reconstruction required

  • KangLiao929/Puffin has no requirements.txt, environment.yml, pyproject.toml, or Dockerfile.
  • You will need to reverse-engineer dependencies from import statements in the source code.
Open KangLiao929/Puffin

Hardware requirements

  • Expect multi-day setup/compute for meaningful reproduction based on current guidance.

Hugging Face artifacts

No direct paper-linked artifacts were found. Showing strongest curated related artifacts for faster exploration.

Datasets

Curated Related

Spaces

No trustworthy spaces matches right now.

Search spaces on Hugging Face

Research context

Tasks

Scientific computing

Methods

Transformer

Domains

Computer vision

Evaluation and human feedback data

Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.

Open in HFEPX
Explore similar papers

Jump to Paper2Code search queries derived from this paper's research context.