HelixWorld: A Real-time Interactive Audio-Visual World Model
Abstract
Domain fit: AI-adjacent · Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.
World simulation is inherently multisensory, demanding synchronized visual and acoustic dynamics in real time. Yet prevailing interactive world models remain strictly silent, focusing exclusively on visual rendering and control while overlooking the acoustic dimension. We present HelixWorld, a real-time interactive audio-visual world model where visual scenes and camera-grounded spatial stereo sound co-evolve natively under user interaction. We curate a high-fidelity spatial audio-visual dataset with true stereo acoustics and metric camera poses, upon which we pre-train a bidirectional teacher conditioned on 6-DoF camera trajectories and user actions. To enable low-latency causal interaction, we distill the teacher into a few-step streaming student via an online trajectory distillation loss, sustaining drift-free joint audio-visual rollouts at 24 FPS on a single GPU. Furthermore, we formalize spatial-acoustic consistency and introduce HelixBench to evaluate whether synthesized sound fields faithfully track dynamic viewpoint motion. Extensive experiments demonstrate that HelixWorld matches state-of-the-art silent world models in visual fidelity and responsiveness, while significantly surpassing existing baselines in camera-aligned spatial-acoustic immersion.
Results and benchmarks
World simulation is inherently multisensory, demanding synchronized visual and acoustic dynamics in real time.
Benchmark evidence is limited
Evidence graph: 4 refs, 4 links.
Utility signals: depth 55/100, grounding 85/100, status medium.
Implementation
Best maintained implementation now
🪐 HelixWorld: real-time interactive audio-visual world model.
631 stars · 33 forks · Last push Oct 26, 2026 · Apache-2.0 license
- License
- CI
- Dependencies
- Docker
Matched via arXiv identifier search · Strong overlap with paper title keywords · Community adoption signal (631 stars)
NoizAI/HelixWorld is the best available implementation candidate based on ranking signals, but recommendation confidence is not yet high. License is declared (Apache-2.0). Dependency/environment manifests are present.
Open NoizAI/HelixWorld- No CI workflows detected
- Selected NoizAI/HelixWorld as the strongest maintained implementation for new work.
- Includes dependency/environment manifest signals.
- Repository activity is within the last 24 months.
Reproduction readiness
Setup required
Dependencies pinned, manual setup needed
- NoizAI/HelixWorld has pyproject.toml but requires manual environment setup.
- No Dockerfile, so you will set up the environment manually.
- No CI pipeline, so test coverage is unknown.
Quick start
git clone https://github.com/NoizAI/HelixWorld.git
pip install -e . Validation caveat
Hugging Face artifacts
No direct paper-linked artifacts were found. Showing strongest curated related artifacts for faster exploration.
Models
- ChantalPellegrini/RaDialog-interactive-radiology-report-generation
344 downloads · 27 likes
Broaden model search
Datasets
- yixuan1999/interactive-world-sim-data
2,183 downloads · 1 likes · Updated Mar 8, 2026
Broaden dataset search
Spaces
No trustworthy spaces matches right now.
Search spaces on Hugging FaceResearch context
Tasks
Scientific computing
Methods
None detected
Domains
None detected
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXJump to Paper2Code search queries derived from this paper's research context.