Skip to content
OpenTrain AIFor AI Companies

HelixWorld: A Real-time Interactive Audio-Visual World Model

Lei Ke, Jiahao Pan, Zeyue Tian, Jiaming Wang, Haoyuan Huang +11 morePublished Sep 29, 2026
arXiv PDF
Researcher verdict
Starting point
Use as implementation starting point
Benchmark evidence
Missing
Not verified yet
Time to first repro
A few hours
Fast first run
Risk flags
1
Review before use

Abstract

Domain fit: AI-adjacent · Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.

World simulation is inherently multisensory, demanding synchronized visual and acoustic dynamics in real time. Yet prevailing interactive world models remain strictly silent, focusing exclusively on visual rendering and control while overlooking the acoustic dimension. We present HelixWorld, a real-time interactive audio-visual world model where visual scenes and camera-grounded spatial stereo sound co-evolve natively under user interaction. We curate a high-fidelity spatial audio-visual dataset with true stereo acoustics and metric camera poses, upon which we pre-train a bidirectional teacher conditioned on 6-DoF camera trajectories and user actions. To enable low-latency causal interaction, we distill the teacher into a few-step streaming student via an online trajectory distillation loss, sustaining drift-free joint audio-visual rollouts at 24 FPS on a single GPU. Furthermore, we formalize spatial-acoustic consistency and introduce HelixBench to evaluate whether synthesized sound fields faithfully track dynamic viewpoint motion. Extensive experiments demonstrate that HelixWorld matches state-of-the-art silent world models in visual fidelity and responsiveness, while significantly surpassing existing baselines in camera-aligned spatial-acoustic immersion.

Results and benchmarks

Freshness tier: hot
World simulation is inherently multisensory, demanding synchronized visual and acoustic dynamics in real time.

Implementation

Best maintained implementation now

Recommended
Confidence: Medium
Reproducibility: Moderate

🪐 HelixWorld: real-time interactive audio-visual world model.

631 stars · 33 forks · Last push Oct 26, 2026 · Apache-2.0 license

  • License
  • CI
  • Dependencies
  • Docker

Matched via arXiv identifier search · Strong overlap with paper title keywords · Community adoption signal (631 stars)

Why this implementation
Confidence: medium

NoizAI/HelixWorld is the best available implementation candidate based on ranking signals, but recommendation confidence is not yet high. License is declared (Apache-2.0). Dependency/environment manifests are present.

Open NoizAI/HelixWorld
Reproduction risks
  • No CI workflows detected
  • Selected NoizAI/HelixWorld as the strongest maintained implementation for new work.
  • Includes dependency/environment manifest signals.
  • Repository activity is within the last 24 months.

Reproduction readiness

Time to first repro: hours
Last checked: Oct 29, 2026

Setup required

Dependencies pinned, manual setup needed

  • NoizAI/HelixWorld has pyproject.toml but requires manual environment setup.
  • No Dockerfile, so you will set up the environment manually.
  • No CI pipeline, so test coverage is unknown.
Open NoizAI/HelixWorld

Quick start

git clone https://github.com/NoizAI/HelixWorld.git
pip install -e .

Hugging Face artifacts

No direct paper-linked artifacts were found. Showing strongest curated related artifacts for faster exploration.

Datasets

Curated Related

Spaces

No trustworthy spaces matches right now.

Search spaces on Hugging Face

Research context

Tasks

Scientific computing

Methods

None detected

Domains

None detected

Evaluation and human feedback data

Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.

Open in HFEPX
Explore similar papers

Jump to Paper2Code search queries derived from this paper's research context.