Skip to content
OpenTrain AIFor AI Companies

Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds

Nan Duan, Haoyang Huang, Weiyang Jin, Haoran Li, Yaowei Li +11 morePublished Aug 24, 2026
arXiv PDF
Researcher verdict
Starting point
Use as implementation starting point
Benchmark evidence
Missing
Not verified yet
Time to first repro
A few days
Plan setup time
Risk flags
2
Review before use

Abstract

Domain fit: AI-adjacent · Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.

Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain stable over extended rollouts. We present JoyAI-Echo-1.5, a unified audio-visual generation system with two purpose-built variants. The long-video variant introduces composable cross-shot memory that aggregates visual evidence across multiple prior shots and speaker cues derived from speech-filtered full-shot audio, enabling persistent character appearance and voice identity across flexible combinations of text, image, and memory conditioning. The world-model variant converts heterogeneous navigation inputs into calibrated metric 6-DoF camera trajectories and injects them through a geometry-aware conditioning pathway, enabling controller-agnostic interaction across flexible viewpoints. To support efficient long-horizon generation, we transform a bidirectional audio-visual backbone into a causal few-step generator using progressive teacher forcing and short- and long-horizon Self-Gradient Forcing on self-generated rollouts. Experiments demonstrate strong performance in both settings. JoyAI-Echo-1.5 achieves improvements over existing long-video baselines in cross-shot consistency, visual quality, text alignment, and speech fidelity. Its world-model variant ranks first on WBench, with an average score of 81.7, and achieves leading visual quality and long-horizon persistence on SANA-WM-Bench. Together, these results indicate that memory, geometric control, and rollout-aware training provide a practical foundation for generating coherent stories and continuously evolving interactive worlds. Project page: https://echo-team-joy-future-academy-jd.github.io/Echo-1.5-Page/.

Results and benchmarks

Freshness tier: hot
Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain stable over extended rollouts.

Implementation

Best maintained implementation now

Recommended
Confidence: Medium
Reproducibility: Limited

JoyAI-Echo-1.5: Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds

2,063 stars · 172 forks · Last push Sep 20, 2026 · NOASSERTION license

  • License
  • CI
  • Dependencies
  • Docker

Matched via arXiv identifier search · Strong overlap with paper title keywords · Community adoption signal (2063 stars)

Why this implementation
Confidence: medium

jd-opensource/JoyAI-Echo is the best available implementation candidate based on ranking signals, but recommendation confidence is not yet high. License is declared (NOASSERTION).

Open jd-opensource/JoyAI-Echo
Reproduction risks
  • No CI workflows detected
  • Dependency manifest is missing
  • Selected jd-opensource/JoyAI-Echo as the strongest maintained implementation for new work.
  • Repository activity is within the last 24 months.

Compare implementation paths

Compare maintenance quality, reproducibility coverage, and evidence confidence before choosing a reproduction baseline.

Maintenance
Recently updated
Confidence
Medium
Reproducibility
Limited
Stars
2,063
Last push
Sep 20, 2026 (40d)

Matched via arXiv identifier search · Strong overlap with paper title keywords

  • No CI pipeline detected
  • No tagged releases
  • No Docker setup
Maintenance
Active
Confidence
Low
Reproducibility
Strong
Stars
7
Last push
Oct 22, 2026 (8d)

Matched via arXiv identifier search

  • No tagged releases
  • No Docker setup
  • Low confidence match

Reproduction readiness

Time to first repro: days
Last checked: Oct 28, 2026

Major work

No dependency manifest, manual reconstruction required

  • jd-opensource/JoyAI-Echo has no requirements.txt, environment.yml, pyproject.toml, or Dockerfile.
  • You will need to reverse-engineer dependencies from import statements in the source code.
Open jd-opensource/JoyAI-Echo

Hardware requirements

  • Expect multi-day setup/compute for meaningful reproduction based on current guidance.

Repositories and ecosystem

No additional verified repositories beyond the primary recommendation.

Hugging Face artifacts

No trustworthy direct or curated related Hugging Face artifacts were found yet. Use targeted searches to quickly locate candidate models, datasets, and demos.

Tip: start with models, then check datasets and spaces if you need evaluation data or demos.

Research context

Tasks

None detected

Methods

None detected

Domains

Computer vision

Evaluation and human feedback data

Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.

Open in HFEPX
Explore similar papers

Jump to Paper2Code search queries derived from this paper's research context.