Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds
Abstract
Domain fit: AI-adjacent · Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.
Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain stable over extended rollouts. We present JoyAI-Echo-1.5, a unified audio-visual generation system with two purpose-built variants. The long-video variant introduces composable cross-shot memory that aggregates visual evidence across multiple prior shots and speaker cues derived from speech-filtered full-shot audio, enabling persistent character appearance and voice identity across flexible combinations of text, image, and memory conditioning. The world-model variant converts heterogeneous navigation inputs into calibrated metric 6-DoF camera trajectories and injects them through a geometry-aware conditioning pathway, enabling controller-agnostic interaction across flexible viewpoints. To support efficient long-horizon generation, we transform a bidirectional audio-visual backbone into a causal few-step generator using progressive teacher forcing and short- and long-horizon Self-Gradient Forcing on self-generated rollouts. Experiments demonstrate strong performance in both settings. JoyAI-Echo-1.5 achieves improvements over existing long-video baselines in cross-shot consistency, visual quality, text alignment, and speech fidelity. Its world-model variant ranks first on WBench, with an average score of 81.7, and achieves leading visual quality and long-horizon persistence on SANA-WM-Bench. Together, these results indicate that memory, geometric control, and rollout-aware training provide a practical foundation for generating coherent stories and continuously evolving interactive worlds. Project page: https://echo-team-joy-future-academy-jd.github.io/Echo-1.5-Page/.
Results and benchmarks
Video generation is progressing beyond isolated clips toward long-form narratives and interactive worlds, requiring models to preserve identities, follow user controls, and remain stable over extended rollouts.
Benchmark evidence is limited
Evidence graph: 3 refs, 3 links.
Utility signals: depth 65/100, grounding 75/100, status medium.
Implementation
Best maintained implementation now
JoyAI-Echo-1.5: Long-Horizon Audio-Visual Generation for Persistent Stories and Interactive Worlds
2,063 stars · 172 forks · Last push Sep 20, 2026 · NOASSERTION license
- License
- CI
- Dependencies
- Docker
Matched via arXiv identifier search · Strong overlap with paper title keywords · Community adoption signal (2063 stars)
jd-opensource/JoyAI-Echo is the best available implementation candidate based on ranking signals, but recommendation confidence is not yet high. License is declared (NOASSERTION).
Open jd-opensource/JoyAI-Echo- No CI workflows detected
- Dependency manifest is missing
- Selected jd-opensource/JoyAI-Echo as the strongest maintained implementation for new work.
- Repository activity is within the last 24 months.
Compare implementation paths
Compare maintenance quality, reproducibility coverage, and evidence confidence before choosing a reproduction baseline.
- Maintenance
- Recently updated
- Confidence
- Medium
- Reproducibility
- Limited
- Stars
- 2,063
- Last push
- Sep 20, 2026 (40d)
Matched via arXiv identifier search · Strong overlap with paper title keywords
- No CI pipeline detected
- No tagged releases
- No Docker setup
- Maintenance
- Active
- Confidence
- Low
- Reproducibility
- Strong
- Stars
- 7
- Last push
- Oct 22, 2026 (8d)
Matched via arXiv identifier search
- No tagged releases
- No Docker setup
- Low confidence match
Reproduction readiness
Major work
No dependency manifest, manual reconstruction required
- jd-opensource/JoyAI-Echo has no requirements.txt, environment.yml, pyproject.toml, or Dockerfile.
- You will need to reverse-engineer dependencies from import statements in the source code.
Hardware requirements
- Expect multi-day setup/compute for meaningful reproduction based on current guidance.
Validation caveat
Repositories and ecosystem
No additional verified repositories beyond the primary recommendation.
These repositories had low-confidence matching signals and are hidden by default.
- will-rice/lipsync-papers
Confidence: Low · 7 stars
Hugging Face artifacts
No trustworthy direct or curated related Hugging Face artifacts were found yet. Use targeted searches to quickly locate candidate models, datasets, and demos.
Datasets
Tip: start with models, then check datasets and spaces if you need evaluation data or demos.
Research context
Tasks
None detected
Methods
None detected
Domains
Computer vision
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXJump to Paper2Code search queries derived from this paper's research context.