Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory
Abstract
Domain fit: AI-adjacent · Paper appears method- or tooling-adjacent to AI workflows with partial ecosystem coverage.
Interactive world models extend video generation from offline clip synthesis toward persistent simulation of interactive virtual worlds, enabling applications in games, robotics, embodied agents, and XR. Achieving stable long-horizon interactive generation, however, remains challenging, as the model must simultaneously preserve scene geometry, dynamic consistency, and camera control while supporting real-time autoregressive generation. Building upon Matrix-Game 3.0, we present Matrix-Game 3.5, as shown in Figure 1, which advances real-time interactive world generation toward geometry-aware and long-horizon consistent simulation through three key improvements. First, we propose a unified geometry-aware memory framework, whose patch-memory and tiled-PRoPE components introduce no additional learnable parameters, combining explicit 3D patch retrieval with projective camera conditioning to enable geometry-consistent camera control and faithful long-horizon scene recall. Second, we introduce a static-dynamic disentangled world representation that separately models static scene geometry and dynamic subjects, preserving both geometric consistency and subject identity throughout long-horizon generation. Third, we develop a two-stage progressive real-time distillation framework that converts a bidirectional diffusion model into a few-step causal generator through Perceptual Flow Matching and curriculum based Self-Rollout DMD, enabling minute-long real-time interactive generation. Extensive experiments demonstrate that, with a unified training corpus spanning Unreal simulation environments, open-world games, and internet videos, MatrixGame 3.5 achieves strong performance in long-horizon scene recall, precise camera control, subject consistency, prompt-driven world generation, and stable real-time open-world interaction.
Results and benchmarks
Interactive world models extend video generation from offline clip synthesis toward persistent simulation of interactive virtual worlds, enabling applications in games, robotics, embodied agents, and XR.
Benchmark evidence is limited
Evidence graph: 2 refs, 1 links.
Utility signals: depth 60/100, grounding 58/100, status medium.
Implementation
No direct implementation yet
Maintained implementation evidence is not confirmed for this paper yet.
Use the implementation status and reproduction sections for the current action plan.
No verified maintained repo yet
There is no verified maintained implementation yet. Use this baseline plan to decide whether to prototype now or defer.
- No direct maintained implementation was found. Use the paper PDF and citation graph to design a baseline reproduction.
- Start from this likely method family: Diffusion.
- Track assumptions and missing details in an experiment log before coding.
Time to first repro: a few days
Recommendation evidence is currently too limited for a maintained-repo choice. Use Implementation Status and Reproduction Path for a practical baseline plan.
- Estimate is based on paper-only reproduction flow
Compare implementation paths
Compare maintenance quality, reproducibility coverage, and evidence confidence before choosing a reproduction baseline.
- Maintenance
- Recently updated
- Confidence
- Medium
- Reproducibility
- Moderate
- Stars
- 182
- Last push
- Sep 7, 2026 (54d)
Matched via arXiv identifier search · Strong overlap with paper title keywords
- No CI pipeline detected
- No tagged releases
- No Docker setup
- Maintenance
- Active
- Confidence
- Low
- Reproducibility
- Moderate
- Stars
- 0
- Last push
- Oct 13, 2026 (17d)
Matched via arXiv identifier search · Strong overlap with paper title keywords
- No CI pipeline detected
- No Docker setup
- Low confidence match
Reproduction readiness
No repo
No verified implementation available
- No maintained repository has been identified for this paper. Check adjacent implementations or HF artifacts below.
Hardware requirements
- Expect multi-day setup/compute for meaningful reproduction based on current guidance.
Validation caveat
Framework baselines
- Hugging Face Diffusers training guide
Practical baseline for diffusion model reproduction.
Repositories and ecosystem
Official
No additional official repositories detected.
Community
- Riemann-Dynamics/Matrix-Game-3.5Confidence: Medium
Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory
182 stars · 9 forks · Last push Sep 7, 2026 · Apache-2.0 license
These repositories had low-confidence matching signals and are hidden by default.
- mkturkcan/matrix-new-york
Confidence: Low · 0 stars
Hugging Face artifacts
No trustworthy direct or curated related Hugging Face artifacts were found yet. Use targeted searches to quickly locate candidate models, datasets, and demos.
Models
Tip: start with models, then check datasets and spaces if you need evaluation data or demos.
Research context
Tasks
Retrieval / indexing, Scientific computing
Methods
Diffusion
Domains
Computer vision, Information Retrieval
Open this paper in HFEPX to review benchmark signals, evaluation modes, and human-feedback protocol context.
Open in HFEPXJump to Paper2Code search queries derived from this paper's research context.