SIREM: Speech-Informed MRI Reconstruction with Learned Sampling

Md Hasan, Nyvenn Castro, Daiqi Liu, Lukas Mulzer, Jana Hutter, Jonghye Woo, Moritz Zaiss, Andreas Maier, Paula A. Perez-Toro · May 18, 2026 · Citations: 0

Open arXiv Find Implementation RSS feed Shortlist (0)

How to use this page

Provisional trust

This page is a lightweight research summary built from the abstract and metadata while deeper extraction catches up.

Best use

Background context only

What to verify

Read the full paper before copying any benchmark, metric, or protocol choices.

Evidence quality

Provisional

Derived from abstract and metadata only.

Abstract

Real-time magnetic resonance imaging (rtMRI) of speech production enables non-invasive visualization of dynamic vocal-tract motion and is valuable for speech science and clinical assessment. However, rtMRI is fundamentally constrained by trade-offs among spatial resolution, temporal resolution, and acquisition speed, often leading to undersampled k-space measurements and degraded reconstructions. We propose SIREM, a speech-informed MRI reconstruction framework that uses synchronized speech as a cross-modal prior. The central idea is that vocal-tract configurations during speech are correlated with the produced acoustics, making part of the image content predictable from audio. SIREM models each frame as a fusion of an audio-driven component and an MRI-driven component through a spatial weighting map. The audio branch predicts articulator-related structure from speech, while the MRI branch reconstructs complementary content from measured k-space data. We further introduce a learnable soft weighting profile over spiral arms, enabling a differentiable study of how k-space arm usage interacts with speech-informed fusion. This yields a unified multimodal formulation that combines audio-driven prediction, MRI reconstruction, and sampling adaptation. We evaluate SIREM on the USC speech rtMRI benchmark against standard baselines, including gridding, wavelet-based compressed sensing, and total variation. SIREM introduces a speech-informed reconstruction paradigm that operates in a substantially higher-throughput regime than iterative methods while preserving anatomically plausible vocal-tract structure. These results establish an initial benchmark for multimodal speech-informed rtMRI reconstruction and highlight the potential of synchronized speech as an auxiliary prior for fast reconstruction. The source code is available at https://github.com/mdhasanai/SIREM

Abstract-only analysis — low confidence

All signals on this page are inferred from the abstract only and may be inaccurate. Do not use this page as a primary protocol reference.

This page is still relying on abstract and metadata signals, not a fuller protocol read.

Human Eval Hub LLM-as-Judge Hub Pairwise Preference Hub

Should You Rely On This Paper?

Signal extraction is still processing. This page currently shows metadata-first guidance until structured protocol fields are ready.

Best use

Background context only

Use if you need

A provisional background reference while structured extraction finishes.

Main weakness

This page is still relying on abstract and metadata signals, not a fuller protocol read.

Trust level

Provisional

Usefulness score

Unavailable

Eval-fit score is unavailable until extraction completes.

Human Feedback Signal

Not explicit in abstract metadata

Evaluation Signal

Weak / implicit signal

Usefulness for eval research

Provisional (processing)

Extraction confidence 0%

If you are doing eval pipeline work, start here:

Human Eval Hub LLM-as-Judge Hub Pairwise Preference Hub Tool-Use Eval Hub

What We Could Verify

These are the protocol signals we could actually recover from the available paper metadata. Use them to decide whether this paper is worth deeper reading.