UltraG-Ray: Physics-Based Gaussian Ray Casting for Novel Ultrasound View Synthesis

View PDF HTML (experimental)

Abstract:Novel view synthesis (NVS) in ultrasound has gained attention as a technique for generating anatomically plausible views beyond the acquired frames, offering new capabilities for training clinicians or data augmentation. However, current methods struggle with complex tissue and view-dependent acoustic effects. Physics-based NVS aims to address these limitations by including the ultrasound image formation process into the simulation. Recent approaches combine a learnable implicit scene representation with an ultrasound-specific rendering module, yet a substantial gap between simulation and reality remains. In this work, we introduce UltraG-Ray, a novel ultrasound scene representation based on a learnable 3D Gaussian field, coupled to an efficient physics-based module for B-mode synthesis. We explicitly encode ultrasound-specific parameters, such as attenuation and reflection, into a Gaussian-based spatial representation and realize image synthesis within a novel ray casting scheme. In contrast to previous methods, this approach naturally captures view-dependent attenuation effects, thereby enabling the generation of physically informed B-mode images with increased realism. We compare our method to state-of-the-art and observe consistent gains in image quality metrics (up to 15% increase on MS-SSIM), demonstrating clear improvement in terms of realism of the synthesized ultrasound images.

Comments: Accepted at MIDL 2026 / to appear in PMLR

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2603.29022 [cs.CV]

(or arXiv:2603.29022v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2603.29022

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Felix Duelmer [view email] [v1] Mon, 30 Mar 2026 21:30:05 UTC (12,829 KB)