PAVAS: Physics-Aware Video-to-Audio Synthesis

View PDF

Abstract:Recent advances in Video-to-Audio (V2A) generation have achieved impressive perceptual quality and temporal synchronization, yet most models remain appearance-driven, capturing visual-acoustic correlations without considering the physical factors that shape real-world sounds. We present Physics-Aware Video-to-Audio Synthesis (PAVAS), a method that incorporates physical reasoning into a latent diffusion-based V2A generation through the Physics-Driven Audio Adapter (Phy-Adapter). The adapter receives object-level physical parameters estimated by the Physical Parameter Estimator (PPE), which uses a Vision-Language Model (VLM) to infer the moving-object mass and a segmentation-based dynamic 3D reconstruction module to recover its motion trajectory for velocity computation. These physical cues enable the model to synthesize sounds that reflect underlying physical factors. To assess physical realism, we curate VGG-Impact, a benchmark focusing on object-object interactions, and introduce Audio-Physics Correlation Coefficient (APCC), an evaluation metric that measures consistency between physical and auditory attributes. Comprehensive experiments show that PAVAS produces physically plausible and perceptually coherent audio, outperforming existing V2A models in both quantitative and qualitative evaluations. Visit this https URL for demo videos.

Subjects:

Computer Vision and Pattern Recognition (cs.CV); Multimedia (cs.MM); Sound (cs.SD)

Cite as: arXiv:2512.08282 [cs.CV]

(or arXiv:2512.08282v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2512.08282

arXiv-issued DOI via DataCite

Submission history

From: Hyun-Bin Oh [view email] [v1] Tue, 9 Dec 2025 06:28:50 UTC (3,716 KB) [v2] Mon, 30 Mar 2026 04:54:15 UTC (12,863 KB)