VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control

View PDF HTML (experimental)

Abstract:Video world models aim to simulate dynamic, real-world environments, yet existing methods struggle to provide unified and precise control over camera and multi-object motion, as videos inherently capture dynamics in the projected 2D image plane. To bridge this gap, we introduce VerseCrafter, a geometry-driven video world model that generates dynamic, realistic videos from a unified 4D geometric world state. Our approach is centered on a novel 4D Geometric Control representation, which encodes the world state as a static background point cloud and per-object 3D Gaussian trajectories. This representation captures each object's motion path and probabilistic 3D occupancy over time, providing a flexible, category-agnostic alternative to rigid bounding boxes and parametric models. We render 4D Geometric Control into 4D control maps for a pretrained video diffusion model, enabling high-fidelity, view-consistent video generation that faithfully follows the specified dynamics. To enable training at scale, we develop an automatic data engine and construct VerseControl4D, a real-world dataset of 35K training samples with automatically derived prompts and rendered 4D control maps. Extensive experiments show that VerseCrafter achieves superior visual quality and more accurate control over camera and multi-object motion than prior methods.

Comments: Project Page: this https URL, Accepted by CVPR 2026

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2601.05138 [cs.CV]

(or arXiv:2601.05138v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2601.05138

arXiv-issued DOI via DataCite

Submission history

From: Sixiao Zheng [view email] [v1] Thu, 8 Jan 2026 17:28:52 UTC (42,718 KB) [v2] Mon, 30 Mar 2026 02:18:28 UTC (58,237 KB)