Models model benchmark announce prediction policy arxiv

CLaD: Planning with Grounded Foresight via Cross-Modal Latent Dynamics

arXiv cs.ROby Andrew Jeong, Jaemin Kim, Sebin Lee, Sung-Eui YoonApril 1, 20261 min read0 views

arXiv:2603.29409v1 Announce Type: new Abstract: Robotic manipulation involves kinematic and semantic transitions that are inherently coupled via underlying actions. However, existing approaches plan within either semantic or latent space without explicitly aligning these cross-modal transitions. To address this, we propose CLaD, a framework that models how proprioceptive and semantic states jointly evolve under actions through asymmetric cross-attention that allows kinematic transitions to query semantic ones. CLaD predicts grounded latent foresights via self-supervised objectives with EMA target encoders and auxiliary reconstruction losses, preventing representation collapse while anchoring predictions to observable states. Predicted foresights are modulated with observations to condition

View PDF HTML (experimental)

Abstract:Robotic manipulation involves kinematic and semantic transitions that are inherently coupled via underlying actions. However, existing approaches plan within either semantic or latent space without explicitly aligning these cross-modal transitions. To address this, we propose CLaD, a framework that models how proprioceptive and semantic states jointly evolve under actions through asymmetric cross-attention that allows kinematic transitions to query semantic ones. CLaD predicts grounded latent foresights via self-supervised objectives with EMA target encoders and auxiliary reconstruction losses, preventing representation collapse while anchoring predictions to observable states. Predicted foresights are modulated with observations to condition a diffusion policy for action generation. On LIBERO-LONG benchmark, CLaD achieves 94.7% success rate, competitive with large VLAs with significantly fewer parameters.

Comments: Project page: this https URL

Subjects:

Robotics (cs.RO)

Cite as: arXiv:2603.29409 [cs.RO]

(or arXiv:2603.29409v1 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2603.29409

arXiv-issued DOI via DataCite (pending registration)