Object-Centric World Models for Causality-Aware Reinforcement Learning

View PDF HTML (experimental)

Abstract:World models have been developed to support sample-efficient deep reinforcement learning agents. However, it remains challenging for world models to accurately replicate environments that are high-dimensional, non-stationary, and composed of multiple objects with rich interactions since most world models learn holistic representations of all environmental components. By contrast, humans perceive the environment by decomposing it into discrete objects, facilitating efficient decision-making. Motivated by this insight, we propose \emph{Slot Transformer Imagination with CAusality-aware reinforcement learning} (STICA), a unified framework in which object-centric Transformers serve as the world model and causality-aware policy and value networks. STICA represents each observation as a set of object-centric tokens, together with tokens for the agent action and the resulting reward, enabling the world model to predict token-level dynamics and interactions. The policy and value networks then estimate token-level cause--effect relations and use them in the attention layers, yielding causality-guided decision-making. Experiments on object-rich benchmarks demonstrate that STICA consistently outperforms state-of-the-art agents in both sample efficiency and final performance.

Comments: Accepted by AAAI-26. Codes are available at this https URL

Subjects:

Machine Learning (cs.LG); Artificial Intelligence (cs.AI)

Cite as: arXiv:2511.14262 [cs.LG]

(or arXiv:2511.14262v3 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2511.14262

arXiv-issued DOI via DataCite

Submission history

From: Yosuke Nishimoto [view email] [v1] Tue, 18 Nov 2025 08:53:09 UTC (6,367 KB) [v2] Thu, 25 Dec 2025 07:22:11 UTC (6,533 KB) [v3] Mon, 30 Mar 2026 07:20:18 UTC (6,533 KB)