IMPASTO: Integrating Model-Based Planning with Learned Dynamics Models for Robotic Oil Painting Reproduction

View PDF HTML (experimental)

Abstract:Robotic reproduction of oil paintings using soft brushes and pigments requires force-sensitive control of deformable tools, prediction of brushstroke effects, and multi-step stroke planning, often without human step-by-step demonstrations or faithful simulators. Given only a sequence of target oil painting images, can a robot infer and execute the stroke trajectories, forces, and colors needed to reproduce it? We present IMPASTO, a robotic oil-painting system that integrates learned pixel dynamics models with model-based planning. The dynamics models predict canvas updates from image observations and parameterized stroke actions; a receding-horizon model predictive control optimizer then plans trajectories and forces, while a force-sensitive controller executes strokes on a 7-DoF robot arm. IMPASTO integrates low-level force control, learned dynamics models, and high-level closed-loop planning, learns solely from robot self-play, and approximates human artists' single-stroke datasets and multi-stroke artworks, outperforming baselines in reproduction accuracy. Project website: this https URL

Subjects:

Robotics (cs.RO); Artificial Intelligence (cs.AI)

Cite as: arXiv:2603.29315 [cs.RO]

(or arXiv:2603.29315v1 [cs.RO] for this version)

https://doi.org/10.48550/arXiv.2603.29315

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yingke Wang [view email] [v1] Tue, 31 Mar 2026 06:33:59 UTC (17,763 KB)