ConceptPrism: Concept Disentanglement in Personalized Diffusion Models via Residual Token Optimization

View PDF HTML (experimental)

Abstract:Personalized text-to-image (T2I) generation has emerged as a key application for creating user-specific concepts from a few reference images. The core challenge is concept disentanglement: separating the target concept from irrelevant residual information. Lacking such disentanglement, capturing high-fidelity features often incorporates undesired attributes that conflict with user prompts, compromising the trade-off between concept fidelity and text alignment. While existing methods rely on manual guidance, they often fail to represent intricate visual details and lack scalability. We introduce ConceptPrism, a framework that extracts shared features exclusively through cross-image comparison without external information. We jointly optimize a target token and image-wise residual tokens via reconstruction and exclusion losses. By suppressing shared information in residual tokens, the exclusion loss creates an information vacuum that forces the target token to capture the common concept. Extensive evaluations demonstrate that ConceptPrism achieves accurate concept disentanglement and significantly improves overall performance across diverse and complex visual concepts. The code is available at this https URL.

Comments: Accepted to CVPR 2026

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2602.19575 [cs.CV]

(or arXiv:2602.19575v2 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2602.19575

arXiv-issued DOI via DataCite

Submission history

From: Minseo Kim [view email] [v1] Mon, 23 Feb 2026 07:46:19 UTC (3,249 KB) [v2] Mon, 30 Mar 2026 12:03:06 UTC (3,688 KB)