A training-free framework for high-fidelity appearance transfer via diffusion transformers

View PDF HTML (experimental)

Abstract:Diffusion Transformers (DiTs) excel at generation, but their global self-attention makes controllable, reference-image-based editing a distinct challenge. Unlike U-Nets, naively injecting local appearance into a DiT can disrupt its holistic scene structure. We address this by proposing the first training-free framework specifically designed to tame DiTs for high-fidelity appearance transfer. Our core is a synergistic system that disentangles structure and appearance. We leverage high-fidelity inversion to establish a rich content prior for the source image, capturing its lighting and micro-textures. A novel attention-sharing mechanism then dynamically fuses purified appearance features from a reference, guided by geometric priors. Our unified approach operates at 1024px and outperforms specialized methods on tasks ranging from semantic attribute transfer to fine-grained material application. Extensive experiments confirm our state-of-the-art performance in both structural preservation and appearance fidelity.

Subjects:

Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2603.26767 [cs.CV]

(or arXiv:2603.26767v1 [cs.CV] for this version)

https://doi.org/10.48550/arXiv.2603.26767

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Shengrong Gu [view email] [v1] Tue, 24 Mar 2026 07:53:50 UTC (34,959 KB)