Revisiting Diffusion Model Predictions Through Dimensionality

View PDF HTML (experimental)

Abstract:Recent advances in diffusion and flow matching models have highlighted a shift in the preferred prediction target -- moving from noise ($\varepsilon$) and velocity (v) to direct data (x) prediction -- particularly in high-dimensional settings. However, a formal explanation of why the optimal target depends on the specific properties of the data remains elusive. In this work, we provide a theoretical framework based on a generalized prediction formulation that accommodates arbitrary output targets, of which $\varepsilon$-, v-, and x-prediction are special cases. We derive the analytical relationship between data's geometry and the optimal prediction target, offering a rigorous justification for why x-prediction becomes superior when the ambient dimension significantly exceeds the data's intrinsic dimension. Furthermore, while our theory identifies dimensionality as the governing factor for the optimal prediction target, the intrinsic dimension of manifold-bound data is typically intractable to estimate in practice. To bridge this gap, we propose k-Diff, a framework that employs a data-driven approach to learn the optimal prediction parameter k directly from data, bypassing the need for explicit dimension estimation. Extensive experiments in both latent-space and pixel-space image generation demonstrate that k-Diff consistently outperforms fixed-target baselines across varying architectures and data scales, providing a principled and automated approach to enhancing generative performance.

Comments: 19 pages, 5 figures

Subjects:

Machine Learning (cs.LG); Computer Vision and Pattern Recognition (cs.CV)

Cite as: arXiv:2601.21419 [cs.LG]

(or arXiv:2601.21419v2 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2601.21419

arXiv-issued DOI via DataCite

Submission history

From: Qing Jin [view email] [v1] Thu, 29 Jan 2026 08:56:55 UTC (1,553 KB) [v2] Fri, 27 Mar 2026 10:30:18 UTC (1,559 KB)