Curved representational Bregman divergences and their applications

View PDF HTML (experimental)

Abstract:By analogy to the terminology of curved exponential families in statistics, we define curved Bregman divergences as Bregman divergences restricted to non-affine parameter subspaces and sub-dimensional Bregman divergences when the restrictions are affine. A common example of curved Bregman divergence is the cosine dissimilarity between normalized vectors: a curved squared Euclidean divergence. We prove that the barycenter of a finite weighted set of parameters under a curved Bregman divergence amounts to the right Bregman projection onto the non-affine subspace of the barycenter with respect to the full Bregman divergence, and interpret a generalization of the weighted Bregman centroid of $n$ parameters as a $n$-fold sub-dimensional Bregman divergence. We demonstrate the significance of curved Bregman divergences with several examples: (1) symmetrized Bregman divergences, (2) pointwise symmetrized Bregman divergences, and (3) the Kullback-Leibler divergence between circular complex normal distributions. We explain how to reparameterize sub-dimensional Bregman divergences on simplicial sub-dimensional domains. We then consider monotonic embeddings to define representational curved Bregman divergences and show that the $\alpha$-divergences are representational curved Bregman divergences with respect to $\alpha$-embeddings of the probability simplex into the positive measure cone. As an application, we report an efficient method to calculate the intersection of a finite set of $\alpha$-divergence spheres. As an application, we report an efficient method to calculate the intersection of a finite set of $\alpha$-divergence spheres.

Comments: 33 pages, 11 figures

Subjects:

Information Theory (cs.IT); Machine Learning (cs.LG)

Cite as: arXiv:2504.05654 [cs.IT]

(or arXiv:2504.05654v5 [cs.IT] for this version)

https://doi.org/10.48550/arXiv.2504.05654

arXiv-issued DOI via DataCite

Submission history

From: Frank Nielsen [view email] [v1] Tue, 8 Apr 2025 04:05:12 UTC (261 KB) [v2] Wed, 25 Jun 2025 07:53:44 UTC (407 KB) [v3] Wed, 17 Dec 2025 06:46:48 UTC (1,064 KB) [v4] Tue, 17 Feb 2026 02:17:07 UTC (1,170 KB) [v5] Fri, 27 Mar 2026 01:58:18 UTC (1,173 KB)