Joint Embedding Variational Bayes

View PDF HTML (experimental)

Abstract:We introduce Variational Joint Embedding (VJE), a reconstruction-free latent-variable framework for non-contrastive self-supervised learning in representation space. VJE maximizes a symmetric conditional evidence lower bound (ELBO) on paired encoder embeddings by defining a conditional likelihood directly on target representations, rather than optimizing a pointwise compatibility objective. The likelihood is instantiated as a heavy-tailed Student--(t) distribution on a polar representation of the target embedding, where a directional--radial decomposition separates angular agreement from magnitude consistency and mitigates norm-induced pathologies. The directional factor operates on the unit sphere, yielding a valid variational bound for the associated spherical subdensity model. An amortized inference network parameterizes a diagonal Gaussian posterior whose feature-wise variances are shared with the directional likelihood, yielding anisotropic uncertainty without auxiliary projection heads. Across ImageNet-1K, CIFAR-10/100, and STL-10, VJE is competitive with standard non-contrastive baselines under linear and (k)-NN evaluation, while providing probabilistic semantics directly in representation space for downstream uncertainty-aware applications. We validate these semantics through out-of-distribution detection, where representation-space likelihoods yield strong empirical performance. These results position the framework as a principled variational formulation of non-contrastive learning, in which structured feature-wise uncertainty is represented directly in the learned embedding space.

Subjects:

Machine Learning (cs.LG); Machine Learning (stat.ML)

Cite as: arXiv:2602.05639 [cs.LG]

(or arXiv:2602.05639v3 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2602.05639

arXiv-issued DOI via DataCite

Submission history

From: Amin Oji [view email] [v1] Thu, 5 Feb 2026 13:18:53 UTC (1,377 KB) [v2] Mon, 30 Mar 2026 02:14:43 UTC (2,672 KB) [v3] Tue, 31 Mar 2026 17:23:11 UTC (2,672 KB)