Automatic Laplace Collapsed Sampling: Scalable Marginalisation of Latent Parameters via Automatic Differentiation

View PDF HTML (experimental)

Abstract:We present Automatic Laplace Collapsed Sampling (ALCS), a general framework for marginalising latent parameters in Bayesian models using automatic differentiation, which we combine with nested sampling to explore the hyperparameter space in a robust and efficient manner. At each nested sampling likelihood evaluation, ALCS collapses the high-dimensional latent variables $z$ to a scalar contribution via maximum a posteriori (MAP) optimisation and a Laplace approximation, both computed using autodiff. This reduces the effective dimension from $d_\theta + d_z$ to just $d_\theta$, making Bayesian evidence computation tractable for high-dimensional settings without hand-derived gradients or Hessians, and with minimal model-specific engineering. The MAP optimisation and Hessian evaluation are parallelised across live points on GPU-hardware, making the method practical at scale. We also show that automatic differentiation enables local approximations beyond Laplace to parametric families such as the Student-$t$, which improves evidence estimates for heavy-tailed latents. We validate ALCS on a suite of benchmarks spanning hierarchical, time-series, and discrete-likelihood models and establish where the Gaussian approximation holds. This enables a post-hoc ESS diagnostic that localises failures across hyperparameter space without expensive joint sampling.

Comments: 28 Pages, 7 Figures. Comments welcome

Subjects:

Machine Learning (cs.LG); Instrumentation and Methods for Astrophysics (astro-ph.IM); Methodology (stat.ME)

Cite as: arXiv:2603.26644 [cs.LG]

(or arXiv:2603.26644v1 [cs.LG] for this version)

https://doi.org/10.48550/arXiv.2603.26644

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Toby Lovick [view email] [v1] Fri, 27 Mar 2026 17:47:45 UTC (149 KB)