Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks
arXiv:2307.07753v2 Announce Type: replace-cross Abstract: In this work, we propose a novel prior learning method for advancing generalization and uncertainty estimation in deep neural networks. The key idea is to exploit scalable and structured posteriors of neural networks as informative priors with generalization guarantees. Our learned priors provide expressive probabilistic representations at large scale, like Bayesian counterparts of pre-trained models on ImageNet, and further produce non-vacuous generalization bounds. We also extend this idea to a continual learning framework, where the — Dominik Schnaus, Jongseok Lee, Daniel Cremers, Rudolph Triebel
View PDF HTML (experimental)
Abstract:In this work, we propose a novel prior learning method for advancing generalization and uncertainty estimation in deep neural networks. The key idea is to exploit scalable and structured posteriors of neural networks as informative priors with generalization guarantees. Our learned priors provide expressive probabilistic representations at large scale, like Bayesian counterparts of pre-trained models on ImageNet, and further produce non-vacuous generalization bounds. We also extend this idea to a continual learning framework, where the favorable properties of our priors are desirable. Major enablers are our technical contributions: (1) the sums-of-Kronecker-product computations, and (2) the derivations and optimizations of tractable objectives that lead to improved generalization bounds. Empirically, we exhaustively show the effectiveness of this method for uncertainty estimation and generalization.
Comments: Accepted to ICML 2023
Subjects:
Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML)
Cite as: arXiv:2307.07753 [cs.LG]
(or arXiv:2307.07753v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2307.07753
arXiv-issued DOI via DataCite
Submission history
From: Dominik Schnaus [view email] [v1] Sat, 15 Jul 2023 09:24:33 UTC (634 KB) [v2] Mon, 30 Mar 2026 02:22:03 UTC (640 KB)
Sign in to highlight and annotate this article
Conversation starters
Daily AI Digest
Get the top 5 AI stories delivered to your inbox every morning.
More about
researchpaperarxiv
![[D] SIGIR 2026 review discussion](https://images.unsplash.com/photo-1558494949-ef010cbdcc31?w=600&q=80)
[D] SIGIR 2026 review discussion
SIGIR 2026 results will be released soon, so I’m opening this thread to discuss reviews and outcomes. Unfortunately, all the papers I reviewed (4 full papers and 6 short papers) were rejected. It seems like this year has been particularly tough for everyone. submitted by /u/snu95 [link] [comments]
Knowledge Map
Connected Articles — Knowledge Graph
This article is connected to other articles through shared AI topics and tags.
More in Research Papers
VRUD: A Drone Dataset for Complex Vehicle-VRU Interactions within Mixed Traffic
arXiv:2604.01134v1 Announce Type: cross Abstract: The Operational Design Domain (ODD) of urbanoriented Level 4 (L4) autonomous driving, especially for autonomous robotaxis, confronts formidable challenges in complex urban mixed traffic environments. These challenges stem mainly from the high density of Vulnerable Road Users (VRUs) and their highly uncertain and unpredictable interaction behaviors. However, existing open-source datasets predominantly focus on structured scenarios such as highways or regulated intersections, leaving a critical gap in data representing chaotic, unstructured urban environments. To address this, this paper proposes an efficient, high-precision method for constructing drone-based datasets and establishes the Vehicle-Vulnerable Road User Interaction Dataset (VRUD
Discussion
Sign in to join the discussion
No comments yet — be the first to share your thoughts!