Premium Only Content

ActInf MathStream 012.1 ~ ELBOing Stein: Variational Bayes with Stein Mixture Inference, Ola Rønning
Ola Rønning
ELBOing Stein: Variational Bayes with Stein Mixture Inference
https://arxiv.org/abs/2410.22948
Ola Rønning, Eric Nalisnick, Christophe Ley, Padhraic Smyth, Thomas Hamelryck
Stein variational gradient descent (SVGD) [Liu and Wang, 2016] performs approximate Bayesian inference by representing the posterior with a set of particles. However, SVGD suffers from variance collapse, i.e. poor predictions due to underestimating uncertainty [Ba et al., 2021], even for moderately-dimensional models such as small Bayesian neural networks (BNNs). To address this issue, we generalize SVGD by letting each particle parameterize a component distribution in a mixture model. Our method, Stein Mixture Inference (SMI), optimizes a lower bound to the evidence (ELBO) and introduces user-specified guides parameterized by particles. SMI extends the Nonlinear SVGD framework [Wang and Liu, 2019] to the case of variational Bayes. SMI effectively avoids variance collapse, judging by a previously described test developed for this purpose, and performs well on standard data sets. In addition, SMI requires considerably fewer particles than SVGD to accurately estimate uncertainty for small BNNs. The synergistic combination of NSVGD, ELBO optimization and user-specified guides establishes a promising approach towards variational Bayesian inference in the case of tall and wide data.
Active Inference Institute information:
Website: https://www.activeinference.institute/
Activities: https://activities.activeinference.institute/
Discord: https://discord.activeinference.institute/
Donate: http://donate.activeinference.institute/
YouTube: https://www.youtube.com/c/ActiveInference/
X: https://x.com/InferenceActive
Active Inference Livestreams: https://video.activeinference.institute/
-
1:43:29
Active Inference Institute
13 days agoActInf GuestStream 121.1 ~ Conscious active inference (Wiest and Puniani)
10 -
15:10
Dr Disrespect
1 day agoI FINALLY Beat Baby Steps
98.6K14 -
2:38:29
MattMorseTV
15 hours ago $12.09 earned🔴Trump's Counter-Antifa CONFERENCE.🔴
15.9K87 -
4:34:25
Side Scrollers Podcast
19 hours agoTwitch CEO Testifies in Congress + Hasan Piker Accused of DOG Abuse + More | Side Scrollers
59.3K26 -
19:54
Forrest Galante
3 days agoPrivate Tour Of America's Best Marine Animal Facility
88.7K10 -
17:00
GritsGG
13 hours agoWon the Game Because of This Easter Egg w/ Bobby Poff!
12.1K -
17:36
The Pascal Show
10 hours ago $1.82 earned'THEY ARE GONNA K*LL ME!' Candace Owens Reveals Charlie Kirk Feared For His Life Days Before Murder!
9.14K14 -
LIVE
Lofi Girl
2 years agoSynthwave Radio 🌌 - beats to chill/game to
255 watching -
38:27
TruthStream with Joe and Scott
3 days agoA roundtable with Lisa, Carole and Michelle. Our travels through Spain and Ireland #497
24.5K3 -
2:05:03
Badlands Media
15 hours agoDevolution Power Hour Ep. 396: The Machine Cracks – CIA Networks, Color Revolutions & Trump’s Playbook
152K24