Premium Only Content
 
			ActInf MathStream 012.1 ~ ELBOing Stein: Variational Bayes with Stein Mixture Inference, Ola Rønning
Ola Rønning
ELBOing Stein: Variational Bayes with Stein Mixture Inference
https://arxiv.org/abs/2410.22948
Ola Rønning, Eric Nalisnick, Christophe Ley, Padhraic Smyth, Thomas Hamelryck
Stein variational gradient descent (SVGD) [Liu and Wang, 2016] performs approximate Bayesian inference by representing the posterior with a set of particles. However, SVGD suffers from variance collapse, i.e. poor predictions due to underestimating uncertainty [Ba et al., 2021], even for moderately-dimensional models such as small Bayesian neural networks (BNNs). To address this issue, we generalize SVGD by letting each particle parameterize a component distribution in a mixture model. Our method, Stein Mixture Inference (SMI), optimizes a lower bound to the evidence (ELBO) and introduces user-specified guides parameterized by particles. SMI extends the Nonlinear SVGD framework [Wang and Liu, 2019] to the case of variational Bayes. SMI effectively avoids variance collapse, judging by a previously described test developed for this purpose, and performs well on standard data sets. In addition, SMI requires considerably fewer particles than SVGD to accurately estimate uncertainty for small BNNs. The synergistic combination of NSVGD, ELBO optimization and user-specified guides establishes a promising approach towards variational Bayesian inference in the case of tall and wide data.
Active Inference Institute information:
Website: https://www.activeinference.institute/
Activities: https://activities.activeinference.institute/
Discord: https://discord.activeinference.institute/
Donate: http://donate.activeinference.institute/
YouTube: https://www.youtube.com/c/ActiveInference/
X: https://x.com/InferenceActive
Active Inference Livestreams: https://video.activeinference.institute/
- 	
				 1:43:29 1:43:29Active Inference Institute1 month agoActInf GuestStream 121.1 ~ Conscious active inference (Wiest and Puniani)161
- 	
				 35:45 35:45Stephen Gardner12 hours ago🔴BREAKING: Election Auditor EXPOSES Democrat Election Fraud Evidence!27.9K46
- 	
				 15:48 15:48Sponsored By Jesus Podcast16 hours agoHow to Stop Being JEALOUS | When Comparison Steals Your Joy8.11K11
- 	
				 DVR DVRDLDAfterDark6 hours ago $6.24 earnedDon't Worry - Things Will Get SO Much Worse! Sometimes It Be Like That17K3
- 	
				 25:41 25:41Robbi On The Record12 hours ago $2.22 earnedThe Billion-Dollar Lie Behind OnlyFans “Empowerment” (Her Testimony Will Shock You) | part II11.1K8
- 	
				 12:22 12:22Cash Jordan4 hours ago"CHICAGO MOB" Fights Back... "ZERO MERCY" Marines DEFY Judge, SMASH ILLEGALS22.3K38
- 	
				 46:58 46:58Brad Owen Poker16 hours agoI Make QUAD ACES!!! BIGGEST Bounty Of My Life! Turning $0 Into $10,000+! Must See! Poker Vlog Ep 32312.9K6
- 	
				 2:52:28 2:52:28TimcastIRL7 hours agoSTATE OF EMERGENCY Declared Over Food Stamp CRISIS, Judge Says Trump MUST FUND SNAP | Timcast IRL233K128
- 	
				 3:22:45 3:22:45Tundra Tactical14 hours ago $20.28 earned🚨Gun News and Game Night🚨 ATF Form 1 Changes, BRN-180 Gen 3 Issues??, and Battlefield 6 Tonight!38.6K3
- 	
				 1:45:13 1:45:13Glenn Greenwald10 hours agoJD Vance Confronted at Turning Point about Israel and Massie; Stephen Miller’s Wife Screams “Racist” and Threatens Cenk Uygur with Deportation; Rio's Police Massacre: 120 Dead | SYSTEM UPDATE #540115K163