How to make your CPU as fast as a GPU - Advances in Sparsity w/ Nir Shavit
#ai #sparsity #gpu
Sparsity is awesome, but only recently has it become possible to properly handle sparse models at good performance. Neural Magic does exactly this, using a plain CPU. No specialized hardware needed, just clever algorithms for pruning and forward-propagation of neural networks. Nir Shavit and I talk about how this is possible, what it means in terms of applications, and why sparsity should play a much larger role in the Deep Learning community.
Sponsor: AssemblyAI
Link: https://www.assemblyai.com/?utm_source=youtube&utm_medium=social&utm_campaign=yannic_autochapters
Check out Neural Magic: https://neuralmagic.com/
and DeepSparse: https://github.com/neuralmagic/deepsparse
OUTLINE:
0:00 Introduction
1:08 Sponsor: AssemblyAI
2:50 Start of Interview
4:15 How the NIR company was founded?
5:10 What is Sparsity about?
9:30 Link between the human brain and sparsity
12:10 Where should the extra resource that the human brain doesn't have go?
14:40 Analogy for Sparse Architecture
16:48 Possible future for Sparse Architecture as standard architure for Neural Networks
20:08 Pruning & Sparsification
22:57 What keeps us from building sparse models?
25:34 Why are GPUs so unsuited for sparse models?
28:47 CPU and GPU in connection with memory
30:14 What Neural Magic does?
32:54 How do you deal with overlaps in tensor columns?
33:41 The best type of sparsity to execute tons of CPU
37:24 What kind of architecture would make the best use out of a combined system of CPUs and GPUs?
41:04 Graph Neural Networks in connection to sparsity
43:04 Intrinsic connection between the Sparsification of Neural Networks, Non Layer-Wise Computation, Blockchain Technology, Smart Contracts and Distributed Computing
45:23 Neural Magic's target audience
48:16 Is there a type of model where it works particularly well and the type where it doesn't?
Links:
Homepage: https://ykilcher.com
Merch: https://ykilcher.com/merch
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://ykilcher.com/discord
LinkedIn: https://www.linkedin.com/in/ykilcher
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
-
1:23:07
Graham Allen
9 hours agoTrump Drops A NUKE On Libertarians!!! + A New Frontrunner For VP?!
57.4K124 -
LIVE
The Michelle Moore Show
6 hours ago(Tues, May 28 @ 3p CST/4p EST) Guests, Jack and Margy Flynn: The Michelle Moore Show (May 28, 2024)
1,632 watching -
21:30
Stephen Gardner
2 hours agoEx-CIA just SHOCKED the world with TERRIFYING new Trump prediction!
5.61K12 -
13:31
Breaking Points
4 hours agoSouth Park EXPOSES US Healthcare DISASTER
15.7K21 -
4:39
Standpoint with Gabe Groisman
4 hours agoClip: Amb Bolton on Isolationism - Protecting America's Interests Does Not End at Our Borders
16.2K6 -
LIVE
ItsMossy
3 hours ago🍀MULTIVERSUS IS BACK🍀!MENU !DISCORD !CLIP🍀
116 watching -
2:38:30
Matt Kim
18 hours agoThat DIDN'T go as planned...
25.1K14 -
43:05
Kimberly Guilfoyle
6 hours agoSPECIAL COVERAGE OF CLOSING ARGUMENTS, Live with Lawyer Mark Paoletta
27.8K22 -
2:46:24
Spittin' Chiclets
7 hours agoSpittin' Chiclets Episode 503: Featuring Special Guests
21.9K1 -
1:55:53
vivafrei
6 hours agoTrump Trial Closing Arguments! E. Jean Carroll to Sue AGAIN? Libertarian BABIES! & MORE! Viva Frei
72K51