The All-In Podcast · David Sacks on AI's exponential rate of progress

The All-In Podcast · David Sacks Explains How AI Will Go 1,000,000x in Four Years
"I would say the rate of progress is exponential right now on at least three key dimensions."
.
1) The models
"So number one is the algorithms themselves. The models are improving at a rate of, I don't know, 3-4x a year."
"They're not just getting faster and better, but qualitatively they're different."
"Remember, we started with pure LLM chatbots."
"Then we went to reasoning models."
.
"We didn't even get to the agents part of it yet, but that's the next big leap after reasoning models."
"We're just starting to scratch the surface there."
.
2) The chips
"Then you've got the chips."
"Depending on how you measure it, each generation of chips is probably 3-4x better than the last."
"It's not just the individual chips that are getting better, they're figuring out how to network them together."
"Like with NVL72, it's like a rack system to create much better performance at the datacenter level."
.
3) The compute:
"And that would be the third area where you're seeing basically exponential progress."
"Just look at the number of GPUs that are being deployed in datacenters."
"So when Elon first started training Grok, I think they had maybe 100K GPUs. Now they're up to 300K. They're on their way to a million. Same thing with OpenAI's data center, Stargate."
"And within a couple years they'll be at, I don't know, 5M GPUs, 10M GPUs?
.
How Sacks gets to 1,000,000x:
"The algorithms, the chips, and the datacenters are all improving or scaling at a rate of 3-4x a year."
"That's 10x every two years."
"Where people don't understand exponential progress is that if you're getting better at 10x every two years, that doesn't mean you'll be at 20x in four years."
"It means you'll be at a 100x."
"So you multiply those things together: the algorithms, the chips, and the raw compute that's available."
.
100x models 🧠
x 100x chips 💾
x 100x compute ⚡️
= 1,000,000x AI 🤖
.
"You're talking about 1,000,000x increase."
.
"Some of which will be captured in price reductions, some of it will be in the performance ceiling, and then some of it will just be in the overall amount of AI compute that's available to the economy."
.
"But the impact of this thing is gonna be absolutely massive."
"And I think people still don't even appreciate that fact because they don't understand exponential progress."

@theallinpod
https://x.com/theallinpod/status/1918715889530130838