“A lot of people want local inference. And somehow they assume that the problem is solved now because we’re going to have open source models, we’ll be able to run them on our computers, and so we won’t need all the hardware in the data center.”
But from an overall allocation perspective, actually it’s pretty terrible to see everyone have a finite amount of DRAM in the world in their own computer and it’s used five percent of the time.
You want maximum utilization in the cloud if you’re trying to manage this very supply-constrained environment. And somehow everyone’s – a lot of people’s intuition is the opposite of that.
Gn bussni
