Console DevTools cover image

Cloud infra, with Kurt Mackey (Fly.io) - S04E11

Console DevTools

00:00

The Problem With AWS's Zero Database Latency

We've gotten a ton of, I think one interesting thing that's happened with models is that people have started to care about latency between the user and the server. So if we can shave 200 milliseconds of a model, this has gotten valuable. And so we're adding GPUs, just a hypothesis is people want to do inference close to their users the same way they want to run CPUs. But it also feels like, you know, it's that whole cell of pickaxe and the gold rush thing. Like we're not actually doing anything with this. We're just selling the silicon people happen to want.

Transcript
Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!
App store bannerPlay store banner
Get the app