2min chapter

108: The Puppet Master

The Unmade Podcast

CHAPTER

The Coffee Machine

I've just gotten used to its huge gravitational influence on the planet now, and therefore, you know, i'm taking it for granted. It's like a back up water tank, cause there's so much in em. And of course, the surfings still going strong. Twice this week end that's just gone past. Ah, you're addicted. I am really quite addicted. If i drop them lick at a service station or something, and they just wait there for an hour while you gojust randomon the side of it,. ranway down to some chip shop. None of them are from k f c, because that's part of the rulesh yes. Butit turns

00:00

Transcript

Episode notes

Speaker 2

So what is the thing that's controversial why would why do people not think that

Speaker 1

that should be part of the algebra. So a lot of people so one of the things that we we spent a lot of time doing was optimizing for this ability to transpose and to kind of treat rows and columns interchangeably we actually built a system that could handle hundreds of billions of columns and there just aren't that many systems in the world that can handle that many columns just because like you have to have metadata for all of your columns in most systems and we had a way basically of kind of having the metadata be inferred at runtime which is which is another another key aspect of the data frames but having these kind of like short but really really fat or wide tables is not something that's extremely common in the kind of world of databases and so that's that's kind of why it ended up being like okay the controversial if you would find this image you would find

Speaker 2

this machine learning though 100%

Speaker 1

yeah 100% yes it's extremely common in machine learning but databases don't like that's changing these days because databases are starting to do a lot more but databases have historically not been well optimized for doing machine learning workloads and so this idea of the transpose and the same world I think is yeah I mean databases would just crash if you try to throw throw a hundred billion columns into them and

Speaker 2

you