2min chapter

Machine Learning Street Talk (MLST) cover image

Is ChatGPT an N-gram model on steroids?

Machine Learning Street Talk (MLST)

CHAPTER

Intro

The chapter explores the effectiveness of hash tables in enhancing transformer predictions, highlighting that 78% of searches produce favorable results. The speaker shares their journey from mathematics to machine learning and discusses the significance of patterns in their intellectual development.

00:00
Speaker 1
Anyways, you have this hash table of 400 things, and you're going to ask, oh, is there something from this hash table of 400 things that is close to my transformer prediction? Right? I'm not going to tell you in advance, I can't predict for you in advance which one it is. I'm just going to say, is there something that matches it? And the kind of result that I got was that 78% of the time, in some sense, you get like a good match. Okay. I'm a machine learning researcher at Google DeepMind. In a former life, I was a mathematician. Like you, Tim, I also have a podcast called the Cartesian Cafe. Hello, everyone, this is Tim Nguyen, and welcome to the Cartesian Cafe, the podcast where an expert guest and I map out a scientific subject in detail. I'm a mathematician who has his hands in several different things. I obtained my PhD at MIT, had a career in academia, as a researcher in mathematical physics and gauge theory, and then I transitioned to industry. Someone once asked me how I proved that my brain conjecture, I said, well, it's just a homeomorphism, I just had to say where every point went. And I did. Yeah, everyone should subscribe to the Cartesian Cafe. Grant Sanderson's been on Can't Complain, so he's the biggest math YouTuber.
Speaker 2
I was definitely one of those kids where from a very young age I self-identified as liking it. I think a lot of that probably had to do with games that my dad would play with me
Speaker 1
that, you know, like a lot of dads just wanted his kids to be interested in science and math and the world and things like that. And I just loved patterns. I've had now two Fields Medalists, Richard Borchards and Michael Friedman. So yeah, been very lucky to have a star-studded guest. If you had to attribute your success and mathematics to any particular qualities, can you name any?

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.
App store bannerPlay store banner

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

Share
& Export

Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode