Optimizing AI Systems for Efficient Processing

In episode 118 of The Gradient Podcast, Daniel Bashir speaks to Andrew Lee.

Andrew is co-founder and CEO of Shortwave, a company dedicated to building a better product experience for email, particularly by leveraging AI. He previously co-founded and was CTO at Firebase.

Have suggestions for future podcast guests (or other feedback)? Let us know here or reach Daniel at editor@thegradient.pub

Subscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter

Outline:

* (00:00) Intro

* (01:43) Andrew’s previous work, Firebase

* (04:48) Benefits of lacking experience in building Firebase

* (08:55) On “abstract reasoning” vs empirical capabilities

* (10:30) Shortwave’s AI system as a black box

* (11:55) Motivations for Shortwave

* (17:10) Why is Google not innovating on email?

* (21:53) Shortwave’s overarching product vision and pivots

* (27:40) Shortwave AI features

* (33:20) AI features for email and security concerns

* (35:45) Shortwave’s AI Email Assistant + architecture

* (43:40) Issues with chaining LLM calls together

* (45:25) Understanding implicit context in utterances, modularization without loss of context

* (48:56) Performance for AI assistant, batching and pipelining

* (55:10) Prompt length

* (57:00) On shipping fast

* (1:00:15) AI improvements that Andrew is following

* (1:03:10) Outro

Links:

* Andrew’s blog and Twitter

* Shortwave

* Introducing Ghostwriter

* Everything we shipped for AI Launch Week

* A deep dive into the world’s smartest email AI