No Priors: Artificial Intelligence | Technology | Startups State Space Models and Real-time Intelligence with Karan Goel and Albert Gu from Cartesia
11 snips
Jun 27, 2024 
Karan Goel, co-founder of Cartesia and inventor of State Space Models at Stanford AI Lab, joins forces with fellow co-founder Albert Gu to discuss groundbreaking advancements in real-time intelligence. They dive into their product Sonic, a text-to-speech engine boasting unparalleled speed and quality. The duo contrasts State Space Models with traditional transformers, emphasizing the efficiency of their innovations. Listeners will also enjoy insights on the future of intelligent systems, emotional speech tech, and the aesthetics of tackling research challenges.
AI Snips
Chapters
Transcript
Episode notes
Cloud Disk Woes
- Karan Goel and Albert Gu's first project involved filling up Google Cloud disk space.
- Goel would expand the disk by one terabyte repeatedly, frustrating Gu.
SSMs and S4
- Albert Gu worked on sequence modeling and recurrent models, particularly state-space models (SSMs).
- He and Goel collaborated on the S4 model, demonstrating SSMs' effectiveness, leading to further research like Mamba for language modeling.
SSMs vs. Transformers
- SSMs, unlike transformers, process sequences incrementally, compressing information into a state and updating it with new data.
- Different SSM variants excel with different data types; early models were good for raw signals but not text, while newer ones like Mamba are better at text modeling but still face trade-offs.