Practical AI: Machine Learning, Data Science, LLM

Mamba & Jamba

Apr 24, 2024

41:15

Snipd AI

Yoav, AI21's co-founder discusses Jamba, a groundbreaking SSM-Transformer open model and the evolution of AI21's language models. They explore Meta's llama 3 model, architecting AI systems, and the release of Jamba as the first open-source model for community innovation.

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Jamba model combines non-transformer goodness with attention layers for high performance and efficiency.

AI21 focuses on enterprise value with AI language models, aiming for true task comprehension and specialized systems.

Deep dives

AI 21's Background and Mission

AI 21's journey began over six years ago with a vision that modern AI requires more than just deep learning. They believe that intelligence, specifically reasoning, goes beyond statistical methods. Starting with Jurassic One, a model exceeding GPT-3's scale, AI 21 evolved into focusing on large language models. Language, intricate and nuanced, became their lens into the human mind, unlocking potential in enterprise data, predominantly text-based.

Model Architecture: Jamba a Solid Space State hybrid

01:11

Understanding Foundation Models and Open Source Initiatives

01:20

Introduction

2min

Evolution and Practical Applications of AI21's Language Models

13min

Analysis of Meta's Latest Language Model Release and Performance Comparison

2min

Architecting AI Systems and Model Evolution

18min

Exploring the Release and Future Potential of the Jamba Model in AI Development

7min

First there was Mamba… now there is Jamba from AI21. This is a model that combines the best non-transformer goodness of Mamba with good ‘ol attention layers. This results in a highly performant and efficient model that AI21 has open sourced! We hear all about it (along with a variety of other LLM things) from AI21’s co-founder Yoav.

Join the discussion

Changelog++ members save 3 minutes on this episode because they made the ads disappear. Join today!

Sponsors:

Fly.io – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs.
Changelog News – A podcast+newsletter combo that’s brief, entertaining & always on-point. Subscribe today.

Featuring:

Yoav Shoham – Twitter, LinkedIn
Chris Benson – Twitter, GitHub, LinkedIn, Website
Daniel Whitenack – Twitter, GitHub, Website

Show Notes:

Something missing or broken? PRs welcome!

Get the Snipd
podcast app

Unlock the knowledge in podcasts with the podcast player of the future.

AI-powered
podcast player

Listen to all your favourite podcasts with AI-powered features

Discover
highlights

Listen to the best highlights from the podcasts you love and dive into the full episode

Save any
moment

Hear something you like? Tap your headphones to save it with AI-generated key takeaways

Share
& Export