Interviewing Tim Dettmers on open-source AI: Agents, scaling, quantization and what's next
Interconnects
Intro
This chapter explores the evolution of open-source AI and its potential advantages over closed models. It discusses AI agents, scaling, and predictions for the future, highlighting the significance of open-source contributions in AI development.
Tim Dettmers does not need an introduction for most people building open-source AI. If you are part of that minority, you’re in for a treat. Tim is the lead developer behind most of the open-source tools for quantization: QLoRA, bitsandbytes, 4 and 8 bit inference, and plenty more. He recently finished his Ph.D. at the University of Washington, is now a researcher at the Allen Institute for AI, and is starting as a professor at Carnegie Mellon University in fall of 2025.
Tim is a joy to talk to. He thinks independently on all the AI issues of today, bringing new perspectives that challenge the status quo. At the same time, he’s sincere and very helpful to work with, working hard to uplift those around him and the academic community. There’s a reason he’s so loved in the open-source AI community.
Find more about Tim on his Twitter or Google Scholar. He also has a great blog where he talks about things like which GPUs to buy and which grad school to choose.
Listen on Apple Podcasts, Spotify, YouTube, and where ever you get your podcasts. For other Interconnects interviews, go here.
Show Notes
Here's a markdown list of companies, people, projects, research papers, and other key named entities mentioned in the transcript:
* QLoRA
* Llama 3
* Claude (AI assistant by Anthropic)
* Transformers (Hugging Face library)
* Gemma (Google's open weight language model)
* Blackwell (NVIDIA GPU architecture)
* Branch Train Merge (research paper)
* "ResNets do iterative refinement on features" (research paper)
* CIFAR-10 and CIFAR-100 (computer vision datasets)
* Lottery Ticket Hypothesis (research paper)
* TRL (Transformer Reinforcement Learning) by Hugging Face
* Tim's work on quantization (this is just one example)
Timestamps
* [00:00:00] Introduction and background on Tim Dettmers
* [00:01:53] Future of open source AI models
* [00:09:44] SWE Bench and evaluating AI systems
* [00:13:33] Using AI for coding, writing, and thinking
* [00:16:09] Academic research with limited compute
* [00:32:13] Economic impact of AI
* [00:36:49] User experience with different AI models
* [00:39:42] O1 models and reasoning in AI
* [00:46:27] Instruction tuning vs. RLHF and synthetic data
* [00:51:16] Model merging and optimization landscapes
* [00:55:08] Knowledge distillation and optimization dynamics
* [01:01:55] State-space models and transformer dominance
* [01:06:00] Definition and future of AI agents
* [01:09:20] The limit of quantization
Transcript and full details: https://www.interconnects.ai/p/tim-dettmers
Get Interconnects (https://www.interconnects.ai/)...
... on YouTube: https://www.youtube.com/@interconnects
... on Twitter: https://x.com/interconnectsai
... on Linkedin: https://www.linkedin.com/company/interconnects-ai
... on Spotify: https://open.spotify.com/show/2UE6s7wZC4kiXYOnWRuxGv
… on Apple Podcasts: https://podcasts.apple.com/us/podcast/interconnects/id1719552353
This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.interconnects.ai/subscribe
The AI-powered Podcast Player
