Unsupervised Learning Ep 47: Chief AI Scientist of Databricks Jonathan Frankle on Why New Model Architectures are Unlikely, When to Pre-Train or Fine Tune, and Hopes for Future AI Policy
17 snips
Nov 12, 2024 Jonathan Frankle, Chief AI Scientist at Databricks, brings deep insight into the fast-paced world of AI. He discusses the evolution of AI models, favoring transformers over LSTMs, and shares strategic insights from the merger of Mosaic and Databricks. Frankle emphasizes the importance of effective AI evaluation benchmarks and customer collaboration in developing AI solutions. Ethical considerations and responsible AI policy also take center stage, as he highlights the need for transparency and community engagement in the rapidly evolving landscape.
AI Snips
Chapters
Transcript
Episode notes
Creative Team Incentives
- Jonathan Frankle incentivizes his team with fun rewards like swords and hair dyeing rather than just money or threats.
- These creative incentives help motivate his team to reach difficult performance goals rapidly.
Transformer Architecture Dominance
- New AI architectures are rare and transformers likely remain dominant because good architectures are hard to find.
- Transformers simplified models compared to LSTMs, driving the current state of NLP models over incremental BERT tweaks.
Start Small, Then Scale AI
- Start AI projects by prompting existing models with small experiments to test viability.
- Iterate by improving data, evaluations, and then move towards fine-tuning or pre-training only when justified by ROI.