The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Rethinking Model Size: Train Large, Then Compress with Joseph Gonzalez - #378
7 snips
May 25, 2020 
Joseph Gonzalez, Assistant Professor at UC Berkeley, joins to discuss his innovative approach to model efficiency. He delves into his research on training large models followed by compressing them, questioning the balance between model size and computational resource use. The talk includes insights on rapid architecture iteration and how larger models can still be efficient. He also shares strategies like weight pruning and dynamic querying that enhance performance without excessive resource investment, making advanced AI more accessible.
AI Snips
Chapters
Transcript
Episode notes
Flipping Helicopters to Fire Monitoring
- Joseph Gonzalez initially wanted to research flipping helicopters with neural networks.
- His advisor suggested focusing on sensor networks for fire monitoring using statistical methods instead.
From MATLAB to GraphLab
- While working on sensor networks, Gonzalez realized his interest in parallel computing.
- This led him to develop GraphLab, a system for large-scale graph analysis.
Systems: The Unsung Hero of Deep Learning
- The deep learning revolution was largely driven by scaling models and making them easier to use, not just new model architectures.
- Tools like Theano and Caffe made model building more accessible, accelerating progress.