I don't think there have been any research breakthroughs that have enabled this quote-unquote AI summer and I hate the summer winter thing. It's more about how the pace of progress will change and where it will have impact over time. People look at certain moments as being important, but in some sense, even fully connected networks are pretty powerful. We've just made them bigger. Bigger is not going to be the long-term future of deep learning, but it is one way to get to these exciting capabilities.
Jonathan Frankle, incoming Harvard Professor and Chief Scientist at MosaicML, is focused on reducing the cost of training neural nets. He received his PhD at MIT and his BSE and MSE from Princeton.
Jonathan has also been instrumental in shaping technology policy related to AI. He worked on a landmark facial recognition report while working as a Staff Technologist at the Center on Privacy and Technology at Georgetown Law.
Thanks to great guest Hina Dixit from Samsung NEXT for the introduction to Jonathan!
Listen and learn...
- Why we can't understand deep neural nets like we can understand biology or physics.
- Jonathan's "lottery hypothesis" that neural nets are 50-90% bigger than they need to be...but it's hard to find which parts aren't necessary.
- How researchers are finding ways to reduce the cost and complexity of training neural nets.
- Why we shouldn't expect another AI winter because "it's now a fundamental substrate of research".
- Which AI problems are a good fit for deep learning... and which ones aren't.
- What's the role for regulation in enforcing responsible use of AI.
- How Jonathan and his CTO Hanlin Tang at MosaicML create a culture that fosters responsible use of AI.
- Why Jonathan says "...We're building a ladder to the moon if we think today's neural nets will lead to AGI."
References in this episode...