Deep learning works best in cases where the data is pretty complicated. There's lots of redundancy that we can use to do things like, you know, speech to text or doing some kind of machine translation. So those are the settings that I look at. It's hard to predict the kind of what's coming next if I knew I'd already have done it in some sense.
Jonathan Frankle, incoming Harvard Professor and Chief Scientist at MosaicML, is focused on reducing the cost of training neural nets. He received his PhD at MIT and his BSE and MSE from Princeton.
Jonathan has also been instrumental in shaping technology policy related to AI. He worked on a landmark facial recognition report while working as a Staff Technologist at the Center on Privacy and Technology at Georgetown Law.
Thanks to great guest Hina Dixit from Samsung NEXT for the introduction to Jonathan!
Listen and learn...
- Why we can't understand deep neural nets like we can understand biology or physics.
- Jonathan's "lottery hypothesis" that neural nets are 50-90% bigger than they need to be...but it's hard to find which parts aren't necessary.
- How researchers are finding ways to reduce the cost and complexity of training neural nets.
- Why we shouldn't expect another AI winter because "it's now a fundamental substrate of research".
- Which AI problems are a good fit for deep learning... and which ones aren't.
- What's the role for regulation in enforcing responsible use of AI.
- How Jonathan and his CTO Hanlin Tang at MosaicML create a culture that fosters responsible use of AI.
- Why Jonathan says "...We're building a ladder to the moon if we think today's neural nets will lead to AGI."
References in this episode...