The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Accelerating Innovation with AI at Scale with David Carmona - #465
Mar 18, 2021
David Carmona, General Manager of Artificial Intelligence & Innovation at Microsoft, dives into AI at scale and the evolution of natural language processing. He shares insights on the shift toward massive models and how attention mechanisms are revolutionizing understanding. Discussing the importance of ethical AI, he also explores the journey from fine-tuning to zero-shot learning, enhancing model adaptability. Their integration into Microsoft products enriches capabilities like semantic search and document summarization, highlighting AI's transformative role in technology.
AI Snips
Chapters
Transcript
Episode notes
AI at Scale: A Paradigm Shift
- AI at Scale is a huge shift in AI development, driven by massive models.
- These models offer new business opportunities beyond just technical achievements.
NLP's ImageNet Moment
- NLP is experiencing an "ImageNet moment" with massive models like ResNet.
- Parameter count has rapidly increased 10x with each new model generation.
Transformer Architecture: The Key to Scalability
- Transformer architecture allows parallel processing, breaking scalability limits of recurrent networks.
- Positional embedding and self-attention are key for transformers to understand word relationships.