The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence) Trends in Natural Language Processing with Sameer Singh - #445
Jan 7, 2021
Sameer Singh, Assistant Professor at UC Irvine and an expert in natural language processing, dives into the latest trends in NLP. He discusses the profound impact of GPT-3 and Transformer models on the field. The conversation highlights the complexities of evaluating language models and their practical vulnerabilities. Sameer brings attention to the limitations of current models in achieving true natural language understanding. Additionally, he shares insights on the intersection of language and vision models, shaping the future of NLP.
AI Snips
Chapters
Transcript
Episode notes
NLP Self-Reflection
- 2020 was a year of self-reflection for NLP, focusing on the capabilities and limitations of powerful pre-trained language models.
- Researchers questioned how these models work and their potential for wider deployment.
GPT-3 Few-Shot Performance
- GPT-3's few-shot performance is noteworthy, allowing it to perform tasks like sentiment analysis without further training.
- Providing a few examples at the beginning of a document enables GPT-3 to generate correct labels.
AI Dungeon
- AI Dungeon uses GPT-2/3 to create dynamic text-based adventure games.
- Players' actions influence the story's progression through GPT-generated content.