The Thesis Review [06] Yoon Kim - Deep Latent Variable Models of Natural Language
Aug 28, 2020
Yoon Kim, a Research Scientist at the MIT-IBM AI Watson Lab, shares insights into his research on deep latent variable models and natural language processing. He discusses uncovering latent structures in language, including vector representations and grammar induction. Yoon explores the complexities of variational inference in generative models and the challenges faced in training these models. Additionally, he reflects on his coding practices and the role of luck and opportunity in navigating academia, emphasizing the importance of inclusivity in tech.
AI Snips
Chapters
Transcript
Episode notes
Power of Deep Latent Variable Models
- Deep latent variable models combine classical latent variables with neural network parameterizations for flexibility and efficiency.
- Neural parameterization often improves model generalization and structured discovery despite identical probabilistic factorizations.
From Deep Learning to Latent Models
- Yoon Kim started his PhD focused on pure deep learning methods and supervised tasks.
- His interest in latent variable models and grammar induction grew through exploring VAEs and linguistics classes during his PhD.
Amortized Variational Inference Advice
- Use amortized variational inference to efficiently approximate posterior inference with neural networks.
- Train generative and inference networks end-to-end via the evidence lower bound to scale to large datasets.