The Thesis Review

Karen Ullrich, a Research Scientist at FAIR, studies the intersection of information theory and machine learning. She discusses her PhD work, highlighting the minimum description length principle and its impact on neural network compression. Their conversation delves into the intricate ties between data compression and cognitive processes, while exploring innovative methods for addressing imaging challenges. Ullrich also shares insights on enhancing differentiability in image reconstruction and offers practical advice for new researchers navigating complex data landscapes.

Danqi Chen is an assistant professor at Princeton University, co-leading the Princeton NLP Group. Her research focuses on fundamental methods for learning representations of language and knowledge, and practical systems including question answering, information extraction and conversational agents. Danqi’s PhD thesis is titled "Neural Reading Comprehension and Beyond", which she completed in 2018 at Stanford University. We discuss her work on parsing, reading comprehension and question answering. Throughout we discuss progress in NLP, fundamental challenges, and what the future holds. Episode notes: https://cs.nyu.edu/~welleck/episode27.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Kevin Ellis, an assistant professor at Cornell and a research scientist at Common Sense Machines, dives into the intriguing world of AI and program synthesis. He discusses his groundbreaking work on DreamCoder, which automates the creation of programming libraries using neural networks. Ellis explores the fusion of AI with natural language and cognitive models, emphasizing Bayesian approaches that mirror human cognition. He shares insights on bridging program synthesis with theorem proving, highlighting the importance of reusable abstractions in machine learning.

Tomas Mikolov is a Senior Researcher at the Czech Institute of Informatics, Robotics, and Cybernetics. His research has covered topics in natural language understanding and representation learning, including Word2Vec, and complexity. Tomas's PhD thesis is titles "Statistical Language Models Based on Neural Networks", which he completed in 2012 at the Brno University of Technology. We discuss compression and recurrent language models, the backstory behind Word2Vec, and his recent work on complexity & automata. Episode notes: https://cs.nyu.edu/~welleck/episode25.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Martin Arjovsky is a postdoctoral researcher at INRIA. His research focuses on generative modeling, generalization, and exploration in RL. Martin's PhD thesis is titled "Out of Distribution Generalization in Machine Learning", which he completed in 2019 at New York University. We discuss his work on the influential Wasserstein GAN early in his PhD, then discuss his thesis work on out-of-distribution generalization which focused on causal invariance and invariant risk minimization. Episode notes: https://cs.nyu.edu/~welleck/episode24.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Simon Du, an Assistant Professor at the University of Washington, delves into the theoretical foundations of deep learning and gradient descent. He discusses the intricacies of addressing non-convex problems, revealing challenges and insights from his research. The conversation highlights the significance of the neural tangent kernel and its implications for optimization and generalization. Simon also shares practical tips for reading research papers, drawing connections between theory and practice, and navigating a successful research career.

Graham Neubig is an Associate Professor at Carnegie Mellon University. His research focuses on language and its role in human communication, with the goal of breaking down barriers in human-human or human-machine communication through the development of NLP technologies. Graham’s PhD thesis is titled "Unsupervised Learning of Lexical Information for Language Processing Systems", which he completed in 2012 at Kyoto University. We discuss his PhD work related to the fundamental processing units that NLP systems use to process text, including non-parametric Bayesian models, segmentation, and alignment problems, and discuss how his perspective on machine translation has evolved over time. Episode notes: http://cs.nyu.edu/~welleck/episode22.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at http://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Michela Paganini, a Research Scientist at DeepMind, focuses on compressing and scaling neural networks. She shares insights from her PhD on machine learning in high energy physics, particularly around the ATLAS experiment at CERN. The conversation delves into jet tagging and the evolution from traditional methods to deep learning. Michela reflects on her transformative experiences at CERN during the Higgs boson discovery and the interplay between physics and machine learning, emphasizing mentorship's role in her innovative journey.

Josef Urban, Principal Researcher at the Czech Institute of Informatics, shares his insights into artificial intelligence and automated theorem proving. He discusses the balance between deductive and inductive reasoning in formal mathematics, alongside the significance of the Mizar system. Topics include the philosophy of mathematics as invention versus discovery and the challenges of formalization. He also highlights advances in premise selection using machine learning, and reflects on his PhD journey that shaped his dedication to meaningful scientific inquiry.

Dumitru Erhan, a Research Scientist at Google Brain, dives into the fascinating world of neural networks. He discusses his groundbreaking PhD work on deep architectures and unsupervised pretraining. The conversation touches on the evolution of deep learning, the significance of regularization hypotheses, and the philosophical nuances in AI task conceptualization. Dumitru shares insights into the transition from traditional computer vision to deep neural networks and highlights the importance of unexpected outcomes in enhancing research understanding.