The Thesis Review

Karen Ullrich, a Research Scientist at FAIR, studies the intersection of information theory and machine learning. She discusses her PhD work, highlighting the minimum description length principle and its impact on neural network compression. Their conversation delves into the intricate ties between data compression and cognitive processes, while exploring innovative methods for addressing imaging challenges. Ullrich also shares insights on enhancing differentiability in image reconstruction and offers practical advice for new researchers navigating complex data landscapes.

Danqi Chen is an assistant professor at Princeton University, co-leading the Princeton NLP Group. Her research focuses on fundamental methods for learning representations of language and knowledge, and practical systems including question answering, information extraction and conversational agents. Danqi’s PhD thesis is titled "Neural Reading Comprehension and Beyond", which she completed in 2018 at Stanford University. We discuss her work on parsing, reading comprehension and question answering. Throughout we discuss progress in NLP, fundamental challenges, and what the future holds. Episode notes: https://cs.nyu.edu/~welleck/episode27.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Kevin Ellis is an assistant professor at Cornell and currently a research scientist at Common Sense Machines. His research focuses on artificial intelligence, program synthesis, and neurosymbolic models. Kevin's PhD thesis is titled "Algorithms for Learning to Induce Programs", which he completed in 2020 at MIT. We discuss Kevin’s work at the intersection of machine learning and program induction, including inferring graphics programs from images and drawings, DreamCoder, and more. Episode notes: https://cs.nyu.edu/~welleck/episode26.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Tomas Mikolov is a Senior Researcher at the Czech Institute of Informatics, Robotics, and Cybernetics. His research has covered topics in natural language understanding and representation learning, including Word2Vec, and complexity. Tomas's PhD thesis is titles "Statistical Language Models Based on Neural Networks", which he completed in 2012 at the Brno University of Technology. We discuss compression and recurrent language models, the backstory behind Word2Vec, and his recent work on complexity & automata. Episode notes: https://cs.nyu.edu/~welleck/episode25.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Martin Arjovsky is a postdoctoral researcher at INRIA. His research focuses on generative modeling, generalization, and exploration in RL. Martin's PhD thesis is titled "Out of Distribution Generalization in Machine Learning", which he completed in 2019 at New York University. We discuss his work on the influential Wasserstein GAN early in his PhD, then discuss his thesis work on out-of-distribution generalization which focused on causal invariance and invariant risk minimization. Episode notes: https://cs.nyu.edu/~welleck/episode24.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Simon Du, an Assistant Professor at the University of Washington, delves into the theoretical foundations of deep learning and gradient descent. He discusses the intricacies of addressing non-convex problems, revealing challenges and insights from his research. The conversation highlights the significance of the neural tangent kernel and its implications for optimization and generalization. Simon also shares practical tips for reading research papers, drawing connections between theory and practice, and navigating a successful research career.

Graham Neubig is an Associate Professor at Carnegie Mellon University. His research focuses on language and its role in human communication, with the goal of breaking down barriers in human-human or human-machine communication through the development of NLP technologies. Graham’s PhD thesis is titled "Unsupervised Learning of Lexical Information for Language Processing Systems", which he completed in 2012 at Kyoto University. We discuss his PhD work related to the fundamental processing units that NLP systems use to process text, including non-parametric Bayesian models, segmentation, and alignment problems, and discuss how his perspective on machine translation has evolved over time. Episode notes: http://cs.nyu.edu/~welleck/episode22.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at http://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Michela Paganini is a Research Scientist at DeepMind. Her research focuses on investigating ways to compress and scale up neural networks. Michela's PhD thesis is titled "Machine Learning Solutions for High Energy Physics", which she completed in 2019 at Yale University. We discuss her PhD work on deep learning for high energy physics, including jet tagging and fast simulation for the ATLAS experiment at the Large Hadron Collider, and the intersection of machine learning and physics. Episode notes: https://cs.nyu.edu/~welleck/episode21.html Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter, and find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Josef Urban, Principal Researcher at the Czech Institute of Informatics, shares his insights into artificial intelligence and automated theorem proving. He discusses the balance between deductive and inductive reasoning in formal mathematics, alongside the significance of the Mizar system. Topics include the philosophy of mathematics as invention versus discovery and the challenges of formalization. He also highlights advances in premise selection using machine learning, and reflects on his PhD journey that shaped his dedication to meaningful scientific inquiry.

Dumitru Erhan, a Research Scientist at Google Brain, dives into the fascinating world of neural networks. He discusses his groundbreaking PhD work on deep architectures and unsupervised pretraining. The conversation touches on the evolution of deep learning, the significance of regularization hypotheses, and the philosophical nuances in AI task conceptualization. Dumitru shares insights into the transition from traditional computer vision to deep neural networks and highlights the importance of unexpected outcomes in enhancing research understanding.