The Thesis Review

Andrew Lampinen is a research scientist at DeepMind. His research focuses on cognitive flexibility and generalization. Andrew’s PhD thesis is titled "A Computational Framework for Learning and Transforming Task Representations", which he completed in 2020 at Stanford University. We talk about cognitive flexibility in brains and machines, centered around his work in the thesis on meta-mapping. We cover a lot of interesting ground, including complementary learning systems and memory, compositionality and systematicity, and the role of symbols in machine learning. - Episode notes: https://cs.nyu.edu/~welleck/episode38.html - Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter - Find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html - Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Joonkoo Park is an Associate Professor and Honors Faculty in the Department of Psychological and Brain Sciences at UMass Amherst. He leads the Cognitive and Developmental Neuroscience Lab, focusing on understanding the developmental mechanisms and neurocognitive underpinnings of our knowledge about number and mathematics. Joonkoo’s PhD thesis is titled "Experiential Effects on the Neural Substrates of Visual Word and Number Processing", which he completed in 2011 at the University of Michigan. We talk about numerical processing in the brain, starting with nature vs. nurture, including the learned versus built-in aspects of neural architectures. We talk about the difference between word and number processing, types of numerical thinking, and symbolic vs. non-symbolic numerical processing. - Episode notes: https://cs.nyu.edu/~welleck/episode37.html - Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter - Find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html - Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Dieuwke Hupkes is a Research Scientist at Facebook AI Research and the scientific manager of the Amsterdam unit of ELLIS. Dieuwke's PhD thesis is titled, "Hierarchy and Interpretability in Neural Models of Language Processing", which she completed in 2020 at the University of Amsterdam. We discuss her work on which aspects of hierarchical compositionality and syntactic structure can be learned by recurrent neural networks, how these models can serve as explanatory models of human language processing, what compositionality actually means, and a lot more. - Episode notes: https://cs.nyu.edu/~welleck/episode36.html - Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter - Find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html - Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Armando Solar-Lezama is a Professor at MIT, and the Associate Director & COO of CSAIL. He leads the Computer Assisted Programming Group, focused on program synthesis. Armando’s PhD thesis is titled, "Program Synthesis by Sketching", which he completed in 2008 at UC Berkeley. We talk about program synthesis & his work on Sketch, how machine learning's role in program synthesis has evolved over time, and more. - Episode notes: https://cs.nyu.edu/~welleck/episode35.html - Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter - Find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html - Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Sasha Rush, an Associate Professor at Cornell Tech and a researcher at Hugging Face, delves into the intricacies of Natural Language Processing. He shares insights from his PhD thesis on Lagrangian Relaxation and its relevance today. The conversation touches on the balance between discrete and continuous algorithms, the evolution of coding practices, and the importance of community in open-source innovations. Additionally, they explore navigating depth and breadth in academia and the necessity of risk-taking in research for true innovation.

Michael R. Douglas, a theoretical physicist and professor at Stony Brook University, shares his insights on string theory and its mathematical connections. He recalls collaborating with legends like Feynman during his PhD. The conversation delves into machine learning's transformative role in science, the challenges of formalizing theories in physics, and the evolving landscape of programming languages in education. Douglas also highlights advancements in proof assistants and their impact on research reliability, painting an exciting picture of the future of theoretical physics.

Andre Martins is an Associate Professor at IST and VP of AI Research at Unbabel in Lisbon, Portugal. His research focuses on natural language processing and machine learning. Andre’s PhD thesis is titled, "The Geometry of Constrained Structured Prediction: Applications to Inference and Learning of Natural Language Syntax", which he completed in 2012 at Carnegie Mellon University and IST. We talk about his work in the thesis on structured prediction in NLP, and discuss connections between his thesis work on later work on sparsity, sparse communication, and more. - Episode notes: https://cs.nyu.edu/~welleck/episode32.html - Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter - Find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html - Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Jay McClelland is a Professor in the Psychology Department and Director of the Center for Mind, Brain, Computation and Technology at Stanford. His research addresses a broad range of topics in cognitive science and cognitive neuroscience, including Parallel Distributed Processing (PDP). Jay's PhD thesis is titled "Preliminary Letter Identification in the Perception of Words and Nonwords", which he completed in 1975 at University of Pennsylvania. We discuss his work in the thesis on the word superiority effect, how it led to the Integrated Activation model, the path to Parallel Distributed Processing and the connectionist revolution, and distributed vs rule-based and symbolic approaches to modeling human cognition and artificial intelligence. - Episode notes: https://cs.nyu.edu/~welleck/episode31.html - Follow the Thesis Review (@thesisreview) and Sean Welleck (@wellecks) on Twitter - Find out more info about the show at https://cs.nyu.edu/~welleck/podcast.html - Support The Thesis Review at www.patreon.com/thesisreview or www.buymeacoffee.com/thesisreview

Dustin Tran, a research scientist at Google Brain, specializes in probabilistic programming and deep learning. He discusses his PhD thesis on integrating probabilistic modeling with deep learning, highlighting the innovative Edward library and new inference algorithms. The conversation dives into the evolution of AI tools like TensorFlow, emphasizing their democratizing impact. Dustin also shares insights on transitioning from PhD to research roles, the importance of addressing uncertainty in neural networks, and the balance between academic benchmarks and practical advancements.

Tengyu Ma, an Assistant Professor at Stanford University, dives deep into the intricate world of non-convex optimization in machine learning. He discusses the fascinating concept that all local minima are global minima, shedding light on overparameterization benefits. Tengyu also reflects on his transformative journey from theory to practical applications and critiques educational programs that balance theory with hands-on research. The historical evolution of optimization techniques and their implications for neural network design are explored, making this a must-listen for aspiring researchers.