Professor Randall Balestriero, an expert in machine learning, dives deep into neural network geometry and spline theory. He introduces the captivating concept of 'grokking', explaining how prolonged training can enhance adversarial robustness. The discussion also highlights the significance of representing data through splines to improve model design and performance. Additionally, Balestriero explores the geometric implications for large language models in toxicity detection, and delves into the challenges of reconstruction learning and the intricacies of representation in neural networks.

We are now sponsored by Weights and Biases! Please visit our sponsor link: http://wandb.me/MLST Patreon: https://www.patreon.com/mlst Yann LeCun thinks that it's specious to say neural network models are interpolating because in high dimensions, everything is extrapolation. Recently Dr. Randall Balestriero, Dr. Jerome Pesente and prof. Yann LeCun released their paper learning in high dimensions always amounts to extrapolation. This discussion has completely changed how we think about neural networks and their behaviour. [00:00:00] Pre-intro [00:11:58] Intro Part 1: On linearisation in NNs [00:28:17] Intro Part 2: On interpolation in NNs [00:47:45] Intro Part 3: On the curse [00:48:19] LeCun [01:40:51] Randall B YouTube version: https://youtu.be/86ib0sfdFtw

Yann LeCun is a French computer scientist known for his pioneering work on convolutional neural networks, optical character recognition and computer vision. He is a Silver Professor at New York University and Vice President, Chief AI Scientist at Meta. Along with Yoshua Bengio and Geoffrey Hinton, he was awarded the 2018 Turing Award for their work on deep learning, earning them the nickname of the "Godfathers of Deep Learning". Dr. Randall Balestriero has been researching learnable signal processing since 2013, with a focus on learnable parametrized wavelets and deep wavelet transforms. His research has been used by NASA, leading to applications such as Marsquake detection. During his PhD at Rice University, Randall explored deep networks from a theoretical perspective and improved state-of-the-art methods such as batch-normalization and generative networks. Later, when joining Meta AI Research (FAIR) as a postdoc with Prof. Yann LeCun, Randall further broadened his research interests to include self-supervised learning and the biases emerging from data-augmentation and regularization, resulting in numerous publications. Episode recorded live at NeurIPS.  YT: https://youtu.be/9dLd6n9yT8U (references are there) Support us! https://www.patreon.com/mlst  Host: Dr. Tim Scarfe TOC: [00:00:00] LeCun interview [00:18:25] Randall Balestriero interview (mostly on spectral SSL paper, first ref)