Machine Learning Street Talk (MLST) #86 - Prof. YANN LECUN and Dr. RANDALL BALESTRIERO - SSL, Data Augmentation, Reward isn't enough [NEURIPS2022]
18 snips
Dec 11, 2022 
Yann LeCun, a pioneer in deep learning and Chief AI Scientist at Meta, joins researcher Randall Balestriero, an expert in learnable signal processing. They dive into self-supervised learning's advancements and the role of data augmentation in improving model efficiency. Exciting topics include innovative techniques for enhancing representations, the challenges of defining intelligence in learning, and the potential of new methodologies like NNClear. Their insights from NeurIPS capture the cutting edge of AI research and its applications, including Marsquake detection.
AI Snips
Chapters
Transcript
Episode notes
Self-Supervised Embeddings
- Hypothetical adjacency matrices reveal self-supervised embeddings' capacity.
- Spectral properties must overlap between surrogate and supervised tasks.
Self-Supervised Learning
- Multiple criteria in self-supervised learning will become the primary method for pre-training neural networks.
- Data augmentation generates labeled data without human intervention, key for self-supervised learning.
Reinforcement Learning Efficiency
- Minimize reinforcement learning use due to its inefficiency.
- Prioritize model predictive control or planning-based approaches, like in MuZero.