Machine Learning Street Talk (MLST) #85 Dr. Petar Veličković (Deepmind) - Categories, Graphs, Reasoning [NEURIPS22 UNPLUGGED]
8 snips
Dec 8, 2022 
Dr. Petar Veličković, a Staff Research Scientist at DeepMind known for his work on Graph Attention Networks, discusses fascinating advancements in deep learning. He explores how category theory enhances geometric deep learning and innovates graph neural networks. The conversation dives into algorithmic reasoning, exposing the shift from manual feature engineering to automated processes. Petar also addresses the challenges of neural networks with extrapolation versus interpolation and shares insights on expander graphs to overcome obstacles in information propagation.
AI Snips
Chapters
Books
Transcript
Episode notes
Category Theory's Power
- Category theory generalizes geometric deep learning concepts.
- It helps build models resistant to non-invertible operations, like data destruction in algorithms.
Edge Message Reuse
- Reusing edge messages for node outputs is common in GNNs, but categorically problematic.
- This creates representational pressure, impacting out-of-distribution performance.
Overcoming Category Theory Fear
- Don't be intimidated by category theory; find accessible resources.
- Start with resources that connect it to familiar deep learning concepts.
