Machine Learning Street Talk (MLST)

#85 Dr. Petar Veličković (Deepmind) - Categories, Graphs, Reasoning [NEURIPS22 UNPLUGGED]

Dec 8, 2022

Dr. Petar Veličković, a Staff Research Scientist at DeepMind known for his work on Graph Attention Networks, discusses fascinating advancements in deep learning. He explores how category theory enhances geometric deep learning and innovates graph neural networks. The conversation dives into algorithmic reasoning, exposing the shift from manual feature engineering to automated processes. Petar also addresses the challenges of neural networks with extrapolation versus interpolation and shares insights on expander graphs to overcome obstacles in information propagation.

36:55

Creator website

Episode guests

Petar Veličković

Highlights

AI Chapters

Episode notes

Category Theory for Deep Learning

01:30

The Curvature of Graphs with Negative One

01:20

Expander Graph Propagation for Efficient Information Sharing in Online Graphs

01:30

Intro

2min

Exploring Category Theory in Deep Learning

12min

Exploring Algorithmic Reasoning and Its Application in Neural Networks

4min

Neural Network Limitations and Extrapolation

9min

Innovations in Graph Neural Networks

7min

Exploring Graph Curvature and Bottlenecks in Message Passing

2min

Dr. Petar Veličković is a Staff Research Scientist at DeepMind, he has firmly established himself as one of the most significant up and coming researchers in the deep learning space. He invented Graph Attention Networks in 2017 and has been a leading light in the field ever since pioneering research in Graph Neural Networks, Geometric Deep Learning and also Neural Algorithmic reasoning. If you haven’t already, you should check out our video on the Geometric Deep learning blueprint, featuring Petar. I caught up with him last week at NeurIPS. In this show, from NeurIPS 2022 we discussed his recent work on category theory and graph neural networks.

https://petar-v.com/

https://twitter.com/PetarV_93/

TOC:

Categories (Cats for AI) [00:00:00]

Reasoning [00:14:44]

Extrapolation [00:19:09]

Ishan Misra Skit [00:27:50]

Graphs (Expander Graph Propagation) [00:29:18]

YT: https://youtu.be/1lkdWduuN14

MLST Discord: https://discord.gg/V25vQeFwhS

Support us! https://www.patreon.com/mlst

References on YT description, lots of them!

Host: Dr. Tim Scarfe

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.