The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)

Transformers On Large-Scale Graphs with Bayan Bruss - #641

Aug 7, 2023

Bayan Bruss, Vice President of Applied ML Research at Capital One, dives into groundbreaking research on applying machine learning in finance. He discusses two key papers presented at ICML, focusing on interpretability in image representations and the innovative global graph transformer model. Listeners will learn about tackling computational challenges, the balance between model sparsity and performance, and the significance of embedding dimensions. With insights into advancing deep learning techniques, this conversation opens new avenues for efficiency in machine learning.

38:36

Creator website

Episode guests

Bayan Bruss

Highlights

AI Chapters

Episode notes

Understanding the Importance of Features and Embeddings in Model Predictions

02:44

Using Contrastive Explanations to Enhance Interpretability

01:22

How to Be a Successful AI and Machine Learning Researcher

01:35

Intro

2min

Advancements in Applied Machine Learning

21min

Transformers in Large-Scale Graphs

8min

Harnessing Sparsity in Graph Connectivity

3min

Comparative Analysis of Graph Models in Deep Learning

2min

Exploring Research Frontiers in Deep Learning and Graph Networks

4min

Today we’re joined by Bayan Bruss, Vice President of Applied ML Research at Capital One. In our conversation with Bayan, we covered a pair of papers his team presented at this year’s ICML conference. We begin with the paper Interpretable Subspaces in Image Representations, where Bayan gives us a dive deep into the interpretability framework, embedding dimensions, contrastive approaches, and how their model can accelerate image representation in deep learning. We also explore GOAT: A Global Transformer on Large-scale Graphs, a scalable global graph transformer. We talk through the computation challenges, homophilic and heterophilic principles, model sparsity, and how their research proposes methodologies to get around the computational barrier when scaling to large-scale graph models.

The complete show notes for this episode can be found at twimlai.com/go/641.

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.