Limitations of Interpretability in AI and Understanding Causal Mechanisms in Biology

In episode 92 of The Gradient Podcast, Daniel Bashir speaks to Kevin K. Yang.

Kevin is a senior researcher at Microsoft Research (MSR) who works on problems at the intersection of machine learning and biology, with an emphasis on protein engineering. He completed his PhD at Caltech with Frances Arnold on applying machine learning to protein engineering. Before joining MSR, he was a machine learning scientist at Generate Biomedicines, where he used machine learning to optimize proteins.

Have suggestions for future podcast guests (or other feedback)? Let us know here or reach us at editor@thegradient.pub

Subscribe to The Gradient Podcast:  Apple Podcasts  | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter

Outline:

* (00:00) Intro

* (02:40) Kevin’s background

* (06:00) Protein engineering early in Kevin’s career

* (12:10) From research to real-world proteins: the process

* (17:40) Generative models + pretraining for proteins

* (22:47) Folding diffusion for protein structure generation

* (30:45) Protein evolutionary dynamics and generative models of protein sequences

* (40:03) Analogies and disanalogies between protein modeling and language models

* (41:45) In representation learning

* (45:50) Convolutions vs. transformers and inductive biases

* (49:25) Pretraining tasks for protein structure

* (51:45) More on representation learning for protein structure

* (54:06) Kevin’s thoughts on interpretability in deep learning for protein engineering

* (56:50) Multimodality in protein engineering and future directions

* (59:14) Outro

Links:

* Kevin’s Twitter and homepage

* Research

* Generative models + pre-training for proteins and chemistry

* Broad intro to techniques in the space

* Protein structure generation via folding diffusion

* Protein sequence design with deep generative models (review)

* Evolutionary velocity with protein language models predicts evolutionary dynamics of diverse proteins

* Protein generation with evolutionary diffusion: sequence is all you need

* ML for protein engineering

* ML-guided directed evolution for protein engineering (review)

* Learned protein embeddings for ML

* Adaptive machine learning for protein engineering (review)

* Multimodal deep learning for protein engineering