The Importance of Interpretability in Large Language Models

6min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

With convolutional neural networks, interpretability had many good advances. We can kind of understand the different layers have different sort of ranges that they're looking at. With large language models, it's perhaps a little more complicated. But I think it's still achievable in the sense that we could kind of ask, well, what kind of prompts this degenerate if I sort of drop out this part of the network? And sort of start getting at a language to even describe these types of aspects of human behavior or psychology from the spoken part in the language bar.

Manolis Kellis is a computational biologist at MIT. Please support this podcast by checking out our sponsors:
– Eight Sleep: https://www.eightsleep.com/lex to get special savings
– NetSuite: http://netsuite.com/lex to get free product tour
– ExpressVPN: https://expressvpn.com/lexpod to get 3 months free
– InsideTracker: https://insidetracker.com/lex to get 20% off

EPISODE LINKS:
Manolis Website: http://web.mit.edu/manoli/
Manolis Twitter: https://twitter.com/manoliskellis
Manolis YouTube: https://youtube.com/@ManolisKellis1

PODCAST INFO:
Podcast website: https://lexfridman.com/podcast
Apple Podcasts: https://apple.co/2lwqZIr
Spotify: https://spoti.fi/2nEwCF8
RSS: https://lexfridman.com/feed/podcast/
YouTube Full Episodes: https://youtube.com/lexfridman
YouTube Clips: https://youtube.com/lexclips

SUPPORT & CONNECT:
– Check out the sponsors above, it’s the best way to support this podcast
– Support on Patreon: https://www.patreon.com/lexfridman
– Twitter: https://twitter.com/lexfridman
– Instagram: https://www.instagram.com/lexfridman
– LinkedIn: https://www.linkedin.com/in/lexfridman
– Facebook: https://www.facebook.com/lexfridman
– Medium: https://medium.com/@lexfridman

OUTLINE:
Here’s the timestamps for the episode. On some podcast players you should be able to click the timestamp to jump to that time.
(00:00) – Introduction
(06:44) – Humans vs AI
(15:50) – Evolution
(37:34) – Nature vs Nurture
(50:03) – AI alignment
(56:27) – Impact of AI on the job market
(1:08:06) – Human gatherings
(1:13:07) – Human-AI relationships
(1:23:11) – Being replaced by AI
(1:35:37) – Fear of death
(1:47:33) – Consciousness
(1:54:58) – AI rights and regulations
(2:00:41) – Halting AI development
(2:13:52) – Education
(2:19:16) – Biology research
(2:26:36) – Meaning of life
(2:29:09) – Loneliness