AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Intro
Exploring the potential of AI systems in medicine, the chapter delves into the concept of an AI doctor offering medical expertise in underserved areas. The guest delves into their personal experience and the technological advancements enabling initiatives like the Ask the Doctor app.
Episode 126
I spoke with Vivek Natarajan about:
* Improving access to medical knowledge with AI
* How an LLM for medicine should behave
* Aspects of training Med-PaLM and AMIE
* How to facilitate appropriate amounts of trust in users of medical AI systems
Vivek Natarajan is a Research Scientist at Google Health AI advancing biomedical AI to help scale world class healthcare to everyone. Vivek is particularly interested in building large language models and multimodal foundation models for biomedical applications and leads the Google Brain moonshot behind Med-PaLM, Google's flagship medical large language model. Med-PaLM has been featured in The Scientific American, The Economist, STAT News, CNBC, Forbes, New Scientist among others.
I spend a lot of time on this podcast—if you like my work, you can support me on Patreon :)
Reach me at editor@thegradient.pub for feedback, ideas, guest suggestions.
Subscribe to The Gradient Podcast: Apple Podcasts | Spotify | Pocket Casts | RSSFollow The Gradient on Twitter
Outline:
* (00:00) Intro
* (00:35) The concept of an “AI doctor”
* (06:54) Accessibility to medical expertise
* (10:31) Enabling doctors to do better/different work
* (14:35) Med-PaLM
* (15:30) Instruction tuning, desirable traits in LLMs for medicine
* (23:41) Axes for evaluation of medical QA systems
* (30:03) Medical LLMs and scientific consensus
* (35:32) Demographic data and patient interventions
* (40:14) Data contamination in Med-PaLM
* (42:45) Grounded claims about capabilities
* (45:48) Building trust
* (50:54) Genetic Discovery enabled by a LLM
* (51:33) Novel hypotheses in genetic discovery
* (57:10) Levels of abstraction for hypotheses
* (1:01:10) Directions for continued progress
* (1:03:05) Conversational Diagnostic AI
* (1:03:30) Objective Structures Clinical Examination as an evaluative framework
* (1:09:08) Relative importance of different types of data
* (1:13:52) Self-play — conversational dispositions and handling patients
* (1:16:41) Chain of reasoning and information retention
* (1:20:00) Performance in different areas of medical expertise
* (1:22:35) Towards accurate differential diagnosis
* (1:31:40) Feedback mechanisms and expertise, disagreement among clinicians
* (1:35:26) Studying trust, user interfaces
* (1:38:08) Self-trust in using medical AI models
* (1:41:39) UI for medical AI systems
* (1:43:50) Model reasoning in complex scenarios
* (1:46:33) Prompting
* (1:48:41) Future outlooks
* (1:54:53) Outro
Links:
* Vivek’s Twitter and homepage
* Papers
* Towards Expert-Level Medical Question Answering with LLMs (2023)
* LLMs encode clinical knowledge (2023)
* Towards Generalist Biomedical AI (2024)
* AMIE
* Genetic Discovery enabled by a LLM (2023)
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode