#02 A clinical introduction to Large language models (LLM), AI chatbots, Med-PaLM

Sep 12, 2023

Discover the potential and pitfalls of large language models in healthcare. The hosts share their eye-opening experiences with AI chatbots, weighing their capabilities against their limitations. Delve into the evolution of language models and explore the critical importance of high-quality training data. The conversation also touches on the philosophical implications of AI behavior and the ongoing challenges in medical diagnostics. Join the exploration of how AI could transform clinical interactions while addressing pressing concerns.

Ask episode

Chapters

Transcript

Episode notes

Intro

00:00 • 4min

Initial Impressions and Reflections on Chat GPT in Healthcare

04:01 • 3min

The Evolution of Language Models

06:39 • 13min

The Psychology Behind Health Information Bias

19:28 • 2min

Training Large Language Models

21:05 • 5min

Enhancing AI in Healthcare: Key Challenges and Solutions

26:33 • 21min

Evolution of Stroke Treatment and AI in Healthcare

47:15 • 3min

Evaluating AI in Medical Practice

50:10 • 19min

Understanding AI Hallucinations

01:09:21 • 9min

Exploring the Nature of Intelligence Beyond Language

01:18:08 • 2min

Limitations of AI in Healthcare Diagnostics

01:20:05 • 21min

In this episode, we introduce large language models in healthcare, their potentials and pitfalls. We put AI chatbots like ChatGPT to the test, discuss our thoughts on Google's Med-PaLM, and dabble in a bit of philosophy of artificial general intelligence.

Like what you're hearing? Support us by subscribing and reaching out to us. We want to encourage open discussion between clinicians and developers.

Dev&Doc is a podcast where doctors and developers deep dive into the potential of AI in healthcare.
👨🏻‍⚕️Doc - Dr. Joshua Au Yeung
🤖Dev - Zeljko Kraljevic
LinkedIn Newsletter
YouTube
Spotify
Apple
Substack
For enquiries - 📧 Devanddoc@gmail.com

Timestamps:
00:00 Start
00:16 Intro
02:04 ChatGPT, A giant leap for mankind?
04:02 Spending two weeks with ChatGPT as a doctor
07:36 History of Large Language Models (LLMs)
10:15 A top down approach to what is an LLM
17:31 Medical language is a language in itself
18:42 A lot of data, that is just wrong
21:05 Self-supervised training LLM
22:05 Instruction based fine tuning a LLM
23:52 Doc summarizing LLM training
25:48 The clinical shortcomings of instruction based tuning
27:33 Reinforcement learning from Human (clinician) feedback
32:22 Doc summarizing LLM, RLHF - A strict vs a progressive parent
34:10 There are still many problems with LLMs, aligning with clinical training data
36:26 Training a LLM on discharge summaries is a bad idea
39:18 Garbage in garbage out - data
40:13 Context windows
40:43 Data cleaning clinical notes
44:31 Bias in scientific domain LLMs PubmedGPT, Galatica
46:31 Data drift in medicine and continual learning
50:01 MedPaLM - instruction tuning to the medical domain
50:23 Model benchmarks do not reflect the real world
59:11 LLM emulating human language, but not the brain. Only one piece of the mind
1:01:10 LLMs on headaches and general knowledge
1:03:25 Where does a LLM fit in into the clinical work flow
1:05:50 Are regulations working against safety?
1:08:26 Cooling down LLMs to pass regulations
1:09:50 Why call it a hallucination? It's a false positive
1:13:57 Examples of bias of ChatGPT - A bad Santa Claus
1:17:00 Do LLMs encode true "understanding"? Can language lead to AGI?
1:20:05 Pregnancy - an acid test for Large Language Models
1:21:55 Training a LLM for the NHS (NHS-LLM)
1:25:25 Tell the model to "think deeply"
1:27:55 Asking ChatGPT to draw a picture of a human O.O
1:31:00 Language is not enough to achieve AGI
1:32:10 What can clinicians do about LLMs? Assisting vs Autonomous
1:39:07 What's next- Forecasting diagnoses with AI

🎞️ Editor - Dragan Kraljević
🎨 Brand design and art direction - Ana Grigorovici