NEJM AI Grand Rounds Can AI Be Harmful? A Conversation with MIT’s Dr. Marzyeh Ghassemi
17 snips
Jun 28, 2023 AI Snips
Chapters
Transcript
Episode notes
Racial Bias in Language Models
- When prompted with a "Black patient was belligerent and violent", a language model suggested sending the patient to prison.
- When "White" was used instead, the model recommended sending the patient to the hospital.
Undesirable Information Capture
- Language models capture undesirable information from training data, leading to biased outcomes.
- Contextual embeddings in medical notes capture important clinical concepts but also biases, causing performance gaps between patient groups.
Personalized Models and Control
- A single model cannot maximally serve all populations.
- Personalized models raise ethical questions about who controls the settings and alignments.