Speech Detection Model and Word Classification Model Architectures

In this episode, I talk with David Moses and Jessie Liu about their recent NEJM paper ‘Neuroprosthesis for decoding speech in a paralyzed person with anarthria’, in which they decoded intended utterances from the brain of an individual with anarthria using an electrode array implanted of sensorimotor cortex and machine learning.

Moses DA, Metzger SL, Liu JR, et al. Neuroprosthesis for decoding speech in a paralyzed person with anarthria. N Eng J Med 2021; 385: 217-27. [doi]

New York Times article

Related papers:

Bouchard KE, Mesgarani N, Johnson K, Chang EF. Functional organization of human sensorimotor cortex for speech articulation. Nature 2013; 495: 327-32. [doi]

Chartier J, Anumanchipalli GK, Johnson K, Chang EF. Encoding of articulatory kinematic trajectories in human speech sensorimotor cortex. Neuron 2018; 98: 1042-54. [doi]

Anumanchipalli GK, Chartier J, Chang EF. Speech synthesis from neural decoding of spoken sentences. Nature 2019; 568: 493-8. [doi]

Moses DA, Leonard MK, Makin JG, Chang EF. Real-time decoding of question-and-answer speech dialogue using human cortical activity. Nat Commun 2019; 10: 3096. [doi]

Makin JG, Moses DA, Chang EF. Machine translation of cortical activity to text with an encoder–decoder framework. Nat Neurosci 2020; 23: 575-82. [doi]

Chang lab website