"The Cognitive Revolution" | AI Builders, Researchers, and Live Player Analysis Reading Minds from Shared Latent Space
Apr 17, 2024
Paul Scotty, the lead author of the groundbreaking MindEye2 paper, dives into the fascinating realm of AI-driven brain imaging. He discusses how the MindEye project reconstructs images from limited fMRI data, pushing the boundaries of neuroscience. The conversation touches on challenges in data processing, the intricacies of capturing brain activity, and innovative AI techniques like CLIP and Stable Diffusion XL. Scotty also explores future applications of brain-computer interfaces and the ethical considerations surrounding this emerging technology.
AI Snips
Chapters
Transcript
Episode notes
Collaboration Origins
- Paul Scotty joined the Lion Discord server to learn machine learning and discovered a project on image reconstruction from brain activity led by Tanishq Abraham.
- This led to a collaboration resulting in MindEye1, leveraging a new dataset and open-source models like CLIP and Stable Diffusion.
MindEye1 limitations
- MindEye1's limitation was training separate models for each person, requiring 30-40 hours of fMRI scans per individual.
- This made it impractical for new subjects or applications beyond perception.
Dataset Details
- Each participant underwent 40 fMRI sessions, viewing 750 images per hour-long session, with each image shown three times.
- They pressed a button if they saw a repeat image, though this data wasn't ultimately used in the models.