In this episode of Machine Learning Street Talk, Tim Scarfe, Connor Shorten and Yannic Kilcher react to Yoshua Bengio’s ICLR 2020 Keynote “Deep Learning Priors Associated with Conscious Processing”. Bengio takes on many future directions for research in Deep Learning such as the role of attention in consciousness, sparse factor graphs and causality, and the study of systematic generalization. Bengio also presents big ideas in Intelligence that border on the line of philosophy and practical machine learning. This includes ideas such as consciousness in machines and System 1 and System 2 thinking, as described in Daniel Kahneman’s book “Thinking Fast and Slow”. Similar to Yann LeCun’s half of the 2020 ICLR keynote, this talk takes on many challenging ideas and hopefully this video helps you get a better understanding of some of them! Thanks for watching!
Please Subscribe for more videos!
Paper Links:
Link to Talk: https://iclr.cc/virtual_2020/speaker_7.html
The Consciousness Prior: https://arxiv.org/abs/1709.08568
Thinking Fast and Slow: https://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555
Systematic Generalization: https://arxiv.org/abs/1811.12889
CLOSURE: Assessing Systematic Generalization of CLEVR Models: https://arxiv.org/abs/1912.05783
Neural Module Networks: https://arxiv.org/abs/1511.02799
Experience Grounds Language: https://arxiv.org/pdf/2004.10151.pdf
Benchmarking Graph Neural Networks: https://arxiv.org/pdf/2003.00982.pdf
On the Measure of Intelligence: https://arxiv.org/abs/1911.01547
Please check out our individual channels as well!
Machine Learning Dojo with Tim Scarfe: https://www.youtube.com/channel/UCXvHuBMbgJw67i5vrMBBobA
Yannic Kilcher: https://www.youtube.com/channel/UCZHmQk67mSJgfCCTn7xBfe
Henry AI Labs: https://www.youtube.com/channel/UCHB9VepY6kYvZjj0Bgxnpbw
00:00:00 Tim and Yannics takes
00:01:37 Intro to Bengio
00:03:13 System 2, language and Chomsky
00:05:58 Cristof Koch on conciousness
00:07:25 Francois Chollet on intelligence and consciousness
00:09:29 Meditation and Sam Harris on consciousness
00:11:35 Connor Intro
00:13:20 Show Main Intro
00:17:55 Priors associated with Conscious Processing
00:26:25 System 1 / System 2
00:42:47 Implicit and Verbalized Knowledge [DONT MISS THIS!]
01:08:24 Inductive Priors for DL 2.0
01:27:20 Systematic Generalization
01:37:53 Contrast with the Symbolic AI Program
01:54:55 Attention
02:00:25 From Attention to Consciousness
02:05:31 Thoughts, Consciousness, Language
02:06:55 Sparse Factor graph
02:10:52 Sparse Change in Abstract Latent Space
02:15:10 Discovering Cause and Effect
02:20:00 Factorize the joint distribution
02:22:30 RIMS: Modular Computation
02:24:30 Conclusion
#machinelearning #deeplearning
