In this engaging conversation, Connor Leahy, CEO of Conjecture and AI safety advocate, shares his insights on the potential risks of artificial general intelligence (AGI). He stresses the crucial need for AI alignment and the importance of empathy in understanding these systems. The discussion dives into the complexities of AI training, the dangers of dehumanizing biases, and the challenges of balancing research with product development. Connor also reflects on personal growth, storytelling, and the pressing need to alleviate human suffering as we navigate the rapidly evolving AI landscape.
02:40:13
forum Ask episode
web_stories AI Snips
view_agenda Chapters
menu_book Books
auto_awesome Transcript
info_circle Episode notes
question_answer ANECDOTE
Initial Skepticism and Shift in Perspective
Tim Scarfe was initially skeptical of GPT-3, seeing it as mere pattern matching.
Connor Leahy's examples and access to the API shifted Scarfe's perspective towards acknowledging GPT-3's reasoning abilities.
insights INSIGHT
Interface and Reliability over Capability
Connor Leahy emphasizes the importance of the interface in interacting with language models.
He argues that ChatGPT's success stems from its improved interface and reliability rather than fundamentally new capabilities.
insights INSIGHT
Language Models as Cognitive Extensions
Tim Scarfe acknowledges the revolutionary impact of language models as an extension of human cognition.
While acknowledging limitations, he emphasizes their utility across diverse applications, highlighting the potential of filling gaps and evolving usage.
Get the Snipd Podcast app to discover more snips from this episode
In 'The Society of Mind', Marvin Minsky presents a revolutionary theory of how the human mind works. He describes the mind as a collection of simple, mindless agents that interact to produce complex cognitive functions. The book is composed of 270 self-contained essays divided into 30 chapters, each contributing to a unified theory of the mind. Minsky explores concepts such as language, memory, learning, consciousness, and the sense of self, offering a philosophical and computational perspective on intelligence. The theory posits that intelligence stems from the diversity and cooperation of these agents, rather than any single principle or formal system.
Support us! https://www.patreon.com/mlst
MLST Discord: https://discord.gg/aNPkGUQtc5
In this podcast with the legendary Connor Leahy (CEO Conjecture) recorded in Dec 2022, we discuss various topics related to artificial intelligence (AI), including AI alignment, the success of ChatGPT, the potential threats of artificial general intelligence (AGI), and the challenges of balancing research and product development at his company, Conjecture. He emphasizes the importance of empathy, dehumanizing our thinking to avoid anthropomorphic biases, and the value of real-world experiences in learning and personal growth. The conversation also covers the Orthogonality Thesis, AI preferences, the mystery of mode collapse, and the paradox of AI alignment.
Connor Leahy expresses concern about the rapid development of AI and the potential dangers it poses, especially as AI systems become more powerful and integrated into society. He argues that we need a better understanding of AI systems to ensure their safe and beneficial development. The discussion also touches on the concept of "futuristic whack-a-mole," where futurists predict potential AGI threats, and others try to come up with solutions for those specific scenarios. However, the problem lies in the fact that there could be many more scenarios that neither party can think of, especially when dealing with a system that's smarter than humans.
https://www.linkedin.com/in/connor-j-leahy/https://twitter.com/NPCollapse
Interviewer: Dr. Tim Scarfe (Innovation CTO @ XRAI Glass https://xrai.glass/)
TOC:
The success of ChatGPT and its impact on the AI field [00:00:00]
Subjective experience [00:15:12]
AI Architectural discussion including RLHF [00:18:04]
The paradox of AI alignment and the future of AI in society [00:31:44]
The impact of AI on society and politics [00:36:11]
Future shock levels and the challenges of predicting the future [00:45:58]
Long termism and existential risk [00:48:23]
Consequentialism vs. deontology in rationalism [00:53:39]
The Rationalist Community and its Challenges [01:07:37]
AI Alignment and Conjecture [01:14:15]
Orthogonality Thesis and AI Preferences [01:17:01]
Challenges in AI Alignment [01:20:28]
Mechanistic Interpretability in Neural Networks [01:24:54]
Building Cleaner Neural Networks [01:31:36]
Cognitive horizons / The problem with rapid AI development [01:34:52]
Founding Conjecture and raising funds [01:39:36]
Inefficiencies in the market and seizing opportunities [01:45:38]
Charisma, authenticity, and leadership in startups [01:52:13]
Autistic culture and empathy [01:55:26]
Learning from real-world experiences [02:01:57]
Technical empathy and transhumanism [02:07:18]
Moral status and the limits of empathy [02:15:33]
Anthropomorphic Thinking and Consequentialism [02:17:42]
Conjecture: Balancing Research and Product Development [02:20:37]
Epistemology Team at Conjecture [02:31:07]
Interpretability and Deception in AGI [02:36:23]
Futuristic whack-a-mole and predicting AGI threats [02:38:27]
Refs:
1. OpenAI's ChatGPT: https://chat.openai.com/
2. The Mystery of Mode Collapse (Article): https://www.lesswrong.com/posts/t9svvNPNmFf5Qa3TA/mysteries-of-mode-collapse
3. The Rationalist Guide to the Galaxy https://www.amazon.co.uk/Does-Not-Hate-You-Superintelligence/dp/1474608795
5. Alfred Korzybski: https://en.wikipedia.org/wiki/Alfred_Korzybski
6. Instrumental Convergence: https://en.wikipedia.org/wiki/Instrumental_convergence
7. Orthogonality Thesis: https://en.wikipedia.org/wiki/Orthogonality_thesis
8. Brian Tomasik's Essays on Reducing Suffering: https://reducing-suffering.org/
9. Epistemological Framing for AI Alignment Research: https://www.lesswrong.com/posts/Y4YHTBziAscS5WPN7/epistemological-framing-for-ai-alignment-research
10. How to Defeat Mind readers: https://www.alignmentforum.org/posts/EhAbh2pQoAXkm9yor/circumventing-interpretability-how-to-defeat-mind-readers
11. Society of mind: https://www.amazon.co.uk/Society-Mind-Marvin-Minsky/dp/0671607405
Machine Learning Street Talk (MLST) 
