The chapter explores the ethical dilemmas and existential risks posed by AI advancement, focusing on the alignment problem and current risks like fraud and scams. It discusses the intertwining concepts of intelligence and autonomy in AI systems and the importance of governing autonomy to prevent harm. Additionally, the conversation touches on open-sourcing AI technology, potential breakthroughs in AI towards super intelligence, and the role of AI in enhancing human collective intelligence.