How can AIs know what we want if *we* don't even know? (with Geoffrey Irving)
Jan 24, 2024
auto_awesome
Geoffrey Irving, an AI safety researcher at DeepMind with a rich background at OpenAI and Google Brain, delves into the intricate challenge of aligning AI systems with human values. He discusses how AIs can misinterpret user intentions and the philosophical differences between being an assistant and an autonomous agent. Irving also examines the biases in AI training, particularly from WEIRD cultures, and the potential for AI to manipulate human emotions. He emphasizes the need for diverse cultural representation and ethical guidelines to ensure responsible AI development.
Defining AI alignment involves choosing tasks that prioritize assistant-type behavior rather than full autonomy.
Addressing biases and balancing accident risk versus malicious risk are important considerations in AI alignment.
Psychological insights can guide the design of AI systems to better align with human values and interpret human feedback accurately.
Deep dives
The Importance of Aligning AI Systems
Aligning AI systems, especially as they become more powerful, is crucial. The goal is to make sure the AI systems do what humans want. This involves defining what it means for the AI system to align with human values and choosing tasks that prioritize assistant-type behavior rather than fully autonomous actions.
The Challenges of AI Alignment
AI alignment faces several challenges, including defining what tasks the AI systems should prioritize and how to ensure they behave as desired. Trusting the information provided by the AI systems is a significant hurdle, as they can hallucinate or provide misleading information. Additionally, addressing biases in AI training and balancing accident risk versus malicious risk are important considerations.
Supervising AI Systems with Human Feedback
Supervising AI systems with human feedback involves creating interactive protocols between humans and machines. The aim is to simulate the process of humans reflecting and reasoning to make more informed decisions. This can be done by having humans engage in dialogue with the AI systems, providing explanations, pointing out potential flaws, and collaborating to make more accurate judgments.
The Role of Psychology in AI Alignment
Psychological insights are valuable in AI alignment for understanding human behavior, decision-making, biases, and preferences. Human psychology can guide the design of AI systems, improving their ability to align with human values and accurately interpret and respond to human feedback.
Importance of Rules and Preference Aggregation
Establishing clear rules and mechanisms for preference aggregation is crucial in aligning AI systems with human values. By defining rules and principles, such as ensuring a neutral point of view or adhering to specific ethical guidelines, AI can be guided to produce desired outcomes. The process often involves working with experts from various fields and engaging in participatory approaches to create rules that align with diverse human preferences. Iterative experimental designs and human feedback training can help refine the rule-based frameworks. While conflicts and challenges may arise as rules multiply and potentially contradict, human judgment and decision-making remain important for resolving such issues.
International Adoption and Ethical Considerations
International adoption presents an opportunity to provide a loving and supportive home for children who may not have such opportunities in their home countries. It can offer a chance for children to thrive with parents who have the means to care for them. However, ethical challenges exist, particularly in ensuring that the adoption process is not exploitative or coercive. It is essential to scrutinize the adoption centers and ensure that children being placed for adoption are genuine orphans or do not belong to parents who were coerced into giving them up. Responsible adoption practices should consider the well-being of the child and aim to create a transparent and ethical process.
What does it really mean to align an AI system with human values? What would a powerful AI need to do in order to do "what we want"? How does being an assistant differ from being an agent? Could inter-AI debate work as an alignment strategy, or would it just result in arguments designed to manipulate humans via their cognitive and emotional biases? How can we make sure that all human values are learned by AIs, not just the values of humans in WEIRD societies? Are our current state-of-the-art LLMs politically left-leaning? How can alignment strategies take into account the fact that our individual and collective values occasionally change over time?
Geoffrey Irving is an AI safety researcher at DeepMind. Before that, he led the Reflection Team at OpenAI, was involved in neural network theorem proving at Google Brain, cofounded Eddy Systems to autocorrect code as you type, and worked on computational physics and geometry at Otherlab, D. E. Shaw Research, Pixar, and Weta Digital. He has screen credits on Ratatouille, WALL•E, Up, and Tintin. Learn more about him at his website, naml.us.
