Former OpenAI Engineer William Saunders on Silence, Safety, and the Right to Warn
Jun 7, 2024
auto_awesome
Former OpenAI engineer William Saunders sheds light on the prioritization of profit over safety in tech companies. He discusses the 'right to warn' for employees raising AI risk concerns, emphasizing transparency and the need for regulatory protection. The episode explores the challenges of AI safety, confidential whistleblowing, and the impact of independent evaluation on tech product safety.
Researchers have a right to warn about AI risks to the public for transparency and accountability.
Interpretability in AI systems is crucial to predict behavior accurately and address potential risks.
Deep dives
Concerns About AI Incentives and Market Dominance
The podcast episode discusses the risks associated with the race to artificial general intelligence (AGI) and market dominance in the AI industry. It highlights how the focus on speed and market success can lead to shortcuts being taken, potentially compromising safety. The absence of current regulations in the US for AI systems raises concerns about early warning signs being missed by internal employees.
Importance of AI Systems Interpretability
The episode delves into the significance of interpretability in AI systems to ensure safety. It explains how understanding the internal workings of large language models is crucial for predicting their behavior accurately in different contexts. The conversation emphasizes the challenges posed by emergent capabilities in AI systems and the necessity for interpretability to address potential risks.
Ethical Dilemmas Surrounding AI Development
The narrative explores the ethical dilemmas faced in AI development, particularly regarding safety and transparency. It highlights instances where compromises in safety protocols were observed at a leading AI company, raising concerns about prioritizing market success over safety. The podcast underscores the need for a balanced approach to AI development that prioritizes thorough safety measures.
Advocating for a 'Right to Warn' in AI Companies
The episode introduces the 'right to warn' principles aimed at promoting transparency and accountability in AI companies. These principles address issues such as non-disparagement agreements, anonymous reporting processes for risk-related concerns, and fostering a culture that encourages open discussions about safety within organizations. The narrative underscores the importance of independent evaluation of safety protocols to ensure public trust and uphold ethical standards in AI development.
This week, a group of current and former employees from OpenAI and Google DeepMind penned an open letter accusing the industry’s leading companies of prioritizing profits over safety. This comes after a spate of high profile departures from OpenAI, including co-founder Ilya Sutskever and senior researcher Jan Leike, as well as reports that OpenAI has gone to great lengths to silence would-be whistleblowers.
The writers of the open letter argue that researchers have a “right to warn” the public about AI risks and laid out a series of principles that would protect that right. In this episode, we sit down with one of those writers: William Saunders, who left his job as a research engineer at OpenAI in February. William is now breaking the silence on what he saw at OpenAI that compelled him to leave the company and to put his name to this letter.