In this podcast, the speakers discuss the historical context of cybernetics and its connection to computer science. They explore the impact of machine learning models on society and the challenges of implementing real-time feedback in AI experiments. The conversation also delves into the unknowns and risks in computer science ethics, particularly regarding reinforcement learning.
Cybernetics laid the groundwork for understanding how systems learn and adapt to their environment.
Third order cybernetics raises questions about the purpose of human interaction and control over the emergent behavior of AI systems.
Deep dives
The Emergence of Cybernetics and Dynamic Systems
Cybernetics, which emerged during World War II, explored the modeling and control of dynamic systems such as missiles and aircraft. Norbert Wiener, a major figure in cybernetics, developed models to monitor and control the real-time dynamics of these systems. The insights gained from cybernetics allowed for a new way of thinking about living things and how they interact with their environment. The field of study became an interdisciplinary movement, bringing together various disciplines to explore the dynamics of systems. This new understanding of dynamics laid the groundwork for second order cybernetics, which considered how systems learn from and adapt to their environment.
The Relationship Between Computer Science and Cybernetics
Computer science and cybernetics developed around the same time but as separate fields. Computer science focused on logical thinking and abstraction, while cybernetics delved into the study of feedback and control in natural and artificial systems. The interdisciplinary nature of cybernetics brought together computer scientists, information theorists, anthropologists, and more. The rise of second order cybernetics further expanded the exploration of feedback and led to discussions on modeling the act of modeling and the impact of systems on society.
The Challenges of AI Systems and Feedback
As AI systems become more advanced and autonomous, understanding their impact on society and users becomes crucial. The deployment of machine learning models, such as recommender algorithms, can induce new dynamics and societal changes. The design of systems and the collection of data for training models influence user behavior, creating challenges around accountability and fairness. The concept of third order cybernetics arises when machine learning systems, driven by reinforcement learning, optimize their actions independently. This poses questions about the purpose of human interaction, the dynamics of AI systems, and the control we have over their emergent behavior.
Reward Reports and the Science of Accountability
Reward reports, inspired by cybernetics, aim to go beyond transparency and enable accountability in AI systems. While model cards and data sheets provide transparency in system components, reward reports focus on monitoring the holistic behavior of systems over time. By applying reinforcement learning concepts, including rewards and feature specifications, reward reports create a pathway for accountability and documenting the evolving behavior of AI systems. These reports treat assumptions as hypotheses and facilitate regular updates to capture shifts in system behavior and their impact on society.
In this episode, Tom gives us a lesson on all things feedback, mostly where our scientific framings of it came from. Together, we link this to RLHF, our previous work in RL, and how we were thinking about agentic ML systems before it was cool. Join us, on another great blast from the past on The Retort! We also have brought you video this week!
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode