Episode 14: Decision Science, MLOps, and Machine Learning Everywhere
Nov 20, 2022
auto_awesome
Hugo Bowne-Anderson discusses decision science, MLOps, and the ubiquity of machine learning models. Topics include decision-making under uncertainty, biases in data collection, MLOps and DevOps convergence, digital feedback loops, Google's search evolution, and the impact of modern algorithms on reality perception.
Principled decision-making involves probabilistic thinking, quantitative risk assessment, skeptical data analysis, and wise decision prioritization.
Assessing decision quality independently of outcomes is essential, focusing on rational evaluation based on probabilities and impact.
Distinguishing between information and knowledge, utilizing scenario planning, and integrating data into decision-making processes improve decision outcomes.
Deep dives
Principled Decision-Making Under Uncertainty
Decision Making in a Time of Crisis essay discusses adopting a principled approach to decision-making under uncertainty. It emphasizes four tools: thinking probabilistically, considering risks quantitatively, analyzing reported data skeptically, and prioritizing decisions wisely. The essay contrasts binary decisions like doubling down in Blackjack to complex real-world choices, underscoring the importance of assessing decision quality regardless of outcomes and distinguishing between risk and uncertainty.
Challenges in Assessing Decision Quality
The essay explores the challenges in evaluating decision quality when outcomes are uncertain. It references Annie Duke's book on decision-making under uncertainty to illustrate how decisions should be assessed independently of outcomes. It highlights human tendencies to judge decisions based on results rather than decision quality, emphasizing the need to rationally evaluate decisions within the context of probabilities and impact.
Navigating Information and Decision Fatigue
The discussion delves into processing vast information amidst decision fatigue, emphasizing the importance of distinguishing between information and knowledge. It touches on the overwhelming nature of decision-making in the modern world, especially during a global crisis like the COVID-19 pandemic. The essay suggests methods like scenario planning to prioritize decisions based on potential impact and the integration of information into decision-making processes for improved outcomes.
Key Points on Data-Centric Software Architecture
Data-centric programming focuses on providing a supportive software architecture for data scientists to enhance productivity. The discussion highlights five crucial layers in data-centric software interfaces, emphasizing the importance of operational tools like Selden for model deployments and waits for model monitoring. It underlines the significance of feature engineering in preparing raw data for model usage, introducing emerging solutions such as Tecton for feature stores and Scale for labeling solutions.
Impacts of Modern Algorithms and Models
Modern algorithms and models like YouTube's recommendation system and Google Search significantly impact users' perceptions of reality. They operate based on varying principles, creating filter bubbles and echo chambers that tailor content and influence user behavior. These models exemplify different feedback loops and alignment of incentives, revealing the complexities of model deployments in shaping individual and societal experiences.
Hugo Bowne-Anderson, host of Vanishing Gradients, reads 3 audio essays about decision science, MLOps, and what happens when machine learning models are everywhere.
