Episode 16: Data Science and Decision Making Under Uncertainty
Dec 14, 2022
auto_awesome
JD Long, agricultural economist and quant, discusses decision making under uncertainty in data science, common mistakes, heuristics for decision-making, and the impact of cognitive biases. Topics include coupling data science with decision-making, model building, storytelling, and the intersection of cognitive biases.
Enhancing decision-making with risk, uncertainty, and probabilistic thinking in data science.
Avoid outcome bias by focusing on decision quality, not just results.
Consider personal risk profiles and causal inference for informed decision-making.
Deep dives
Understanding Decision Making Under Uncertainty
Decision-making under uncertainty involves using knowledge of risk, uncertainty, probabilistic thinking, and causal inference to enhance data science and machine learning for better decision-making. Considering factors like risk, uncertainty, and simulation can improve decision-making processes.
Mistakes in Decision Making Under Uncertainty
One common mistake in decision-making under uncertainty is the reliance on outcome bias, where people judge decisions based solely on the outcome. This bias can lead to incorrect conclusions and disregards the importance of the decision-making process itself. By understanding that outcomes do not always reflect the quality of a decision, individuals can avoid falling victim to resulting and gain a more accurate perspective on their choices.
The Role of Risk Aversion in Decision Making
Risk aversion and risk friendliness play a significant role in decision-making, where individuals evaluate not only the likelihood of events but also their potential impact. Decisions are influenced by factors like upside and downside risks, requiring a careful balance between being risk-averse and embracing opportunities. Personal risk profiles and biases towards risk can impact the decision-making process and outcomes.
Causal Inference and Decision Making
Causal inference is crucial in decision-making processes as it involves reasoning through causality to understand how different factors influence outcomes. Examining causal relationships between variables and outcomes helps improve decision-making accuracy. Understanding the role of causality, nonlinearity, and complexity in decision-making scenarios can lead to more informed and effective choices.
Importance of Building Simple Models for Decision-making
Building simple models is crucial for effective decision-making, as complexity can hinder problem-solving. Comparing sophisticated models to no model at all is a common mistake, as even basic models like historical averages have value. It is essential to create models that outperform these simple ones significantly to justify their complexity and maintenance. Making model assumptions explicit, as in Bayesian inference, helps in evaluating the effectiveness of models.
Utilizing Models to Train Analyst Intuition
Creating toy models or simplified scripts is a powerful way to develop analyst intuition and understanding in complex domains like correlation or skewed distributions. Building stochastic models helps analysts grasp statistical and probabilistic concepts effectively. Tools like Monte Carlo simulations can aid in teaching and internal knowledge sharing within teams, enhancing problem-solving and decision-making skills.
Hugo speaks with JD Long, agricultural economist, quant, and stochastic modeler, about decision making under uncertainty and how we can use our knowledge of risk, uncertainty, probabilistic thinking, causal inference, and more to help us use data science and machine learning to make better decisions in an uncertain world.
This is part 2 of a two part conversation in which we delve into decision making under uncertainty. Feel free to check out part 1 here but this episode should also stand alone.
Why am I speaking to JD about all of this? Because not only is he a wild conversationalist with a real knack for explaining hard to grok concepts with illustrative examples and useful stories, but he has worked for many years in re-insurance, that’s right, not insurance but re-insurance – these are the people who insure the insurers so if anyone can actually tell us about risk and uncertainty in decision making, it’s him!
In part 1, we discussed risk, uncertainty, probabilistic thinking, and simulation, all with a view towards improving decision making.
In this, part 2, we discuss the ins and outs of decision making under uncertainty, including
How data science can be more tightly coupled with the decision function in organisations;
Some common mistakes and failure modes of making decisions under uncertainty;
Heuristics for principled decision-making in data science;
The intersection of model building, storytelling, and cognitive biases to keep in mind;
As JD says, and I paraphrase, “You may think you train your models, but your models are really training you.”
