Learning Bayesian Statistics
#56 Causal & Probabilistic Machine Learning, with Robert Osazuwa Ness
Podcast summary created with Snipd AI
Quick takeaways
- Causal inference reveals hidden variables behind correlations in data analysis.
- Bayesian decision theory quantifies uncertainty for informed decision-making processes.
- Causal generative machine learning estimates directional effects for complex modeling.
Deep dives
Relationship Between Fire Track Size and Damage in a Fire
The podcast discusses the curious relationship between the size of fire tracks used to extinguish fires and the amount of damage caused during a fire. Contrary to intuition, larger fire tracks do not lead to more damage. The underlying variable influencing this correlation is the seriousness of the fire, acting as a common cause for both the track size and damage.
Causal Inference and Machine Learning
The episode delves into the complexities of causal inference and machine learning featuring Herbert Osas Ness, a research scientist specializing in the intersection of causal and probabilistic machine learning. Causal inference poses challenges in ensuring accurate reporting of correlations and understanding causal relationships, intricately linking methodology and decision-making processes in data analysis.
Bayesian Approach to Modeling and Inference
The conversation transitions to Bayesian statistics, where the speaker emphasizes the importance of quantifying subjective knowledge and uncertainty using probability for inference. Bayesian decision theory emerges as a key strategy guiding decision-making processes and highlighting the significance of utilizing Bayesian methods for modeling and inference in various research and teaching endeavors.
Importance of Causal Generative Machine Learning
Causal generative machine learning is crucial when focusing on the causal effects of treatments on outcomes. It involves estimating not just correlations but also the directions of effects, making the modeling process more complex yet targeted. The generative approach to model building is highlighted as it simplifies the understanding of causal relationships compared to analyzing correlations.
Challenges in Advancing Causal Generative Modeling
Advancing causal generative modeling faces obstacles in terms of programming accessibility and resource allocation. Building structured models for counterfactual reasoning and expanding formal causal inference theory beyond directed acyclic graphs (DAGs) are crucial frontiers to address. The field seeks to enhance reproducibility, interpretability, and scalability to overcome barriers to wider adoption.
Did you know there is a relationship between the size of firetrucks and the amount of damage down to a flat during a fire? The bigger the truck sent to put out the fire, the bigger the damages tend to be. The solution is simple: just send smaller firetrucks!
Wait, that doesn’t sound right, does it? Our brain is a huge causal machine, so it can instinctively feel it’s not credible that size of truck and amount of damage done are causally related: there must be another variable explaining the correlation. Here, it’s of course the seriousness of the fire — even better, it’s the common cause of the two correlated variables.
Your brain does that automatically, but what about your computer? How do you make sure it doesn’t just happily (and mistakenly) report the correlation? That’s when causal inference and machine learning enter the stage, as Robert Osazuwa Ness will tell us.
Robert has a PhD in statistics from Purdue University. He currently works as a Research Scientist at Microsoft Research and a founder of altdeep.ai, which teaches live cohort-based courses on advanced topics in applied modeling.
As you’ll hear, his research focuses on the intersection of causal and probabilistic machine learning. Maybe that’s why I invited him on the show… Well, who knows, causal inference is very hard!
Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work at https://bababrinkman.com/ !
Thank you to my Patrons for making this episode possible!
Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, Adam Bartonicek, William Benton, Alan O'Donnell, Mark Ormsby, James Ahloy, Robin Taylor, Thomas Wiecki, Chad Scherrer, Nathaniel Neitzke, Zwelithini Tunyiswa, Elea McDonnell Feit, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Joshua Duncan, Ian Moran, Paul Oreto, Colin Caprani, George Ho, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Raul Maldonado, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Matthew McAnear, Michael Hankin, Cameron Smith, Luis Iberico, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Aaron Jones, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland and Aubrey Clayton.
Visit https://www.patreon.com/learnbayesstats to unlock exclusive Bayesian swag ;)
Links from the show:
- Robert's webpage: https://www.microsoft.com/en-us/research/people/robertness/
- Robert on Twitter: https://twitter.com/osazuwa
- Robert on GitHub: https://github.com/robertness
- Robert on LinkedIn: https://www.linkedin.com/in/osazuwa/
- Do-calculus enables causal reasoning with latent variable models, Arxiv: https://arxiv.org/abs/2102.06626
- Integrating Markov processes with structural causal modeling enables counterfactual inference in complex systems, NeurIPS Proceedings: https://proceedings.neurips.cc/paper/2019/hash/2d44e06a7038f2dd98f0f54c4be35e22-Abstract.html
- Causality 101 with Robert Ness, The TWIML AI Podcast: https://www.youtube.com/watch?v=UNEZztT5lpk
- Causal Modeling in Machine Learning, PyData Boston: https://www.youtube.com/watch?v=1BioSmE5m6s
- Pyro -- Deep Universal Probabilistic Programming: http://pyro.ai/
- Statistical Rethinking website: http://xcelab.net/rm/statistical-rethinking/
- The Book of Why -- The New Science of Cause and Effect : https://www.goodreads.com/book/show/36204378-the-book-of-why
- The Theory That Would Not Die -- How Bayes' Rule Cracked the Enigma Code : https://www.goodreads.com/book/show/10672848-the-theory-that-would-not-die
Remember Everything You Learn from Podcasts
