#97 Probably Overthinking Statistical Paradoxes, with Allen Downey
Learning Bayesian Statistics
Understanding Statistical Paradoxes and Sampling Biases
This chapter explores the concept of statistical paradoxes and the challenges of sampling biases. It discusses the importance of considering the whole distribution of treatment effects to avoid misleading conclusions, using the example of the Simpson paradox. The chapter also highlights the challenges of conducting experiments with small sample sizes in the social sciences.
Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!
In this episode, I had the pleasure of speaking with Allen Downey, a professor emeritus at Olin College and a curriculum designer at Brilliant.org. Allen is a renowned author in the fields of programming and data science, with books such as "Think Python" and "Think Bayes" to his credit. He also authors the blog "Probably Overthinking It" and has a new book by the same name, which he just released in December 2023.
In this conversation, we tried to help you differentiate between right and wrong ways of looking at statistical data, discussed the Overton paradox and the role of Bayesian thinking in it, and detailed a mysterious Bayesian killer app!
But that’s not all: we even addressed the claim that Bayesian and frequentist methods often yield the same results — and why it’s a false claim. If that doesn’t get you to listen, I don’t know what will!
Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work at https://bababrinkman.com/ !
Thank you to my Patrons for making this episode possible!
Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor,, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox, Andreas Kröpelin, Raphaël R, Nicolas Rode, Gabriel Stechschulte, Arkady, Kurt TeKolste, Gergely Juhasz, Marcus Nölke, Maggi Mackintosh, Grant Pezzolesi, Avram Aelony, Joshua Meehl, Javier Sabio, Kristian Higgins, Alex Jones, Gregorio Aguilar, Matt Rosinski, Bart Trudeau, Luis Fonseca, Dante Gates, Matt Niccolls, Maksim Kuznecov, Michael Thomas, Luke Gorrie and Cory Kiser.
Visit https://www.patreon.com/learnbayesstats to unlock exclusive Bayesian swag ;)
Links from the show:
- LBS #41, Thinking Bayes, with Allen Downey: https://learnbayesstats.com/episode/41-think-bayes-allen-downey/
- Allen’s blog: https://www.allendowney.com/blog/
- Allen on Twitter: https://twitter.com/allendowney
- Allen on GitHub: https://github.com/AllenDowney
- Order Allen’s book, Probably Overthinking It, at a 30% discount with the code UCPNEW: https://press.uchicago.edu/ucp/books/book/chicago/P/bo206532752.html
- The Bayesian Killer App: https://www.allendowney.com/blog/2023/03/20/the-bayesian-killer-app/
- Bayesian and Frequentist Results Are Not the Same, Ever: https://www.allendowney.com/blog/2021/04/25/bayesian-and-frequentist-results-are-not-the-same-ever/
- Allen’s presentation on the Overton paradox: https://docs.google.com/presentation/d/1-Uvby1Lfe1BTsxNv5R6PhXfwkLUgsyJgdkKtO8nUfJo/edit#slide=id.g291c5d4559e_0_0
- Video on the Overton Paradox, from PyData NYC 2022: https://youtu.be/VpuWECpTxmM
- Thompson sampling as a dice game: https://allendowney.github.io/TheShakes/
- Causal quartets – Different ways to attain the same average treatment effect: http://www.stat.columbia.edu/~gelman/research/unpublished/causal_quartets.pdf
- LBS #89, Unlocking the Science of Exercise, Nutrition & Weight Management, with Eric Trexler: https://learnbayesstats.com/episode/89-unlocking-science-exercise-nutrition-weight-management-eric-trexler/
- How Minds Change, David McRaney: https://www.davidmcraney.com/howmindschangehome
Abstract
We are happy to welcome Allen Downey back to ur show and he has great news for us: His new book “Probably Overthinking It” is available now.
You might know Allen from his blog by the same name or his previous work. Or maybe you watched some of his educational videos which he produces in his new position at brilliant.org.
We delve right into exciting topics like collider bias and how it can explain the “low brith weight paradox” and other situations that only seem paradoxical at first, until you apply causal thinking to it.
Another classic Allen can unmystify for us is Simpson’s paradox. The problem is not the data, but your expectations of the data. We talk about some cases of Simpson’s paradox, for example from statistics on the Covid-19 pandemic, also featured in his book.
We also cover the “Overton paradox” - which Allen named himself - on how people report their ideologies as liberal or conservative over time.
Next to casual thinking and statistical paradoxes, we return to the common claim that frequentist statistics and Bayesian statistics often give the same results. Allen explains that they are fundamentally different and that Bayesian should not shy away from pointing that out and to emphasise the strengths of their methods.
Transcript
This is an automatic transcript and may therefore contain errors. Please get in touch if you're willing to correct them.
The AI-powered Podcast Player
