#117 Unveiling the Power of Bayesian Experimental Design, with Desi Ivanova
Learning Bayesian Statistics
Exploring the Legacy of Information Theory and AI
This chapter examines Claude Shannon's groundbreaking contributions to information theory and their lasting influence on machine learning and language models. It reflects on his work done without computers and imagines a conversation with him about the future of artificial intelligence and the quest for artificial general intelligence.
Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!
Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work!
Visit our Patreon page to unlock exclusive Bayesian swag ;)
Takeaways:
- Designing experiments is about optimal data gathering.
- The optimal design maximizes the amount of information.
- The best experiment reduces uncertainty the most.
- Computational challenges limit the feasibility of BED in practice.
- Amortized Bayesian inference can speed up computations.
- A good underlying model is crucial for effective BED.
- Adaptive experiments are more complex than static ones.
- The future of BED is promising with advancements in AI.
Chapters:
00:00 Introduction to Bayesian Experimental Design
07:51 Understanding Bayesian Experimental Design
19:58 Computational Challenges in Bayesian Experimental Design
28:47 Innovations in Bayesian Experimental Design
40:43 Practical Applications of Bayesian Experimental Design
52:12 Future of Bayesian Experimental Design
01:01:17 Real-World Applications and Impact
Thank you to my Patrons for making this episode possible!
Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor,, Chad Scherrer, Zwelithini Tunyiswa, Bertrand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox, Andreas Kröpelin, Raphaël R, Nicolas Rode, Gabriel Stechschulte, Arkady, Kurt TeKolste, Gergely Juhasz, Marcus Nölke, Maggi Mackintosh, Grant Pezzolesi, Avram Aelony, Joshua Meehl, Javier Sabio, Kristian Higgins, Alex Jones, Gregorio Aguilar, Matt Rosinski, Bart Trudeau, Luis Fonseca, Dante Gates, Matt Niccolls, Maksim Kuznecov, Michael Thomas, Luke Gorrie, Cory Kiser, Julio, Edvin Saveljev, Frederick Ayala, Jeffrey Powell, Gal Kampel, Adan Romero, Will Geary, Blake Walters, Jonathan Morgan, Francesco Madrisotti, Ivy Huang and Gary Clarke.
Links from the show:
- Come see the show live at PyData NYC: https://pydata.org/nyc2024/
- Desi’s website: https://desirivanova.com/
- Desi on GitHub: https://github.com/desi-ivanova
- Desi on Google Scholar: https://scholar.google.com/citations?user=AmX6sMIAAAAJ&hl=en
- Desi on Linkedin: https://www.linkedin.com/in/dr-ivanova/
- Desi on Twitter: https://x.com/desirivanova
- LBS #34, Multilevel Regression, Post-stratification & Missing Data, with Lauren Kennedy: https://learnbayesstats.com/episode/34-multilevel-regression-post-stratification-missing-data-lauren-kennedy/
- LBS #35, The Past, Present & Future of BRMS, with Paul Bürkner: https://learnbayesstats.com/episode/35-past-present-future-brms-paul-burkner/
- LBS #45, Biostats & Clinical Trial Design, with Frank Harrell:https://learnbayesstats.com/episode/45-biostats-clinical-trial-design-frank-harrell/
- LBS #107, Amortized Bayesian Inference with Deep Neural Networks, with Marvin Schmitt: https://learnbayesstats.com/episode/107-amortized-bayesian-inference-deep-neural-networks-marvin-schmitt/
- Bayesian Experimental Design (BED) with BayesFlow and PyTorch: https://github.com/stefanradev93/BayesFlow/blob/dev/examples/michaelis_menten_BED_tutorial.ipynb
- Paper – Modern Bayesian Experimental Design: https://arxiv.org/abs/2302.14545
- Paper – Optimal experimental design; Formulations and computations: https://arxiv.org/pdf/2407.16212
- Information theory, inference and learning algorithms, by the great late Sir David MacKay: https://www.inference.org.uk/itprnn/book.pdf
- Patterns, Predictions and Actions, Moritz Hard and Ben Recht https://mlstory.org/index.html
Transcript
This is an automatic transcript and may therefore contain errors. Please get in touch if you're willing to correct them.
The AI-powered Podcast Player
