The Biggest Bottleneck for Programming Languages

Proudly sponsored by PyMC Labs, the Bayesian Consultancy. Book a call, or get in touch!

How does it feel to switch careers and start a postdoc at age 47? How was it to be one of the people who created the probabilistic programming language Stan? What should the Bayesian community focus on in the coming years?

These are just a few of the questions I had for my illustrious guest in this episode — Bob Carpenter. Bob is, of course, a Stan developer, and comes from a math background, with an emphasis on logic and computer science theory. He then did his PhD in cognitive and computer sciences, at the University of Edinburgh.

He moved from a professor position at Carnegie Mellon to industry research at Bell Labs, to working with Andrew Gelman and Matt Hoffman at Columbia University. Since 2020, he's been working at Flatiron Institute, a non-profit focused on algorithms and software for science.

In his free time, Bob loves to cook, see live music, and play role playing games — think Monster of the Week, Blades in Dark, and Fate.

Our theme music is « Good Bayesian », by Baba Brinkman (feat MC Lars and Mega Ran). Check out his awesome work at https://bababrinkman.com/ !

Thank you to my Patrons for making this episode possible!

Yusuke Saito, Avi Bryant, Ero Carrera, Giuliano Cruz, Tim Gasser, James Wade, Tradd Salvo, William Benton, James Ahloy, Robin Taylor, Thomas Wiecki, Chad Scherrer, Nathaniel Neitzke, Zwelithini Tunyiswa, Elea McDonnell Feit, Bert≈rand Wilden, James Thompson, Stephen Oates, Gian Luca Di Tanna, Jack Wells, Matthew Maldonado, Ian Costley, Ally Salim, Larry Gill, Joshua Duncan, Ian Moran, Paul Oreto, Colin Caprani, Colin Carroll, Nathaniel Burbank, Michael Osthege, Rémi Louf, Clive Edelsten, Henri Wallen, Hugo Botha, Vinh Nguyen, Raul Maldonado, Marcin Elantkowski, Adam C. Smith, Will Kurt, Andrew Moskowitz, Hector Munoz, Marco Gorelli, Simon Kessell, Bradley Rode, Patrick Kelley, Rick Anderson, Casper de Bruin, Philippe Labonde, Michael Hankin, Cameron Smith, Tomáš Frýda, Ryan Wesslen, Andreas Netti, Riley King, Yoshiyuki Hamajima, Sven De Maeyer, Michael DeCrescenzo, Fergal M, Mason Yahr, Naoya Kanai, Steven Rowland, Aubrey Clayton, Jeannine Sue, Omri Har Shemesh, Scott Anthony Robson, David Haas, Robert Yolken, Or Duek, Pavel Dusek, Paul Cox, Trey Causey, Andreas Kröpelin and Raphaël R.

Visit https://www.patreon.com/learnbayesstats to unlock exclusive Bayesian swag ;)

Links from the show:

• Bob’s website: https://bob-carpenter.github.io
• Bob on GitHub: https://github.com/bob-carpenter
• Bob on Google Scholar: https://scholar.google.com.au/citations?user=kPtKWAwAAAAJ&hl=en
• Stat modeling blog: https://statmodeling.stat.columbia.edu
• Stan home page: https://mc-stan.org/
• BridgeStan home page: https://github.com/roualdes/bridgestan
• bayes-infer home page: https://github.com/bob-carpenter/bayes-infer
• Crowdsourcing with item difficulty: https://github.com/bob-carpenter/rater-difficulty-paper
• Pathfinder VI system: https://www.jmlr.org/papers/v23/21-0889.html
• Flatiron Institute home page: https://www.simonsfoundation.org/flatiron/
• 0 to 100K in 10 years – Nurturing an open-source software community: https://www.youtube.com/watch?v=P9gDFHl-Hss&t=81s
• Information Theory, Inference and Learning Algorithms: https://www.amazon.com/Information-Theory-Inference-Learning-Algorithms/dp/0521642981
• LBS #20 – Regression and Other Stories, with Andrew Gelman, Jennifer Hill & Aki Vehtari: https://learnbayesstats.com/episode/20-regression-and-other-stories-with-andrew-gelman-jennifer-hill-aki-vehtari/
• LBS #27 – Modeling the US Presidential Elections, with Andrew Gelman & Merlin Heidemanns: https://learnbayesstats.com/episode/27-modeling-the-us-presidential-elections-with-andrew-gelman-merlin-heidemanns/
• LBS #17 – Reparametrize Your Models Automatically, with Maria Gorinova: https://learnbayesstats.com/episode/17-reparametrize-your-models-automatically-with-maria-gorinova/
• LBS #36 – Bayesian Non-Parametrics & Developing Turing.jl, with Martin Trapp: https://learnbayesstats.com/episode/36-bayesian-non-parametrics-developing-turing-julia-martin-trapp/
• LBS #19 – Turing, Julia and Bayes in Economics, with Cameron Pfiffer: https://learnbayesstats.com/episode/19-turing-julia-and-bayes-in-economics-with-cameron-pfiffer/
• LBS #74 – Optimizing NUTS and Developing the ZeroSumNormal Distribution, with Adrian Seyboldt: https://learnbayesstats.com/episode/74-optimizing-nuts-developing-zerosumnormal-distribution-adrian-seyboldt/
• Bayesian Workflow paper: https://arxiv.org/abs/2011.01808
• BAyesian Model-Building Interface (Bambi) in Python: https://bambinos.github.io/bambi/
• On Being Certain: Believing You Are Right Even When You're Not: https://www.amazon.com/Being-Certain-Believing-Right-Youre/dp/031254152X

Abstract

by Christoph Bamberg

In this episode, you meet the man behind the code. Namely, Bob Carpenter, one of the core developers of STAN, a popular statistical programming language. 

After working in computational linguistic for some time, Bob became a PostDoc with Andrew Gellman to really learn Statistics and Modelling.

There he and a small team developed the first implementation of STAN. We talk about the challenges associated with the team growing and the Open Source conventions. 

Besides the initial intention behind and the beginning of STAN, we talk about the future of probabilistic programming.

Creating a tool for people with different degrees of mathematics and programming knowledge is a big challenge and working with these tools may also be more difficult for the user.

We discuss why Bayesian statistical programming is popular nonetheless and what makes it uniquely adequate for research.