Learning Bayesian Statistics BITESIZE | Understanding Simulation-Based Calibration, with Teemu Säilynoja
Jul 4, 2025
Teemu Säilynoja, an expert in simulation-based calibration and probabilistic programming, shares insights into the vital role of simulation-based calibration (SBC) in model validation. He discusses the challenges of developing SBC methods, focusing on the importance of prior and posterior analyses. The conversation dives into practical applications using tools like Stan and PyMC, and the significance of smart initialization in MCMC fitting. Teemu's expertise shines as he highlights strategies, including the Pathfinder approach, for navigating complex Bayesian models.
AI Snips
Chapters
Transcript
Episode notes
What Is Simulation-Based Calibration?
- Simulation-Based Calibration (SBC) checks if your model implementation and inference algorithm together are calibrated and working as expected.
- SBC involves ranking prior draws among posterior samples, expecting a uniform distribution of ranks for a well-calibrated model.
How to Implement SBC Practically
- Use packages like SPC in R and SimuC in Python to run SBC easily on popular probabilistic programming languages.
- Parallelize model refits to speed up SBC since it requires multiple model fits.
Why Posterior SBC Matters
- Posterior SBC focuses on regions of parameter space relevant to observed data, avoiding misleading results from unlikely prior areas.
- It replaces prior predictive samples with posterior predictive ones, allowing targeted calibration checks in practice.