How to Quantify Your Uncertainty in Probability Arguments

Tim Palmer graduated from Oxford with a PhD in mathematical physics, working on general relativity, and got a postdoc to work with Stephen Hawking. He turned it down and moved into the field of meteorology, and then moved on to Climate Change studies, where he pioneered the development of what is called ‘ensemble forecasting’ to predict both long term climate change, as well as short term weather predictions. This technique has now become a standard in the field, and is necessary to properly account for possible chaotic behavior in atmospheric systems.

Even simple classical systems can be chaotic—implying that even minute changes in initial conditions can sometimes produce dramatic variations in their later evolution. The canonical hyperbolic example is a butterfly flapping its wings in Kansas might later cause a violent storm on the Eastern Seaboard.

On first glance, it may seem that this would imply all predictivity must go out the window, but over the past 40 years techniques have been developed for dealing with the so-called ‘fractal’ distributions that often result from chaotic dynamics, and as a result, it has become possible to constrain the range of possible long term outcomes of chaotic behavior.

Tim Palmer has recently written a new book, entitled The Primacy of Doubt, which provides a wonderful discussion about the importance of accounting for doubt and uncertainty in a wide variety of systems, from weather to medicine, and even includes discussions of there possible implications of his ideas for the fundamentals of quantum mechanics and gravity. While I am more skeptical of his nevertheless intriguing latter arguments, Tim and I had a fascinating and informative discussion about his own experiences as a scientist, and the importance of explicitly incorporating a range of initial conditions when exploring weather and climate predictions.

For many people, uncertainty is something to be avoided, but in physics, uncertainty is an inherent part of our understanding of the world, and it must be faced head-on. Being able to make quantitative predictions with likelihoods that have meaning requires it, and science is the only area of human inquiry where we can state with great quantitative accuracy what the likelihood is that a given prediction will be correct. This is a triumph of the scientific process and deserves to be better understood. In this regard, there are fewer better guides than Tim Palmer, and it was a delight to spend time with him on this podcast, which will enlighten and entertain.

As always, an ad-free video version of this podcast is also available to paid Critical Mass subscribers. Your subscriptions support the non-profit Origins Project Foundation, which produces the podcast. The audio version is available free on the Critical Mass site and on all podcast sites, and the video version will also be available on the Origins Project Youtube channel as well.