What Is a Markov Chain?

In this episode, our guest is Sean Meyn, Professor and Robert C. Pittman Eminent Scholar Chair in the Department of Electrical and Computer Engineering at the University of Florida. The episode features Sean’s adventures in the areas of Markov chains, networks and Reinforcement Learning (RL) as well as anecdotes and trivia about beekeeping and jazz.

Outline
00:00 - Intro
00:22 - Sean’s early steps
03:53 - Markov chains
08:45 - Networks
18:26 - Stochastic approximation
25:00 - Reinforcement Learning
38:57 - The intersection of Reinforcement Learning and  Control
42:37 - Favourite theorem
44:05 - Beekeeping and jazz
48:47 - Outro

Episode links
Sean’s website: https://meyn.ece.ufl.edu/
Sean’s books: shorturl.at/CFGRY (and T. Sargent's review: shorturl.at/hlGNR)
G. Zames: shorturl.at/JPRWX (see also: shorturl.at/chiw5)
State space model: shorturl.at/hST07
The life and work of A.A. Markov: shorturl.at/qsv35
Fluid model: shorturl.at/HKN56
M/M/1 queue: shorturl.at/dQW36
Borkar-Meyn theorem: shorturl.at/eSTV4
NCCR Automation Symposia: shorturl.at/csv03 (see also shorturl.at/ekpZ3)
V. Konda’s PhD Thesis: shorturl.at/bdrv7

Support the show

Podcast info
Podcast website: https://www.incontrolpodcast.com/
Apple Podcasts: https://tinyurl.com/5n84j85j
Spotify: https://tinyurl.com/4rwztj3c
RSS: https://tinyurl.com/yc2fcv4y
Youtube: https://tinyurl.com/bdbvhsj6
Facebook: https://tinyurl.com/3z24yr43
Twitter: https://twitter.com/IncontrolP
Instagram: https://tinyurl.com/35cu4kr4

Acknowledgments and sponsors
This episode was supported by the National Centre of Competence in Research on «Dependable, ubiquitous automation» and the IFAC Activity fund. The podcast benefits from the help of an incredibly talented and passionate team. Special thanks to L. Seward, E. Cahard, F. Banis, F. Dörfler, J. Lygeros, ETH studio and mirrorlake . Music was composed by A New Element.