Navigating Python's GIL Challenges

9min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

This chapter explores the evolution of parallel computing in Python, focusing on the Global Interpreter Lock (GIL) and its impact on multi-threading performance. It discusses the historical context of GIL, challenges faced by developers, and potential advancements like PyScript and Pyodide that could enhance concurrency. The conversation highlights the trade-offs between improving multi-threading capabilities and maintaining compatibility with existing libraries, reflecting on the cultural attitudes surrounding threading in Python.

Python performance has come a long way in recent times. And it's often the data scientists, with their computational algorithms and large quantities of data, who care the most about this form of performance. It's great to have Stan Seibert back on the show to talk about Python's performance for data scientists. We cover a wide range of tools and techniques that will be valuable for many Python developers and data scientists.

Episode sponsors

Posit
Talk Python Courses

Links from the show

Stan on Twitter: @seibert
Anaconda: anaconda.com
High Performance Python with Numba training: learning.anaconda.cloud
PEP 0703: peps.python.org
Python 3.13 gets a JIT: tonybaloney.github.io
Numba: numba.pydata.org
LanceDB: lancedb.com
Profiling tips: docs.python.org
Memray: github.com
Fil: a Python memory profiler for data scientists and scientists: pythonspeed.com
Rust: rust-lang.org
Granian Server: github.com
PIXIE at SciPy 2024: github.com
Free threading Progress: py-free-threading.github.io
Free Threading Compatibility: py-free-threading.github.io
caniuse.com: caniuse.com
SPy, presented at PyCon 2024: us.pycon.org
Watch this episode on YouTube: youtube.com
Episode transcripts: talkpython.fm

--- Stay in touch with us ---
Subscribe to us on YouTube: youtube.com
Follow Talk Python on Mastodon: talkpython
Follow Michael on Mastodon: mkennedy