Developer Voices

PyO3: From Python to Rust and Back Again (with David Hewitt)

Jul 3, 2024

David Hewitt, the principal committer to PyO3, shares insights into Python and Rust integration. Topics include bridging Rust and Python, exploring lifetimes in Rust, smart pointers, error handling, traits comparison, resource handling, and bringing Rust's power to the Python world.

01:34:29

Creator website

Episode guests

David Hewitt

AI Summary

Highlights

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

PyO3 bridges Python and Rust by navigating differences in memory management and error handling.

David Hewitt transitions from Python to Rust to optimize performance and coding experiences.

PyO3 reduces overheads to enhance performance efficiency in executing computationally intensive operations.

Deep dives

Interop between Python and Rust

Working on a project to facilitate interop between Python and Rust, the speaker discusses the fundamental challenge of connecting languages with different design philosophies. While exploring PyO3, a project enabling seamless interop, the discussion delves into the differences in memory management, threading, and error handling between Python and Rust. Emphasizing the complexities of making contrasting languages harmoniously coexist, the speaker highlights the detailed design work and technical considerations required for effective interop.

Making Rust and Python Functions Interoperable in Py03

04:15

Manipulating Interpreter for Rust Functions

01:39

Optimizing Performance in PyO3 Binding Layer

04:59

Intro

2min

Bridging Rust and Python with David Hewitt

7min

Bridging Python to Rust with PyO3

21min

Exploring the Concept of Lifetimes in Rust and Their Significance in Programming

5min

Exploring Smart Pointers and the Global Interpreter Lock in PyO3

2min

Rust's Error Handling and Traits: A Comparison with Python and Conversion to Python Exceptions

15min

Integrating Rust and Python for Proper Resource Handling

24min

Bringing Rust's Power to the Python World

16min

Exploring the Ergonomic Design of Rust and Python Integration

2min

There’s huge pressure on Python at the moment to get faster, ideally without changing at all. One increasingly–popular way of achieving that impossible task is to push the performance critical code down into C, C++, or Rust. And this week we’re focussing on the Python route, as we take a look at PyO3.

David Hewitt’s the principal committer to PyO3, and he joins us to go through the easy parts, the hard parts, and the works in progress, giving us an insight into how Python and Rust work under the hood, and quite how much work it takes to make them work as one.

–

PyO3 User Guide: https://pyo3.rs/v0.22.0/

PyO3 on Github: https://github.com/PyO3/pyo3

Polars: https://pola.rs/

Tokio: https://tokio.rs/

Trio: https://trio.readthedocs.io/

Robyn: https://github.com/sparckles/Robyn

Faster CPython: https://github.com/faster-cpython

Maturin: https://www.maturin.rs/

–

David on Mastodon: https://fosstodon.org/@davidhewitt

David on Twitter: https://x.com/davidhewittdev