AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Efficient Data Processing: Eager vs. Lazy Execution
This chapter explores the benefits of using lightweight data formats and the efficiency gained from avoiding unnecessary conversions. It contrasts eager and lazy execution in data processing, highlighting how lazy execution can optimize memory usage and enhance performance when working with large datasets.
How does a Python tool support all types of DataFrames and their various features? Could a lightweight library be used to add compatibility for newer formats like Polars or PyArrow? This week on the show, we speak with Marco Gorelli about his project, Narwhals.
Narwhals is a project aimed at library maintainers rather than end users. We discuss how the added compatibility benefits users by supporting modern features like lazy evaluation. We cover several projects Marco has been working with to implement Narwhals, including Altair, scikit-lego, and Ibis.
We also discuss how Marco started contributing to open-source projects. Marco has contributed to both pandas and Polars, which helps explain his interest in growing compatibility between libraries. He also offers advice on making your first contribution.
This episode is sponsored by CodeRabbit.
Course Spotlight: Differences Between Python’s Mutable and Immutable Types
In this video course, you’ll learn how Python’s mutable and immutable data types work internally and how you can take advantage of mutability or immutability to power your code.
Topics:
Show Links:
Level up your Python skills with our expert-led courses:
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
Listen to all your favourite podcasts with AI-powered features
Listen to the best highlights from the podcasts you love and dive into the full episode