Talk Python To Me #480: Ahoy, Narwhals are bridging the data science APIs
6 snips
Oct 9, 2024 
Marco Gorelli, a contributor at Quansight Labs and creator of the Narwhals library, discusses the challenges of achieving compatibility across various data frame libraries like Pandas and Polars. He highlights how Narwhals enhances code consistency and simplifies transitions for developers. The conversation dives into performance comparisons between Polars and cuDF, the advantages of integrating Rust, and the significance of typing in open-source projects. Marco also emphasizes the importance of community engagement in evolving data science tools.
AI Snips
Chapters
Transcript
Episode notes
Narwhals Overview
- Narwhals is a compatibility layer between data frame libraries, mainly for tool builders.
- It wraps different APIs, focusing on Polars expressions, and simplifies library development.
Narwhals for Pandas
- A maintainer considered using Narwhals even for just Pandas.
- This was to avoid dealing with Pandas API changes between versions, simplifying their life.
Narwhals and Dependencies
- Narwhals minimizes dependency issues in constrained environments.
- Users can bring their own data frame library, reducing installation and compatibility problems.