Marco Gorelli

In this enlightening discussion, Marco Gorelli, a Senior Software Engineer at Quansight Labs and a core developer of the Polars and Narwhals libraries, shares his insights on optimizing data operations. He explains when to use Polars over Pandas and its unique features like lazy evaluation and string optimizations. Marco also delves into the Narwhals library, bridging compatibility with Pandas. He shares his strategies for winning forecasting competitions and addresses the need for greater diversity in data science. Prepare for a deep dive into the future of data manipulation!

Marco Gorelli, a data scientist and creator of the Narwhals project, shares his journey into open source and the mission behind Narwhals, which enhances compatibility between DataFrame libraries like Polars and PyArrow. He discusses the benefits of lazy evaluation and how this simplifies data processing. The conversation also delves into community engagement in open source, offering advice for newcomers, and highlights collaborations with libraries such as Altair and scikit-lego. Marco's insights make for an engaging exploration of modern data compatibility.

Marco Gorelli, a contributor at Quansight Labs and creator of the Narwhals library, discusses the challenges of achieving compatibility across various data frame libraries like Pandas and Polars. He highlights how Narwhals enhances code consistency and simplifies transitions for developers. The conversation dives into performance comparisons between Polars and cuDF, the advantages of integrating Rust, and the significance of typing in open-source projects. Marco also emphasizes the importance of community engagement in evolving data science tools.