What are the new ways to describe your data in pandas 2.0? Will the addition of Apache Arrow to the data back end foster the growth of data interoperability? This week on the show, we talk with pandas core developer Marc Garcia about the release of pandas 2.0.

Marc shares his background and work on pandas. We discuss the history of data representation in pandas and the need to move beyond NumPy. We also talk about how Apache Arrow only solves some of the issues.

We dig into the potential of an Apache Arrow back end and how it could offer interoperability between data platforms. We also cover the moderate adoption and backward-compatibility concerns. Marc also shares his thoughts on making pandas more extensible.

Course Spotlight: The pandas DataFrame: Working With Data Efficiently

In this course, you’ll get started with pandas DataFrames, which are powerful and widely used two-dimensional data structures. You’ll learn how to perform basic operations with data, handle missing values, work with time-series data, and visualize data from a pandas DataFrame.

Topics:

00:00:00 – Introduction
00:02:07 – Getting involved with the pandas project
00:03:48 – Continued growth of the platform
00:06:49 – Parallel branch development
00:09:19 – The introduction of Apache Arrow
00:18:53 – Working with NumPy data in pandas
00:30:18 – Arrow data types and strings
00:41:23 – Video Course Spotlight
00:42:37 – Interoperability of Arrow data back end
00:50:36 – Could pandas be more extensible?
01:00:49 – Python DataFrame Summit 2023
01:08:12 – What are you excited about in the world of Python?
01:11:13 – What do you want to learn next?
01:12:12 – How can people follow your work online?
01:13:46 – Thanks and Goodbye

Show Links:

Level up your Python skills with our expert-led courses:

Support the podcast & join our community of Pythonistas

465. Tara Beth Leach and Sean Palmer: Deconstruction