Parquet Query - Columnar Format

Most structures in Python or sort of C++ are what you would call, I guess, like an array of structures. Instead this columnar format Parquet arrow and it sounds like this rust library is trying to take advantage of that. The advantage is their index match. So index seven across each of the arrays represents one instance of your structure,. But you can scan the array by grabbing those blocks of data down and going through them depending on how large your structure is.

Transcript

Play full episode

The AI-powered Podcast Player

Save insights by tapping your headphones, chat with episodes, discover the best highlights - and more!

Get the app