Optimizing Data Processing: The Challenges of CSV Handling in Pandas

Wes McKinney’s fingerprints are all over the modern data stack — from inventing Pandas to co-creating Arrow. But before all that, Wes was organizing speedrun communities and hacking together better ways to wrangle datasets in finance. In this conversation, he shares his origin story and what makes good tools good. Stay tuned for part 2, coming soon.

What’s Inside:

• How frustration with data work led Wes to build pandas (and leave a PhD)
• A nostalgic dive into the GoldenEye speedrunning scene
• Why read_csv performance is a deeply personal crusade
• Lessons from convincing friends to quit finance and go open source
• Founding startups, launching Arrow, and the Ibis origin story
• The beauty of letting contributors take the reins
• Shout-out to Philip Cloud, pandas’ resident pun master
• Why open communities win — and what it takes to build them