The Test Set by Posit Wes McKinney: Part 1 — Building Pandas, Arrow, and a speedrunning legacy
Jul 14, 2025
Wes McKinney, an influential open-source software developer and the mastermind behind Pandas and Apache Arrow, shares his journey from speedrunning communities to revolutionizing data processing. He discusses how frustration with data work sparked Pandas' creation and dives into the early days of organizing the GoldenEye speedrunning scene. Wes also reflects on the importance of community in open-source development, the challenges of optimizing CSV handling, and the thrill of collaborating with like-minded creators in the data space.
AI Snips
Chapters
Books
Transcript
Episode notes
Pandas Born From Frustration
- Wes McKinney started pandas as an internal tool at AQR because data cleaning was painfully inefficient.
- He left his PhD to work on pandas full-time and published Python for Data Analysis in 2012.
Speedrunning Taught Community Skills
- In the late 1990s Wes ran the GoldenEye speedrunning community and built processes for verifying runs.
- That early community stewardship taught him governance and community development skills he later reused in open source.
Optimization Mindset Transfers To Data
- Wes links a speedrunner's drive for efficiency to his obsession with optimizing data workflows.
- Improving bottlenecks like CSV parsing yields outsized productivity gains for analysts.
