Speaking with the Creator of Pandas on the Future of Open Source (Wes McKinney) - KNN Ep. 184
Mar 22, 2024
auto_awesome
The podcast discusses the origin and evolution of the pandas library, Wes McKinney's transition to a new role at Posit, and his insights on the future of open source. Topics include balancing progress with tradition, community-driven open-source projects, and the evolution of open source in technology.
Venture capital influence raises transparency concerns in tech community.
Community-driven open source development ensures sustainability and inclusivity.
Affordable resources aim to democratize tech advancements in open source.
Deep dives
Challenges in Open Source Accessibility and Impact of Venture Capital
Venture capital influence in the tech community is raising concerns about potential corruption and lack of transparency. The threat of private cabals working on projects creates barriers to participation and understanding. Access to compute power and specific data is identified as a key limitation hindering open source innovation and development.
Transition to Open Source Community Led Development
The shift from individual-led to community-driven open source development is vital in ensuring sustainability and growth. Projects like Pandas exemplify successful transitions where core developers take over leadership. Emphasizing public communication and decision-making fosters collaboration and inclusivity within the open source community.
Evolution Towards Open Source Dominance and Collaboration
Microsoft's evolution from closed to open source reflects the industry trend towards open collaboration. However, the spectrum of openness in tech companies varies, with Microsoft leveraging open source ID platforms like VS Code to expand service integration. The dual strategy of open source and revenue generation drives modern tech companies.
Mitigating Barriers in Open Source Innovation
Challenges related to cost and access to resources, such as computing hardware for large language models, inhibit broader participation and innovation in open source. Solutions like affordable GPU job startups and custom silicon development aim to address resource constraints and democratize technological advancements.
Future Prospects for Open Source and Data Innovation
Amidst resource barriers, advancements in open source efficiency and collaboration hold promise for future data science development. Efforts in modularization, horizontal integration, and shared runtime components indicate progress towards a user-focused, scalable, and collaborated data ecosystem. Exploration of innovations and shared technologies sets the stage for continuous advancement in the data science community.
Today I had the pleasure of speaking with Wes McKinney. Wes is well known in the open source community for his work building the pandas library. He also has a fascinating background leading the production of other data tooling with a focus on ubiquitous language-agnostic, hardware-optimized analytical computing. In this episode we learn about the origin of pandas, Wes' transition into a new role at Posit, and his perspective on the future of open source.
Podcast Sponsors, Affiliates, and Partners: - Pathrise - http://pathrise.com/KenJee | Career mentorship for job applicants (Free till you land a job) - Taro - http://jointaro.com/r/kenj308 (20% discount) | Career mentorship if you already have a job - 365 Data Science (57% discount) - https://365datascience.pxf.io/P0jbBY | Learn data science today - Interview Query (10% discount) - https://www.interviewquery.com/?ref=kenjee | Interview prep questions
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode