The Joe Reis Show

Wes McKinney, the creator of Pandas and Apache Arrow, now works at Posit on Positron, a cutting-edge data science IDE. He delves into the innovative, React-based features of Positron and its integration with TypeScript and Jupyter. McKinney shares insights on optimizing data science with DuckDB and Wasm, enhancing workflows through AI, and navigating coding complexities. He reminisces about the creation of Pandas during financial turmoil and the evolution of Arrow for improved data processing. Outside of coding, he enjoys video gaming and language learning.

Tech debt is draining a massive chunk of IT budgets, with estimates as high as 60%. Legacy tech in big companies can feel like a ticking time bomb, especially with undocumented code. The discussion raises a provocative question: could AI be the solution to effectively tackle this issue? Large language models might just help in migrating those outdated systems. Plus, there's a peek into personal endeavors in the data industry.

Anne-Claire Baschet, Chief Data and AI Officer at Miracle, and Yoann Benoit, Head of Data at Himaya, tackle the intriguing concept of the 'Data Death Cycle.' They explore how misaligned priorities can derail data projects and stress the need for a mindset shift towards user-centric approaches. By drawing parallels between data and product development, they emphasize the significance of understanding user needs and enhancing collaboration among teams. Their insights illuminate how integrating AI early can lead to more successful outcomes.

Larry Burns, an expert in data teams, shares his insights on how these teams can overcome challenges to add real value to organizations. He discusses the struggle to move beyond being seen as cost centers and emphasizes the importance of integrating data roles into core business processes. Larry highlights the necessity of refining communication skills and fostering interdisciplinary collaboration. He also showcases transformative success stories, like a medical supply company’s pivot to a data-driven model, proving how effective data management can enhance customer satisfaction and drive revenue.

The podcast dives into the ethics of speaker compensation at major conferences. It highlights the disparity between steep ticket prices and the lack of payment for speakers, calling this imbalance exploitative. Insightful thoughts are shared on the need for greater respect and recognition of speakers' contributions. Listeners interested in speaking at conferences will find valuable perspectives on advocating for fair compensation.

Vijay Yadav, Director of Data Science at Merck, shares insights on integrating generative AI within the pharmaceutical sector. He discusses the critical need for high-quality data and structured approaches to optimize knowledge management and enhance medicine delivery. Vijay emphasizes the importance of linking product structures with transactions and utilizing graph databases for data quality. He also addresses the challenges of unstructured data and the need for user engagement in developing effective AI solutions. The conversation wraps up with thoughts on measuring success and reframing perceptions of AI.

In my newsletter last week, I wrote "Data’s still a mess. Most data initiatives fail. Data teams are seen as a cost center and not getting the support they deserve. Same as it ever was." Here, I unpack those four sentences. Data teams need to stop stop playing to not lose. Instead, they need to play to win!

Navnit Shukla is a solutions architect with AWS. He joins me to chat about data wrangling and architecting solutions on AWS, writing books, and much more. Navnit is also in the Coursera Data Engineering Specialization, dropping knowledge on data engineering on AWS. Check it out! Data Wrangling on AWS: https://www.amazon.com/Data-Wrangling-AWS-organize-analysis/dp/1801810907 LinkedIn: https://www.linkedin.com/in/navnitshukla/

I've spent the last three weeks visiting the UK, Australia, and New Zealand. Here are my observations and anecdotes about the data and ML/AI industry from countless chats with executives, practitioners, and pundits.

Ilya Reznik, an experienced leader in machine learning with a rich background at Adobe, Twitter, and Meta, shares his insights on building effective ML teams. He discusses the challenges of hiring in unfamiliar tech environments and the importance of robust data pipelines. The conversation also touches on the rapid adoption of large language models across industries and the complexities of AI implementation. Ilya highlights the need for innovative thinking and transparency in evaluating machine learning practices while fostering community engagement in Utah's tech scene.