The future of data science teams, integrating AI into data science workflows, building data apps for stakeholders - Barry McCardel - The Data Scientist Show #078
Jan 21, 2024
auto_awesome
Barry McCardel, Founder and CEO of Hex, talks about the future of data science teams, integrating AI into data science workflows, building data apps for stakeholders, and the potential of conversational user experiences in data work.
AI should enhance, not replace, data professionals; their value lies in critical thinking and effective communication.
Accelerating feedback loops between business questions and data-driven decisions can be facilitated by AI and tools like Hex.
Building a successful data team requires versatile professionals with skills in data infrastructure, analytics, and business impact.
Deep dives
AI Features Should Augment, Not Replace Human Insight
The skepticism towards AI replacing data professionals is unfounded. AI features should be used to enhance and expedite human insights, not replace them. Data work requires creativity, critical thinking, and human intuition that AI cannot fully replicate. While AI can accelerate certain tasks and eliminate mundane work, the value of data professionals lies in their deep understanding of problems, data analysis, hypothesis formation, and effective communication.
The Demand for Data Insights Remains Boundless
The demand for data insights in organizations is limitless. There are millions of questions and decisions that can benefit from data insights every day. However, many organizations face challenges in accessing timely and relevant data from their data teams. The goal should be to accelerate the feedback loops between business questions and data-informed decisions. AI and tools like Hax can play a role in improving efficiency and meeting the growing demand for data-driven insights.
Flexible and Collaborative Data Workspaces Are Key
Building a successful data team requires finding versatile professionals who can adapt to the evolving needs of the organization. When hiring the first data team members, consider analytics engineers who can set up data infrastructure, transform and clean data, and understand the business questions that need to be addressed. The modern data stack, with tools like cloud data warehouses, SQL, Python, ETL, and BI tools, has become the new reality. Collaborative and unified platforms like Hax can simplify workflows, consolidate tools, improve collaboration, and streamline data analysis and reporting processes.
The Importance of Full Stack Data Science
Full stack data science involves not only conducting research and building models, but also focusing on the business impact and improving decision-making. It goes beyond just the technical skills and encompasses the ability to get work into production and build interfaces or UIs. The goal is to bring the research to a point of impact and make it easily accessible without requiring deep knowledge of specific packages or frameworks.
Measuring ROI and the Role of Data Practitioners
Measuring the ROI of a data team should go beyond financial models and focus on the satisfaction and value provided to stakeholders. Instead of calculating ROI based on complex financial metrics, it is more insightful to gauge the stakeholders' willingness to recommend and desire for more collaboration with the data team. The success of a data team lies in their impact on decision-making, the ability to monitor and improve models, and their integration with functional teams within the organization.
Barry McCardel is the cofounder and CEO of Hex(free trial: hex.tech/dsshow), a collaborative data workspace. Their customers include FiveTran, Notion, and Anthropic. We talked about what does the future of data team look like, how to tackle challenges of data team collaborations, and how to leverage AI in data science’s workflow.
60-day Free Trial: hex.tech/dsshow
Barry’s LinkedIn: https://www.linkedin.com/in/barrymccardel
(00:00:00) Introduction
(00:01:25) Is AI replacing data scientists?
(00:06:08) Are data science teams getting smaller?
(00:09:54) What is Hex?
(00:11:24) How to communicate with stakeholders
(00:24:29) Should data scientists be full stack?
(00:31:23) How data team measure ROI
(00:33:35) Quantitative vs qualitative analysis
(00:35:33) When you shouldn't use data? Data vs product intuition
(00:41:39) How to hire your first data team?
(00:48:59) Is the modern data stack dead?
(00:53:55) GenAI in data science workflows
(00:59:03) Future of data scientist
(01:02:30) New features in Hex
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
