CDO Matters Ep. 33 | What CDOs Need to Know about Data Engineers with Joe Reis
Sep 21, 2023
auto_awesome
Joe Reis, co-author of the bestselling O'Reilly book, 'Fundamentals of Data Engineering', discusses the importance of understanding and managing data engineers effectively. They also talk about the challenges and strategies for bridging the gap between development and data teams, as well as the future of models in data engineering and the significance of having a clear data strategy.
Data engineering forms the foundation for successful data science and machine learning initiatives.
Collaboration and understanding between application development teams and data engineering teams are crucial for productive work environments.
A customer-centric approach and alignment with business outcomes are essential for effective data initiatives.
Deep dives
Understanding the Importance of Data Engineering
The podcast episode emphasizes the significance of data engineering as the foundation for successful data science and machine learning initiatives. The speaker discusses the temptation for companies to focus solely on AI and machine learning without establishing a solid data engineering foundation. The importance of data engineering in setting the stage for data-driven success is highlighted, as well as the challenges that can arise when organizations neglect this crucial step.
Bridging the Dev and Data Divide
The podcast explores the divide between application development teams and data engineering teams, stressing the need for better collaboration and understanding between the two. The speaker emphasizes the importance of building empathy and relationships between the teams to foster a cohesive and productive work environment. The discussions highlight the challenges that can arise from the lack of communication and cooperation between dev and data teams, and the potential benefits that can be realized when they work together towards a common goal.
Taking a Customer-Centric Approach
The podcast episode emphasizes the importance of prioritizing a customer-centric approach to data initiatives. The speaker discusses the significance of focusing on the needs and preferences of the end customer and aligning data strategies with those objectives. They stress the need for strong leadership and a clear understanding of business outcomes to drive data initiatives effectively. The discussions highlight the importance of designing products and solutions that are intuitive, easy to use, and directly contribute to delivering value to customers.
Considering the Need for Data Science
The podcast delves into the question of whether a new CDO needs a dedicated data science team. The speaker urges CDOs to thoroughly assess the current state and goals of the organization before deciding on the need for data science capabilities. They discuss the potential advantages and trade-offs of building a custom data science team versus leveraging off-the-shelf solutions or dedicated models. The importance of having clarity on the company's data strategy and goals is highlighted in making informed decisions regarding data science resources.
Revisiting the Importance of Data Modeling
The podcast episode underscores the significance of data modeling in the data landscape. The speaker notes that data modeling practices have been overlooked and underutilized in recent years, highlighting the absence of focus on concepts like conceptual and logical data modeling. They discuss the need to revisit and evolve data modeling practices to meet the evolving needs of data-intensive initiatives. The speaker also highlights the value of a comprehensive approach to modeling data across the entire data lifecycle, encompassing creation, usage, and application in analytics and machine learning.
Data Engineers play a critical role in enabling the success of any data and analytics function. In this episode of the CDO Matters Podcast, Malcolm interviews Joe Reis, the co-author of the bestselling O’Reilly book, “Fundamentals of Data Engineering”. In the discussion, Joe gives a sneak peek into the minds of data engineers and what makes them tick. Data leaders, particularly those with less technical backgrounds, should find this discussion a helpful tool to improve their relationships with the people critical to the success of their data and analytics operation.