Episode 9: Why 90% of Data Science Fails—And How to Fix It -- With Eric Colson
Jan 30, 2025
auto_awesome
Eric Colson, former Chief Algorithms Officer at Stitch Fix and VP of Data Science at Netflix, discusses why 90% of data science initiatives fail. He emphasizes the need to treat data scientists as strategic drivers rather than mere service providers. Colson highlights the power of cognitive repertoires in problem-solving and advocates for a culture of experimentation, where trial and error leads to innovation. He also shares insights on restructuring data teams to transform them from cost centers into revenue generators, enhancing business value through collaboration.
Empowering data scientists as strategic drivers rather than support functions unlocks their potential for business innovation and growth.
Emphasizing trial and error in data initiatives fosters a culture of experimentation, leading to more significant achievements despite typical failures.
Decoupling algorithm development from application interfaces allows data scientists to prioritize impactful ideas and enhance their contributions to overall business strategy.
Deep dives
The Role of Data Scientists in Organizations
Data scientists are often perceived as a support function rather than essential contributors in organizations. This role limits their potential as they primarily respond to requests from business teams, which can undervalue their insights and ideas. It is essential for organizations to recognize the dual flow of ideas, where data scientists should be allowed to contribute concepts that drive business strategy. Encouraging collaboration will not only harness their full capabilities but also unlock significant opportunities that may otherwise be overlooked.
Emerging Centrality of Data Science
The role of data science is evolving to become increasingly central within companies, moving beyond traditional support structures. Companies are beginning to understand that data is not just an asset but a critical component for learning and innovation. As a result, there will be a growing demand for positions like chief data officers to influence company strategies effectively. This shift has broader implications for society, as learning from complex systems can yield valuable insights applicable beyond just business.
Leveraging Ideas from Data Scientists
Organizations often miss out on the innovative ideas data scientists can bring to the table by treating them as mere executors of predefined requirements. A significant opportunity lies in fostering an environment where data scientists can generate ideas that capture new possibilities for growth and efficiency. Mismanagement, such as overwhelming them with ad hoc requests, creates a vicious cycle, leading to wasted potential. A bidirectional exchange of ideas can enhance the value derived from their unique cognitive repertoires and insights.
Embracing Trial and Error in Experimentation
Trial and error is fundamental to the success of data-driven initiatives, allowing organizations to discover valuable strategies through experimentation. Data scientists typically face lower costs when exploring new ideas due to the accessible nature of data and experimentation tools. Once ideas are validated through small-scale experiments, they can be rolled out more broadly, amplifying positive outcomes. Understanding this asymmetry between wins and losses encourages a culture where taking calculated risks is rewarded rather than penalized.
Creating Spaces for Autonomous Data Science
To empower data scientists and maximize their impact, organizations should create dedicated spaces for algorithm development and experimentation. This begins by decoupling algorithmic logic from application interfaces, giving data scientists control over the insights and decisions derived from data. By fostering autonomy, data scientists can better prioritize revenue-generating ideas and reduce dependencies on engineering teams for execution. Such structural changes enhance motivation and accountability while positioning data scientists as key contributors to business success.
In this episode of High Signal, Eric Colson—former Chief Algorithms Officer at Stitch Fix and VP of Data Science and Machine Learning at Netflix—breaks down why most companies fail to unlock the full potential of their data science teams. Drawing from years of experience leading data functions at top tech companies, Eric shares how organizations can shift from treating data scientists as a service function to empowering them as strategic drivers of business impact.
Key topics from the conversation include:
Data Science as a Strategic Function: Why many companies limit their data teams to answering business requests instead of leveraging their ideas for competitive advantage.
Beyond Skills—The Power of Cognitive Repertoires: How data scientists' unique ways of framing problems can lead to breakthrough innovations.
Trial and Error as a Competitive Advantage: Why most experiments fail—but scaling experimentation is the key to big wins.
Decoupling Algorithms from Applications: How separating data science from engineering enables rapid iteration and direct business impact.
Shifting from Cost Center to Revenue Generator: Practical steps for structuring data teams to drive measurable value and long-term success.
💡 Tune in to learn how leading companies structure their data teams for impact, why experimentation beats rigid planning, and how treating data science as a strategic function can unlock new business opportunities.