CDO Matters Ep. 43 | Practical AI and Data Science
Feb 8, 2024
auto_awesome
Former nuclear physicist turned data scientist, Santona Tuli, shares actionable insights for data leaders implementing AI and data science. Topics include measuring unseen phenomena, challenges in data observability, impact of variables on business performance, data-driven decision making, leveraging knowledge graphs, challenges with data, and the future of AI.
Utilizing knowledge graphs and alternative information architectures can help uncover valuable insights hidden in company data.
Data professionals should strive to combine data analysis with heuristics and instincts for a holistic understanding.
Deep dives
Unlocking the Value of Production Data
UpSolver specializes in ingesting streaming data sources like message buses and databases to help organizations bring their valuable application data into their data infrastructure. They offer solutions for landing data in warehouses like Snowflake or lake houses using technologies such as Apache Iceberg. Their goal is to unlock the untapped value of production data, which is often overlooked when focusing on known KPIs and dashboards. By leveraging different information architectures and integrating various data sources, organizations can gain valuable insights that were previously hidden.
The Challenges of Utilizing AI to Transform Business
Many companies have vast amounts of data that remain untapped and could potentially transform their businesses. However, leveraging AI to discover these hidden insights requires significant resources, experimentation, and R&D. Data leaders face the challenge of navigating this process without risking their positions. One approach is to focus on utilizing knowledge graphs to uncover new relationships and patterns within the data. By exploring alternative information architectures and adopting specialized data stores, such as graph databases or search databases, companies can begin uncovering valuable insights that were previously overlooked.
The Future of AI and AGI
While the future of AI is promising, the concept of Artificial General Intelligence (AGI) surpassing human intelligence is not imminent. AGI development may continue in the future, but it is unlikely to be achieved within the next few decades. The limitations of processing power and the current understanding of AI indicate a longer timeframe for achieving AGI. It is crucial to proactively consider the laws, governance, and ethics surrounding AGI development to foster a positive future and avoid dystopian scenarios.
Balancing Data-Driven Decisions and Gut Feelings
Making data-driven decisions is vital, but it's essential to remember that gut feelings and experiences also play a significant role. Data professionals should aim for a holistic understanding by combining data analysis with heuristics and instincts. While numbers and data collection are valuable, it is crucial to assess biases, assumptions, and limitations when communicating data to stakeholders. Ideally, data professionals should strive to incorporate uncertainty ranges and error bars into their analysis to provide a more accurate representation of data and avoid misleading interpretations.
In this 43rd episode of the CDO Matters Podcast, Malcolm interviews Santona Tuli, a former Nuclear Physicist turned data scientist. Santona shares some provocative and actionable insights to data leaders looking to take a more practical approach to implementing AI and data science into their organizations.
If you’re looking to show value from AI, quickly, Santona shares her recommendations for everything that data leaders should be considering in their search for the optimal way to embrace these transformative new technologies.