CDO Matters Ep. 22 | Data Fabric Demystified
CDO Matters Podcast
The Role of Active Metadata in Data Fabrics
The more data you throw at this thing, the more data that can be used to train analysis and train the graphs. Data starts to inform its own classification management and use,. We move away from people and move towards more automated ways of doing that. So that's a kind of key thing. Number two, and number three, it's enterprise wide. It's across an organization. I really see this being an enterprise wide paradigm, especially for very, very big companies.
ChatGPT. Composable data and analytics. Connected governance. The data fabric.
These are all hot trends and much-hyped tools and frameworks in 2023. But one of the challenges with “bleeding-edge” technologies is a lack of a clear definition — much less a clear guide on how to take advantage of these nascent tools.
Good thing we have a former Gartner analyst with thousands of hours of client inquiries under his belt to help finally demystify one of the most-hyped trends in data management today: the data fabric.
In this 22nd episode of CDO Matters, Malcolm focuses exclusively on providing an extremely “deep dive” into the data management architecture known as the data fabric. The data fabric has gone from relative obscurity to nearly the top of Gartner’s hype cycle in just a few short years. For this reason, forward-leaning CDOs should understand exactly what the data fabric is and how it can benefit their organization.
Starting with a basic definition of a data fabric, Malcolm proceeds to break this rather complex phenomenon down into its component parts — using language that is precise but highly digestible for any non-technical CDO or data leader. Translating the complexity of a fabric into understandable critical capabilities, he shows how a data fabric can eventually be leveraged as a transformational tool for any organization.
For those wondering exactly what capabilities are needed to enable a data fabric, Malcolm reviews the top three capabilities that he believes must be present for any solution to be considered a data fabric. Armed with this information, it becomes clear that the data fabric remains — at least in the short term — an aspiration for most companies given the sophistication and governance maturity needed to enable them.
Malcolm reviews a conceptual architecture of the data fabric and discusses how fabric capabilities will be deeply integrated into — and across — several legacy data management systems, including master data management (MDM), data quality, data governance and data integration platforms. He challenges the notion that any one solution will be used to enable data fabrics — and instead outlines how several software and analytical solutions will need to be deeply integrated to enable fabric capabilities.
Finally, Malcolm ends his demystification of data fabrics by sharing a few key considerations to help CDOs cut through all of the hype related to data fabrics. This includes practical actions data leaders can take now to better position themselves for leveraging data fabrics soon.
Update from Malcolm: This episode was recorded before the production launch of ChatGPT. In the span of just a few weeks, I believe the data fabric has gone from a conceptual framework to something that could easily be envisioned within a modern data estate.
Put another way — if the entire internet before 2012 could be used to train an AI-enabled language model, all your enterprise data could most certainly be used to train AI models that are optimized to support data management use cases. I now believe the data fabric is how AI will be operationalized, at scale, to optimize — and eventually automate — the creation, consumption and management of data within your organization.
I’ve struggled for the last two years to visualize exactly how data fabrics could be implemented at scale, but thanks to ChatGPT, I no longer have this struggle. Hopefully after watching — or listening — to this episode, you come to a similar conclusion.
Key Moments
- [7:51] The Hype Behind the Data Fabric
- [10:46] Diving Deep into the Fabric
- [13:01] Data Fabrics Defined
- [21:31] Key Fabric Capabilities
- [22:15] Active Metadata
- [27:01] Incorporating AI Technologies
- [29:56] Synthesizing Metadata with an Intelligence Layer
- [35:31] The Data Fabric Architecture
- [40:06] Key Fabric Considerations
- [52:31] Closing Thoughts
Key Takeaways
The Future of the Data Fabric and AI Dependency (12:38)
“We are making a pivot away from people defining the [data governance] rules, the integration patterns, the data quality standards. We are moving away from people deciding that to robots deciding that. That’s a spectrum. We’re largely people-driven today…we’re early in the days of the spectrum from entirely people-driven to entirely robot-driven. We’re early in those days, but where the data fabric goes and the ultimate path here is towards a world highly dependent on the [machines].” — Malcolm Hawker
What Can Metadata Do for You? (17:40)
“What active metadata really means is that, if you had a lot of metadata, and you has some pretty sophisticated analytical tools, and you had some pretty sophisticated new technologies, you could make that data tell you a lot of things about the state of your data enterprise. For example, in theory, you could know when data was accurate or inaccurate.” — Malcolm Hawker
The Current State of Data Fabrics (46:12)
“Data fabrics don’t exist yet. You can’t go buy one. There is a ton of promise here. But between where we are, and where we need to go and between concept and theory, there are some really major roadblock issues we need to overcome. And frankly, there’s a lot of technology that doesn’t even exist yet.” — Malcolm Hawker
EPISODE LINKS & RESOURCES:
Remember Everything You Learn from Podcasts
