Sign up for Data Mesh Understanding's free roundtable and introduction programs here: https://landing.datameshunderstanding.com/
Sponsored by NextData, Zhamak's company that is helping ease data product creation.
For more great content from Zhamak, check out her book on data mesh, a book she collaborated on, her LinkedIn, and her Twitter.
This episode is part of the greater AI/ML conversation I had with Zhamak. To start, Zhamak recognizes we aren't where we want to be in terms of capabilities - ways of working or tooling - to make this a reality just yet. But, if we can make it so data scientists can trust and easily consume from data products - that we create data products that don't care what use case type - regular analytics or AI/ML - can we remove a lot of the complexity they face? Do they need feature stores for data they aren't transforming? If they can get continued access and know the quality, why create a separate process that has fragility instead of trust the data product owners upstream?
I wasn’t smart enough in the moment to talk about do we need to have a copy of the training data itself for reproducibility but folks smarter on ML than I am can answer that one, probably in the affirmative. But overall, there is a lot of complexity in the way we do AI/ML because data scientists can't trust the sources of their data and they feel the need to take control because if they don't, their models break. So we need to earn their trust and show them a better way. But again, we aren't there yet, so let's work to make this a reality in the future.
Please Rate and Review us on your podcast app of choice!
If you want to be a guest or give feedback (suggestions for topics, comments, etc.), please see here
Data Mesh Radio episode list and links to all available episode transcripts here.
Provided as a free resource by Data Mesh Understanding / Scott Hirleman. Get in touch with Scott on LinkedIn if you want to chat data mesh.
If you want to learn more and/or join the Data Mesh Learning Community, see here: https://datameshlearning.com/community/
All music used this episode was found on PixaBay and was created by (including slight edits by Scott Hirleman): Lesfm, MondayHopes, SergeQuadrado, ItsWatR, Lexin_Music, and/or nevesf
Data Mesh Radio