MLOps Coffee Sessions #77 with Scott Hirleman, Data Mesh - The Data Quality Control Mechanism for MLOps?
// Abstract
Scott covers what a data mesh is at a high level for those not familiar. Data mesh is potentially a great win for ML/MLOps as there is very clear guidance on creating useful, clean, well-documented/described, and interoperable data for "unexpected use". So instead of data spelunking being a harrowing task, it can be a very fruitful one. And that one data set that was so awesome?
Well, it wasn't a one-off; it's managed as a product with regular refreshes! And there is a LOT more ownership/responsibility on data producers to make sure the downstream doesn't break. Might sound like kumbaya for MLOps (or total BS?) re far cleaner data and fewer upstream breaks, so let's discuss the realities and limitations!
// Bio
A self-professed "chaotic (mostly) good character", Scott is focused on helping the data mesh community accelerate towards finding solutions for some of data management's hardest challenges. He founded the Data Mesh Learning community specifically to gather enough people to exchange ideas, much of which is patterned after the MLOps community. He hosts the Data Mesh Radio podcast, where he dives deep into topics related to data mesh to provide the data community with useful perspectives and thoughts on data mesh.
--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletter, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Adam on LinkedIn: https://www.linkedin.com/in/aesroka/
Connect with Scott on LinkedIn: https://www.linkedin.com/in/scotthirleman/
Timestamps:
[00:00] Takeaways
[04:47] Merchandise
[05:50] What is data mesh?
[08:17] What is a data product?
[11:14] Second layer of data mesh
[13:15] Data standards
[15:51] Third layer of data mesh
[17:13] Cultural aspect of data mesh
[21:56] Data mesh documentation
[24:29] Tooling challenges
[27:55] Data mesh in practice
[31:40] Difference in experiences
[36:05] Baby steps to a fully pledged data mesh
[42:05] How data mesh relates to ML
[48:30] Data mesh vs data mess jokes
[49:02] High risks in data mesh
[52:47] Quick wins
[56:10] Wrap up
