AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Evolution of Data Quality and Introduction to Data Contracts
The modern AI stack is anticipated to be richer and more complex than the previous data stack. Data quality involves monitoring the flow of data, detecting changes in sources, distribution, volume, and shape of data. Tools like Monte Carlo and test-based approaches are utilized for understanding and ensuring data quality. Data contracts, akin to microservices in software engineering, entail declaring inputs, outputs, guarantees on performance to facilitate collaboration and data management in a decentralized data environment. This ensures that changes in data format are communicated effectively to prevent disruptions in downstream systems.