
Data Pipeline Testing with Bartosz Mikulski
The Engineering Side of Data
00:00
How to Handle Bad Data in Production
When these tools see a range, a value of a column falls outside of a range, right? And you get an error. Is that, how do you recommend that typically handled in production? Are those errors logged? Are there notifications sent out? What have you seen as a good way to kind of mitigate that bad data once these tools find it in production? I'm not sure if that's good. But what makes sense for me is by separating the data into, like, maybe two buckets, like the data that is correct and data that is incorrect.
Transcript
Play full episode