Data streaming and Benthos
Go Time: Golang, Software Engineering
00:00
Using Machine Learning to Solve a Data Engineering Problem
There's lots and they've all got their kind of specific use cases and operational complexities, which does factor in. So there's things like Spark is probably, I think if you're talking about data streaming, Spark is going to come up. And cool stuff like materialize, there's KSQLDB. Those are tools that will essentially solve a data engineering problem on top of that stream. It's usually around some sort of aggregation in real time of the data. The other side of data engineering is making that data what you want it to be. That used to be something that was done in a batched way, so every day, maybe you'd kick off a process that did that
Transcript
Play full episode
