Data streaming and Benthos
Go Time: Golang, Software Engineering
Using Machine Learning to Solve a Data Engineering Problem
There's lots and they've all got their kind of specific use cases and operational complexities, which does factor in. So there's things like Spark is probably, I think if you're talking about data streaming, Spark is going to come up. And cool stuff like materialize, there's KSQLDB. Those are tools that will essentially solve a data engineering problem on top of that stream. It's usually around some sort of aggregation in real time of the data. The other side of data engineering is making that data what you want it to be. That used to be something that was done in a batched way, so every day, maybe you'd kick off a process that did that
00:00
Transcript
Play full episode
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
