Building a Reliable Kafka Data Processing Pipeline With Lily Mara
The InfoQ Podcast
00:00
The Challenges of Writing a High Level Kafka Consumer
We're at the point now where we haven't even introduced this asynchronous processing for this particular endpoint, right? So this is all happening synchronously inside of our HTTP handlers. And so you're talking about the input stream, like you're getting more requests than you can handle. It wasn't that the async going out was being too slow. It was you got thousands or millions of requests a second and that was overloading Postgres.
Transcript
Play full episode
