The likes of LinkedIn and Uber use Pinot to power some astonishingly high-scale queries against realtime data. The numbers alone would make an impressive case-study. But behind the headline lies a fascinating set of architectural decisions and constraints to get there. So how does Pinot work? How does it process queries? How are the various roles split across a cluster? And equally important - what does it *not* try to achieve.
Joining me to go through the nuts and bolts of how Pinot handles SQL queries is Tim Berglund, veteran technology explainer of the realtime-data world. He takes us through Pinot step-by-step, covering the roles of brokers, servers, controllers and minions as we build up the picture of a query engine that's interesting in theory and massively performant in practice.
–
Apache Pinot: https://pinot.apache.org/
Apache Pinot Docs: https://docs.pinot.apache.org/
StarTree: https://startree.ai/
Event Driven Design episode with Bobby Calderwood: https://youtu.be/V7vhSHqMxus
Tim on Twitter: https://twitter.com/tlberglund
Kris on Mastodon: http://mastodon.social/@krisajenkins
Kris on LinkedIn: https://www.linkedin.com/in/krisjenkins/
Kris on Twitter: https://twitter.com/krisajenkins
–
#podcast #softwaredevelopment #apachepinot #database #dataengineering #sql
