In this episode of The Geek Narrator podcast, hosted by Kaivalya Apte, we welcome a special guest, Kishore Gopalakrishna from StarTree, co-author of Apache Pinot and other notable projects. Kishore shares his extensive experience in building real-time analytics and streaming systems, including Apache Pino, Espresso, Apache Helix, and Third Eye. The episode delves into the motivations and challenges behind creating these systems, the innovations they brought to distributed systems, and the impact of community on open-source projects. Kishore also discusses the evolution of testing methodologies, cost optimizations in transactional and analytical systems, and key considerations for companies evaluating real-time analytics solutions.
Don't miss this in-depth conversation packed with valuable insights for both seasoned developers and tech enthusiasts!
Chapters:
00:00 Introduction
03:13 Building Distributed Systems at LinkedIn
08:57 Testing and Challenges in Distributed Systems
30:50 Advantages of Columnar Storage
33:04 The Importance of Upserts
34:24 Building a Strong Open Source Community
41:10 Challenges and Lessons in System Design
51:35 Real-Time Analytics: Do You Need It?
StarTree: https://startree.ai/
Apache Pinot: https://pinot.apache.org/
If you like this episode, please hit the like button and share it with your network.
Also please subscribe if you haven't yet.
Database internals series: https://youtu.be/yV_Zp0Mi3xs
Popular playlists:
Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA-
Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17
Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d
Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN
Stay Curios! Keep Learning!
#distributedsystems #kafka #s3 #streaming #realtimeanalytics #database #pinot #startree
