The GeekNarrator

Hey Everyone, In this episode I talk to Anjul Sahu, Founder of CloudRaft, about all things Kubernetes. There are many common questions that we try to answer. If you ever wondered if you should adopt Kubernetes as a company? or learn Kubernetes as a Developer? or why people fail adopting Kubernetes? Or even if Kubernetes is worth the hype? Then this episode will bring a ton of clarity. Chapters 00:00 Introduction 02:37 History and Background 06:06 k8s use cases 08:24 k8s features 09:34 Some basics of k8s 11:54 Common problems people face with k8s 15:48 k8s - challenges 18:59 Should you use k8s? 24:02 Alternatives of k8s 26:54 Top reasons why people fail with k8s 31:38 As a dev - should I learn k8s? Is it good for my career? Do give this episode a like and share it with your network. Also please subscribe to the channel for content like this. Other playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Anjul Sahu: https://anjul.dev/ Cloudraft: https://www.cloudraft.io/ Our website: www.geeknarrator.com References: Kubernetes course: https://kodekloud.com/courses/kubernetes-for-the-absolute-beginners-hands-on/ Kubernetes Hard Way: https://github.com/kelseyhightower/kubernetes-the-hard-way Kind: https://kind.sigs.k8s.io/ k3s: https://k3s.io/ k0s: https://k0sproject.io/ Cheers, The GeekNarrator

In this video I talk to David Jacot who works as a Staff Software Engineer at  @Confluent  and has been a long time Kafka user, committer and PMC member. We covered how Kafka works internally in great depth. We use Kafka for various use cases and it works great, but going one level below the abstraction and truly understanding the protocols, techniques and algorithms used is a fun ride. Chapters: 00:00 Kafka Internals with David Jacot 03:33 Defining Kafka 05:16 Kafka Architecture(s) 11:39 Write Path - Producer sending data 18:35 How does replication work? 25:47 How do we track replication progress? 30:42 Failure Modes: Leader fails 38:18 Consumers: Push vs Pull 40:54 Consumers: How does fetch works? 49:03 Consuming number of bytes vs records 50:50 Optimising consumption 01:00:21 Offset management and choosing partitions 01:09:10 Ending notes I hope you like this episode and more importantly you learnt some amazing techniques Kafka uses to ensure durability, low latency, simplicity and scalability in its architecture. Do give this episode a like and share it with your network. Also please subscribe to the channel for content like this. Other playlists: Realtime streaming systems: https://www.youtube.com/playlist?list=PLL7QpTxsA4se-mAKKoVOs3VcaP71X_LA- Software Engineering: https://www.youtube.com/playlist?list=PLL7QpTxsA4sf6By03bot5BhKoMgxDUU17 Distributed systems and databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4sfLDUnjBJXJGFhhz94jDd_d Modern databases: https://www.youtube.com/playlist?list=PLL7QpTxsA4scSeZAsCUXijtnfW5ARlrsN Other episodes: KsqlDB: https://youtu.be/2yE86P6uD_0 Exactly once semantics: https://youtu.be/twgbAL_EaQw David's Linkedin: https://www.linkedin.com/in/davidjacot/ our website: www.geeknarrator.com Cheers, The GeekNarrator

In this episode I talk to @gkcs on the Practical system design for Senior Engineers. Gaurav brings in a ton of experience in Competitive Programming, Teaching System Design and working on real world problems at companies like @Uber and Directi. Currently he is running his startup called @interviewready_ that helps folks perform better at Interviews. Do check him out if you haven't. I truly believe System Design in the real world requires ton of other skills along with great technical skills. It becomes even more critical at Senior levels. So in this episode we discuss our view and experience on the same while providing some concrete examples from our real world experience. I hope this episode brings in some clarity and adds some value to folks at all levels. At junior level, you would know what to expect and how to approach, At senior level, you would know what you might be missing or can do more to get to the next level. Chapters: 00:00 System Design as Senior Engineers - Things that no one talks about about. 04:11 Fundamentals 08:32 Do you get to choose the Tech stack? 16:47 Thinking about the future while being fast 21:18 Real world vs Interviews 23:45 Importance of Communication Skills 30:50 IC does not mean Isolated Contributor 35:20 Senior Engineering is a journey not a Certificate. 39:17 Volatility, Uncertainity, Complexity, Ambiguity 42:18 Importance of Decision Making 47:49 Reducing the blast radius 53:10 Leading by example Give it a watch and share with your network. Hit the like button and don't forget to subscribe to the channel. Cheers, The GeekNarrator

Hey Everyone, In the 43rd episode I speak with Tim Berglund on Realtime Analytics with Apache Pinot. Chapters: 00:00 Introduction 01:22 What do we mean by analytics and realtime analytics? 05:35 Can we define realtime in millis, seconds or minutes? 08:54 What is the fundamental difference between traditional analytics systems and Apache Pinot? 12:19 Was Kafka one of the reasons Apache Pinot could reach its full potential? 16:50 E-commerce Application example - How do I get my data in? 20:07 How is data stored (structured) on the disk? 23:31 Are joins available in Apache Pinot? 26:07 Joins vs pre-computing at ingestion 27:15 How is historical data ingested into Apache Pinot? 28:14 Types of indexes available in Apache Pinot 35:42 Do indexes cause write amplification? Is that a problem in Apache Pinot? 40:02 Point lookups in Apache Pinot 42:54 Anamoly Detection 45:51 Coming up in Apache Pinot Links: StarTree https://startree.ai/ Apache Pinot: https://pinot.apache.org/ Joins in Pinot: https://startree.ai/blog/apache-pinot... Apache Pinot Indexes: https://docs.pinot.apache.org/basics/... Other playlists: Distributed systems:    • Distributed Syste...   Modern Databases:    • Modern Databases   Serverless Architecture:    • Serverless Archit...   Software Engineering:    • Software Engineering   I hope you like the episode. Like, share and subscribe to the channel. Cheers, The GeekNarrator

In this video I talk to Philip Fried from Estuary about Batch vs Realtime Stream Processing. Philip brings a ton of experience in the world of data processing and has shared some of the best practices in implementing these systems. We dive deep into the world of data processing, covering batch and streaming systems, their challenges, tradeoffs and use cases. Chapters: 00:00 Batch vs Realtime Stream Processing 03:25 What is Batch and Reatlime processing? 18:29 How does Batch and Realtime compare in terms of Latency and Throughput? 27:24 Where is the cost saving coming from? Compute?Storage? or Network? 31:38 Moving from Batch to Stream processing 37:50 How is Idempotency implemented in Streaming systems? 48:50 How do we approach Schema evolution in Batch and Streaming systems? 57:16 Summary - key points to keep in mind Do checkout Estuary if you deal with a ton of data, and don't want to deal with the painful operations, infrastructure management, schema migrations etc and only want to focus on building highly scalable and resilient applications. References: Estuary: https://estuary.dev/ Flow documentation: https://docs.estuary.dev If you like this video please hit the like button, share it with your network (whoever works with a ton of data) and subscribe to the channel. Feel free to watch related episodes in the playlist:    • Distributed Syste...   Modern Databases:    • Modern Databases   Software Engineering:    • Software Engineering   Distributed Systems:    • Distributed Systems   Cheers, The GeekNarrator

Hey Everyone, In this video I talk to Patrick McFadin from DataStax. We uncovered the new features in Cassandra 5.0 and discussed how ACID transactions are achieved in the new version. This is a deep dive into the features of Cassandra, consensus protocols and how Accord is different as compared to Paxos, RAFT, Spanner and Calvin. Chapters: Cassandra 5.0 - ACID transactions and Vector Search 00:00 Introduction 01:45 List of features in the new Cassandra 04:51 Who needs ACID properties? 07:20 Why didn't Cassandra have ACID properties so far? 10:35 Why is Accord consensus protocol well suited for Cassandra? 16:40 Lets take a gaming example to see how Transactions work 21:55 Whats happening behind the scenes in a Transactions? 27:44 What happens when there are failures? 33:41 How is upgrade to the new version going to look like? 35:48 How is the latency impacted because of transactions? 40:23 What was missing in lightweight transactions? 42:24 Vector Search - What is it? How does it work? Previous episode on Cassandra:    • Apache Cassandra ...   Other playlists to watch: Distributed systems and Databases:    • Distributed Syste...   Software Engineering:    • Software Engineering   Distributed systems:    • Distributed Systems   Modern Databases:    • Modern Databases   Patrick's Linkedin: https://www.linkedin.com/in/patrick-m... Astra: astra.datastax.com Cassandra: https://cassandra.apache.org/_/index.... I hope you liked this episode. If you did, please hit the like button, share it with your network and subscribe to the channel. Cheers, The GeekNarrator

Hey Everyone, In this episode I discuss Twisp's journey with Mike Parsons, CTO and Co-Founder of Twisp. We have discussed how they ended up building a cloud native distributed ledger database on top of DynamoDB, why they chose DynamoDB? what were the challenges they faced and many highly technical and very interesting topics. Chapters: 00:00 Introduction - Twisp - CTO and Co-Founder Mike Parsons 03:01 What do we mean by a Distributed Ledger Database? 07:13 Is it similar to Blockchain? 08:32 Challenges faced with Relational Database setup 19:39 Why DynamoDB was the right choice? 24:40 What was the scale you were building for? 26:05 Who uses a Ledger database? 29:30 What are the basic primtives in a ledger database? 33:55 Is every primitive Immutable? How is it stored? 38:07 Impact of Immutability on Data storage 41:33 How does idempotency work in Twisp? 44:53 How did you provide ACID guarantees on top of DynamoDB? 52:30 How do you ensure correctness? 57:13 Interesting real world usecases where Twisp is used. 01:02:33 Are you planning to build even higher level primitives? References: Clocks: https://cse.buffalo.edu/tech-reports/... and http://rystsov.info/2018/10/01/tso.html TicToc: https://people.csail.mit.edu/sanchez/... Hekaton Paper: https://www.microsoft.com/en-us/resea... Hermitage Test suite: https://github.com/ept/hermitage Online Async Schema Change in F1 : https://static.googleusercontent.com/... Designing Data intensive Applications: https://www.oreilly.com/library/view/... P Programming language: https://github.com/p-org/P Twisp: https://www.twisp.com/ AWS Dynamo DB: https://aws.amazon.com/pm/dynamodb/?t... Connect with Mike: https://www.linkedin.com/in/parsnips/ Follow me: Twitter: https://twitter.com/thegeeknarrator Linkedin: https://www.linkedin.com/in/kaivalyaa... Watch other playlists: Databases:    • Modern Databases   Distributed systems:    • Distributed Systems   Software Engineering:    • Software Engineering   Sofk skills:    • Soft Skills   Like, share and subscribe to the channel folks. See you in the other episode. Cheers, The GeekNarrator

In this episode I talk to Yan Cui, who is an AWS Serverless Hero, all about Serverless technologies. Chapters: 00:00 Serverless Architecture with Yan Cui 01:58 What do we mean by Serverless Architecture? 05:42 What is the core problem Serverless solves? 11:06 Do we need to think differently to be able to use Serverless? 15:27 What is the difference between serverless and managed services? 19:17 Is Vendor Lock-in really a problem? 27:42 Multicloud - Is it really worth it? 33:46 Is ColdStart a real problem? What kind of apps get impacted? 43:25 Monitoring serverless applications 48:22 Usecases when serverless may not be the best solution 54:27 Future of serverless 57:31 How should a developer learn about serverless? I hope you enjoy the discussion and learn from it. Please hit the like button, share it with your network and also subscribe to the channel. References: Yan Cui - https://theburningmonk.com Courses - https://productionreadyserverless.com/ Corey Quinn on MultiCloud -    • Corey Quinn: The ...   Linkedin Yan - https://www.linkedin.com/in/theburnin... Twitter Yan - https://twitter.com/theburningmonk Other playlists to watch: Distributed Systems and Databases -    • Distributed Syste...   Software Engineering -    • Software Engineering   Distributed systems practices -    • Distributed Systems   Cheers, The GeekNarrator

It was an honour to host Dr. Venkat Subramaniam on the podcast. Dr. Venkat Subramaniam is an award-winning author, founder of Agile Developer, Inc., and an instructional professor at the University of Houston. He has mentored tens of thousands of software developers in the US, Canada, Europe, and Asia, and is a regularly-invited speaker at several international conferences. Venkat helps his clients effectively apply and succeed with agile practices on their software projects. In this episode, Dr. Venkat has shared his experience on becoming productive, managing time effectively, learning and Software Engineering. There is so much to learn from the way Venkat approaches learning and keeping up with the pace. No wonder, how he is able to do so many different things. Chapters: 00:00:00 Introduction into Venkat's Journey 00:04:23 How is Venkat able to do so many things? 00:15:23 Saying NO and prioritisation 00:44:42 Keeping up with the pace and learn 00:53:55 Engineering practices every engineer must follow 01:14:40 Ending notes, resources etc.. This episode is full of stories, experiences and learnings from Dr. Venkat's vast experience. The analogies he has shared are amazing. I hope you like this discussion and also learn a ton of things that matter. Venkat Subramaniam: https://twitter.com/venkat_s Agile Developer: https://agiledeveloper.com/ If you like this discussion, please like, share, subscribe and press the bell icon. Keep supporting and more amazing discussions coming your way. Cheers, The GeekNarrator

Daniel Ciocîrlan from RockTheJVM discusses functional programming on the JVM, including concepts, learning approaches, object-oriented programming, immutability, recursion, concurrency models, Project Loom, and a comparison of Scala and Kotlin. They also explore mental models for parallel, concurrent, and distributed systems, and the importance of building something real when learning Scala.