Developer Voices

Ever read a bad README? We all have, and most of the time, we’ve just moved right along. A programmer that can’t communicate their ideas will find no-one uses their software. And that’s true even outside of the open-source world. The best software doesn’t win - the best software _that people can understand_ wins. So how do we get better at communicating our code? What do we talk about when we talk about software?Joining to discuss that question is a data-streaming expert and skilled communicator, Francesco Tisiot. Unusually, this episode is recorded on location, as we met up in the hallway of a recent tech conference.Francesco on Twitter: https://twitter.com/FTisiotFrancesco on LinkedIn: https://www.linkedin.com/in/francescotisiot/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/#podcast #podcasts #devrel #opensource #software #presentations

Zain Hasan, a semantic search and augmented LLMs expert, joins the podcast to discuss the challenges of teaching large language models. They explore the concept of vector databases and their role in enhancing chat bots. The episode delves into optimizing search in a fictional service, the size and storage of indexes in vector databases, and the concept of multi-modality in vector search. The discussion also includes implementing semantic search at home using Weaviate and a conversation on Weaviate, an open-source database with managed instances.

Real-time data is gradually becoming a standard requirement in systems design. Our customers are beginning to demand it, our colleagues in other departments are starting to expect it. Whether you’re letting people book a taxi, recommending their next binge-watch, or delivering business reports to management, faster data is just obviously better. Or is it?Does real-time data matter everywhere, or does it just have sweet spots in some sectors and some use-cases? Is it a cost-benefit question - is the idea great in theory, but still too hard to adopt in practice? Would everyone be streaming their data live if streaming their data live was easier? If the future of data is, “now, not later,” then what’s holding that future back?In this week’s Developer Voices we talk to Thomas Camp of Ably, and chew through the use-cases, software stacks, and education needed to speed up the way we process data. We consider everything from the front-end to the back, from user experience to business needs, and from greenfield projects to incrementally adapting existing systems.If you’re wondering what all the batch vs. streaming fuss is about, or you want to know how you can drag the industry there sooner, we have some answers. It’ll only take an hour. 😉--Thomas on LinkedIn: https://www.linkedin.com/in/thomascamp333/Ably: https://ably.com/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

How far would you go to get the kind of database you want? How deep into the stack would you dive to re-architect a system for the kind of performance, reliability and scale you believe in? Today's guest has decided to go all in, as he’s tackling the database problem from the fsync up. In this week’s Developer Voices we talk to Joran Dirk Greef, whose ambitions—combined with the lacklustre performance of his project's payment system—have led him to build a new database called TigerBeetle, that tackles some meaty problems. They’re attempting to build a database that can be durable in the face of fsync-corner cases, highly available in the face of all kinds of hidden network problems, and performant enough to outpace existing financial systems. And on top of all those goals, they’re doing it with an interesting new language you may not have heard of - Zig.What makes him want to take on this big a challenge? What problems keep him awake at night? And what is he doing to turn all that ambition into an achievable launch strategy? Listen on and find out…–TigerBeetle on Twitter: https://twitter.com/TigerBeetleDBTigerBeetle on YouTube: https://www.youtube.com/channel/UC3TlyQ3h6lC_jSWust2leGgKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Joran’s QCon ‘23 Talk: https://www.youtube.com/channel/UC3TlyQ3h6lC_jSWust2leGgViewstamped Replication Revisited (paper): https://pmg.csail.mit.edu/papers/vr-revisited.pdfGithub Test Cases for Journal recovery code: https://github.com/tigerbeetle/tigerbeetle/blob/b4dd441502894cbe9d48cb90ff0bc6a12c378591/src/vsr/journal.zig#L1181-L1213MySQL transactions per second vs fsyncs per second: https://sirupsen.com/napkin/problem-10-mysql-transactions-per-second

Michael Drogalis, a seasoned entrepreneur known for his Kafka-based company acquisition, is pushing boundaries by attempting to launch four tech businesses in a year. He shares insights on the emotional rollercoaster of entrepreneurship, the importance of aligning personal values with goals, and overcoming the $10,000 demo problem with innovative data solutions. Michael emphasizes the need for customer engagement, storytelling, and resilience, while embracing public vulnerability in his journey. It's a blend of intuition, experimentation, and honest reflections on startup life.

Software developer Ben Gamble joins the podcast to discuss the challenges of assembling a complete software system, the significance of open source, exploring data platforms and software building, optimizing data access and analysis, challenges of joining large data sets and real-time data processing, and the benefits and challenges of different database systems.

Alasdair Brown, a data expert with Tiny Bird, dives into the world of ClickHouse, a high-performance analytics database. He explains how ClickHouse facilitates lightning-fast queries on massive datasets and its origins as a Google Analytics-like tool. The discussion touches on the evolution from traditional databases to scalable solutions, overcoming data integration challenges, and the importance of real-time data management. Alasdair also shares tips for beginners on utilizing ClickHouse and TinyBird for an enhanced data experience.

Neil Buesing, a seasoned Apache Kafka® user and respected voice from the Kafka community, discusses the problems Kafka can solve, including queuing, real-time transitioning, and software integration headaches. He also explores the challenges of adopting new technology, the relationship between Kafka and microservices, and achieving client satisfaction. The significance of structured business data and the future prospects of adopting new technologies are also examined.

How do distributed systems work? If you’ve got a database spread over three servers, how do they elect a leader? How does that change when we spread those machines out across data centers, situated around the globe? Do we even need to understand how it works, or can we relegate those problems to an off the shelf tool like Zookeeper?Joining me this week is Distributed Systems Doctor—Benjamin Bengfort—for a deep dive into consensus algorithms. We start off by discussing how much of “the clustering problem” is your problem, and how much can be handled by a library. We go through many of the constraints and tradeoffs that you need to understand either way. And we eventually reach Benjamin’s surprising message - maybe the time is ripe to roll your own. Should we be writing our own bespoke Raft implementations? And if so, how hard would that be? What guidance can he offer us? Somewhere in the recording of this episode, I decided I want to sit down and try to implement a leader election protocol. Maybe you will too. And if not, you’ll at least have a better appreciation for what it takes. Distributed systems used to be rocket science, but they’re becoming deployment as usual. This episode should help us all to keep up!--KubeCon talk on the FCD bug: https://kccncna2022.sched.com/event/182N9/lessons-learned-from-etcd-the-data-inconsistency-issues-marek-siarkowicz-google-benjamin-wang-vmwareThe Raft paper by Diego Ongaro and John Ousterhout: https://raft.github.io/raft.pdfThe EPaxos Algorithm: https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdfLevelDB: https://github.com/google/leveldbBenjamin on Twitter: https://twitter.com/bbengfortBenjamin on LinkedIn: https://www.linkedin.com/in/bbengfortBenjamin on GitHub: https://github.com/bbengfortRotational Labs: https://rotational.io (check out the blog!)Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/

How do you get started as a programmer? And how do experienced programmers help them as they start their journey?This week's guest is a developer-turned-teacher, James Q. Quick. A former coder and developer advocate, he's s been been working on tutorials, courses and bootcamps to teach aspiring developers how to get started with JavaScript. We talk about why people get into programming as a new career, what they need to succeed, and what James thinks is the best indicator of success.We also discuss what's new and exciting in JavaScript, James' favourite up & coming libraries, and dive into the JavaScript vs. TypeScript debate.James’s YouTube channel: https://www.youtube.com/c/jamesqquickJames’s website: https://www.jamesqquick.com/Astro website builder: https://astro.build/James’ Astro course: https://astrocourse.dev/Svelte framework: https://svelte.dev/Vite build tool: https://vitejs.dev/James on Twitter: https://twitter.com/jamesqquickKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/