Developer Voices

One of the recurring themes in the big data & data streaming worlds at the moment is developer experience. It seems like every major tool is trying to answer this question: how do we make large-scale data processing feel trivial?In some places the answer is any library you like as long as it’s Python. In other realms, a mixture of Java and SQL shows promise. But as this week’s guest—Luca Pette—would say, the Unix design metaphor has plenty to give and keep on giving.So in this episode of Developer Voices we look at TypeStream - his Kotlin project that provides a shell-like interface to data pipelines, and is gradually expanding to make integration pipelines as simple as `cat /dev/kafka | tee /dev/postgres`.--Luca on Twitter: https://twitter.com/lucapetteLuca on LinkedIn: https://www.linkedin.com/in/lucapette/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/TypeStream homepage: https://www.typestream.io/TypeStream installation guide: https://docs.typestream.io/tutorial/installationCrafting interpreters: https://craftinginterpreters.com/…by Bob Nystrom: https://twitter.com/munificentbobNuShell: https://github.com/nushell/nushell#podcast #apachekafka #bigdata

This week we're back on systems programming with Hare. A C-like language for the ages. We talk to its creator, Drew DeVault, about what he thinks we can learn from the past 50 years of programming, and how we can build that hindsight into a new language that will last for the next 100. In among all that long-term ambition we talk cover everything from error handling, typed unions and linear types, to metaprogramming and Drew's microkernel operating system. It's called Ares, and it is, of course, built in Hare.--Drew's Homepage: https://drewdevault.com/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/ A summary of Hare’s features: https://harelang.org/tutorials/introduction/Hare Community Resources: https://harelang.org/community/SXMO Mobile: https://sxmo.org/QBE Compiler Backend: https://c9x.me/compile/users.htmlAres OS Source Code: https://sr.ht/~sircmpwn/helios/OSDev Wiki: https://wiki.osdev.org/Expanded_Main_PageThe Ares System [pdf]: https://mirror.drewdevault.com/ares.pdf#programming #podcast #harelang #qbe #microkernel

Michael Drogalis, a former employee turned entrepreneur, shares his journey of launching four startups in four months. He discusses the surprising success of his first venture, revealing how user feedback shaped his focus. Balancing product development with marketing in a tight timeline poses challenges, but he navigates these pressures creatively. Michael also explores opportunities in childcare solutions and humorously reflects on the art of naming—both children and businesses—underscoring the importance of solving real problems.

Integration is probably the last, hardest, and least well thought-out part of any large software project. So anything that makes the data-streaming job easier is worth knowing about. So this week we turn our attention to Apache Flink, a flexible system for grabbing, transforming and shipping data between systems using Java, Python or good ol’ SQL. So this week Robert Metzger—Apache Flink expert and PMC member—joins us to explain what problems Flink solves and how it solves them reliably. We cover the range from simple use cases to realtime aggregations & joins to its high availability strategy.If you’re working on systems that include more than one database, then you’re definitely going to face the kinds of problems that Flink tackles.--Apache Flink: https://flink.apache.org/Robert on Twitter: https://twitter.com/rmetzger_Robert on LinkedIn: https://www.linkedin.com/in/metzgerrobert/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/–#software #programming #podcast #flink #apacheflink #dataintegration

Loris Cro from the Zig Foundation discusses Zig's ambition to replace C and its language design features. They compare Zig to Rust and Go and highlight its performance, safety, and interoperability with C. The podcast explores Zig's advantages, challenges, and funding structure. Topics covered include real-time audio processing, strings, 'defer' statement, compile-time string checking, and designing a small and consistent programming language.

Would you ever take on a rewrite of one of the largest and most popular Apache projects? And if so, what would you keep the same and what would you change?This week we’re talking to Christina Lin, who’s part of Redpanda, a company that’s rewriting parts of the Apache Kafka ecosystem in C++, with the aim of getting performance gains that aren’t feasible in Java. It seems like a huge mountain to climb, and a fascinating journey to be on, so let’s ask why and how they’ve taken on this challenge…Christina on Twitter: https://twitter.com/Christina_wmKris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/Redpanda: https://redpanda.com/Redpanda University: https://university.redpanda.com/Seestar framework: https://seastar.io/Apache Flink: https://flink.apache.org/#redpanda #kafka #apachekafka #streaming #python

Developer Gunnar Morling discusses Debezium capturing real-time data from various databases for replication and cache invalidation. They explore challenges in data synchronization, incremental snapshotting, and migrating to microservices using Kafka Connect and Apache Flink. The conversation emphasizes architecture evolution, team dynamics, and risk management strategies for seamless data movement and integration.

Ever read a bad README? We all have, and most of the time, we’ve just moved right along. A programmer that can’t communicate their ideas will find no-one uses their software. And that’s true even outside of the open-source world. The best software doesn’t win - the best software _that people can understand_ wins. So how do we get better at communicating our code? What do we talk about when we talk about software?Joining to discuss that question is a data-streaming expert and skilled communicator, Francesco Tisiot. Unusually, this episode is recorded on location, as we met up in the hallway of a recent tech conference.Francesco on Twitter: https://twitter.com/FTisiotFrancesco on LinkedIn: https://www.linkedin.com/in/francescotisiot/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/#podcast #podcasts #devrel #opensource #software #presentations

Zain Hasan, a semantic search and augmented LLMs expert, joins the podcast to discuss the challenges of teaching large language models. They explore the concept of vector databases and their role in enhancing chat bots. The episode delves into optimizing search in a fictional service, the size and storage of indexes in vector databases, and the concept of multi-modality in vector search. The discussion also includes implementing semantic search at home using Weaviate and a conversation on Weaviate, an open-source database with managed instances.

Real-time data is gradually becoming a standard requirement in systems design. Our customers are beginning to demand it, our colleagues in other departments are starting to expect it. Whether you’re letting people book a taxi, recommending their next binge-watch, or delivering business reports to management, faster data is just obviously better. Or is it?Does real-time data matter everywhere, or does it just have sweet spots in some sectors and some use-cases? Is it a cost-benefit question - is the idea great in theory, but still too hard to adopt in practice? Would everyone be streaming their data live if streaming their data live was easier? If the future of data is, “now, not later,” then what’s holding that future back?In this week’s Developer Voices we talk to Thomas Camp of Ably, and chew through the use-cases, software stacks, and education needed to speed up the way we process data. We consider everything from the front-end to the back, from user experience to business needs, and from greenfield projects to incrementally adapting existing systems.If you’re wondering what all the batch vs. streaming fuss is about, or you want to know how you can drag the industry there sooner, we have some answers. It’ll only take an hour. 😉--Thomas on LinkedIn: https://www.linkedin.com/in/thomascamp333/Ably: https://ably.com/Kris on Twitter: https://twitter.com/krisajenkinsKris on LinkedIn: https://www.linkedin.com/in/krisjenkins/