Software Misadventures

Well-known for his insightful and meticulous write-ups on testing distributed systems, Kyle (aka Aphyr) joins the show to chat about the origins of Jepsen, how he built a business around testing distributed systems, his writing process, favorite databases, and more.   Segments: (00:03:29) From Physics to Software Engineering (00:07:47) The origins of Jepsen (00:09:41) Turning Jepsen into a full-time venture (00:13:14) Jepsen's testing philosophy (00:16:30) The consulting journey (00:19:16) Structuring a consultancy (00:22:32) Setting boundaries (00:24:32) Pricing misadventures (00:29:17) Pros and cons of being an independent consultant (00:32:08) Managing your time when working for yourself (00:38:23) Best part of the job (00:41:13) Early writing influences (00:45:25) LLMs and AI-generated content (00:48:17) “The period where you can trust what you read is actually very recent” (00:51:33) How to become a better writer (00:54:25) Developing a formal understanding of distributed systems (00:59:30) Common faults in distributed systems (01:01:17) The complexity of testing distributed systems (01:07:32) Communicating criticism effectively (01:10:26) Advice for distributed systems engineers (01:13:46) “Anybody trying to sell you a distributed lock is selling you sawdust and lies” (01:16:31) Failure mode documentation (01:18:52) The pitfalls of containerization (01:20:17) Lightning round - favorite databases   Show Notes: “Anybody who is trying to sell you a distributed lock is trying to sell you sawdust and Lies”: https://martin.kleppmann.com/2016/02/08/how-to-do-distributed-locking.html Kyle’s excellent write-ups on testing distributed systems: https://jepsen.io/analyses Kyle’s blog: https://aphyr.com/posts Training courses that Kyle runs: https://jepsen.io/services/training   Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com Music: Vlad Gluschenko — Forest License: Creative Commons Attribution 3.0 Unported: https://creativecommons.org/licenses/by/3.0/deed.en

From creating one of the Python’s most influential libraries to co-founding Voltron Data, Wes joins the show to chat about why the book cover of the pandas book doesn’t feature a panda, open source pitfalls to avoid, the pros and cons of hiring engineers at a non-profit, and more.   Segments: (00:02:50) Guang’s complaint about the pandas book cover (00:04:38) Quarto and Open Access Publishing (00:12:00) Convincing Wall Street to Open Source (00:15:31) Publishing the first python package over Christmas  (00:18:01) Doubling Down on Building pandas (00:23:23) Personal sacrifices for the sake of impact (00:26:28) The Evolution of Open-Source (00:29:19) “Open source development started out as a very privileged activity” (00:32:40) The Consulting Trap (00:35:17) The Startup Trap (00:39:29) The Corporate User Trap (00:44:21) Avoiding the Startup Trap (00:46:54) Non-Profit vs. For-Profit (00:48:09) The Challenges of Hiring Engineers in a Non-Profit Setting (00:50:08) The Benefits of Remote Work for Open Source Development (00:52:15) Balancing Open Source and Enterprise Interests (00:57:25) New Funding Models for Open Source? (01:00:01) Getting into VC (01:06:19) The Future of Composable Data Systems   Show Notes: - online edition of pandas book: https://wesmckinney.com/book/ - the new digital publishing tool that Wes recommends: https://quarto.org/   Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com   Music: Vlad Gluschenko — Forest License: Creative Commons Attribution 3.0 Unported: https://creativecommons.org/licenses/by/3.0/deed.en

Matt Klein, co-founder of bitdrift and creator of Envoy, shares his journey from Microsoft to tackling mobile observability challenges. He discusses the pivotal early advice to 'write some proxy in Python' and the spin-out from Lyft. Klein reveals the complexities of mobile observability, the necessity of clear software licensing, and the lessons learned from startup failures. He emphasizes the importance of diverse experiences in engineering and achieving a work-life balance while being impact-driven in development.

From being a distinguished engineer at Sun Microsystems to co-founding Oxide Computer Company to build a new kind of server, Bryan joins the show to chat about being told that he’s on a suicide mission when starting Oxide, the moment he felt “I’m actually living HBO Silicon Valley”, and lessons from Sun. And much more. Chapters: (00:02:24) The Origin of Bryan's Nom-de-Guerre: "Colonel of Data Corruption" (00:04:02) What Debugging Performance Issues at Twitter in the Early Days Revealed About Silicon Valley (00:13:37) Value of Formal Education and the Experience That Everyone Should Have (00:16:02) Balancing Following One's Passion vs. Having Stability (00:21:14) What Shaped Bryan's Sense of Integrity (00:25:39) The Moments When Values Are Instilled (00:30:25) The Dark Side of Tech (00:35:12) "Economic Opportunities Attract Economic Opportunists" (00:40:35) The Origins of Oxide Computers (00:50:20) Building the A-Team (00:52:18) "Compaq Was the Most Successful Startup" (00:55:51) The Venture Capitalist's Dilemma (01:03:04) Being Told "You're on a Suicide Mission" (01:07:12) The Lifestyle of the "Lifestyle Business" (01:09:30) The Harsh Reality of Raising Venture Capital (01:13:12) The Challenges of Building Hardware (01:16:36) Why You Should Think About Not Only Gross Margin but Net Margin (01:19:14) Hardware and Software Co-Design (01:22:06) The Frustrations of Infrastructure Deployment (01:26:46) Finding the Right VCs (01:28:16) "Oh My God, I'm Actually Living HBO Silicon Valley" (01:33:12) Oxide's Principles and Lessons from Sun Microsystems (01:39:51) Sun's Unspoken Values (01:45:03) Sun's Legacy of Empowering Employees (01:48:53) Sun's Missed Opportunities (01:53:04) The Reason Why Sun Survived the Dot-Com Crash (01:56:21) "God Bless the Early Adopters" (01:57:39) A Tweet from Shopify's CEO (02:01:24) The Hard Thing About Hard Things (02:12:55) The Hardest Moment in Oxide's History   Show Notes: - Oxide’s principles: https://oxide.computer/principles - Requests for Discussion (RFDs): https://rfd.shared.oxide.computer/ - Toby’s tweet: https://x.com/tobi/status/1793798092212367669 - Bryan on twitter: https://x.com/bcantrill Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com Music: Vlad Gluschenko — Forest License: Creative Commons Attribution 3.0 Unported: https://creativecommons.org/licenses/by/3.0/deed.en

From writing the first lines of Kafka over a Christmas break as a LinkedIn engineer to running a public company as the CEO of Confluent, Jay joins the show to chat about how he and his co-founders convinced investors to take a chance on their vision, what many engineers get wrong about communication, and why engineers can make great CEOs - even when coding is not in the job description. And much more. Segments: (00:01:16) The Shaved Head Bet (00:04:07) Fundraising (00:12:16) The Role of Technical Background in VCs (00:15:48) The power of believing in the possibility of important changes (00:18:29) The Journey to starting Confluent (00:27:11) Kafka's Controversial Beginnings (00:34:30) Effective Communication in Engineering (00:44:20) The Early Days of Kafka (00:48:31) The Power of Storytelling (00:57:19) Early days of Confluent (01:03:06) Do Engineers Make Good CEOs? (01:07:59) A Typical Day in the Life of a CEO (01:12:24) The Evolution of Data Streaming Show Notes: - “The log” blog post that solidified Jay and his co-founders' conviction to found Confluent: https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying - Jay on twitter: https://x.com/jaykreps Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com Music: Vlad Gluschenko — Forest License: Creative Commons Attribution 3.0 Unported: https://creativecommons.org/licenses/by/3.0/deed.en

If you’ve worked on data problems, you probably have heard of Airflow and Superset, two powerful tools that have cemented their place in the data ecosystem. Building successful open-source software is no easy feat, and even fewer engineers have done this back to back. In part 2 of the conversation, we talk about Max’s journey in open source. Segments:    (00:03:27) “Project-Community Fit” in Open Source    (00:08:31) Fostering Relationships in Open Source    (00:10:58) Dealing with Trolls    (00:13:40) Attributes of Good Open Source Contributors    (00:20:01) How to Get Started with Contributing    (00:27:58) Origin Stories of Airflow and Superset    (00:33:27) Biggest Surprise since Founding a VC-backed Company?    (00:38:47) Picking What to Work On    (00:41:46) Advice to Engineers for Building the Next Airflow/Superset?    (00:42:35) The 2 New Open Source Projects that Max is Starting    (00:52:10) Challenges of Being a Founder    (00:57:38) Open Sourcing Ideas Show Notes: Part 1 of our conversation: https://softwaremisadventures.com/p/maxime-beauchemin-llm-ready Max on LinkedIn: https://www.linkedin.com/in/maximebeauchemin/ SQL All Stars: https://github.com/preset-io/allstars Governator: https://github.com/mistercrunch/governator Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

If you’ve worked on data problems, you probably have heard of Airflow and Superset, two powerful tools that have cemented their place in the data ecosystem. Building successful open-source software is no easy feat, and even fewer engineers have done this back to back. In Part 1 of this conversation, we chat about how to adapt to the LLM-age as engineers.   Segments: (00:01:59) The Rise and Fall of the Data Engineer (00:11:13) The Importance of Executive Skill in the Era of AI (00:13:53) Developing the first reflex to use AI (00:17:47) What are LLMs good at? (00:25:33) Text to SQL (00:28:19) Promptimize (00:32:16) Using tools LangChain (00:35:02) Writing better prompts   Show Notes: - Max on Linkedin: https://www.linkedin.com/in/maximebeauchemin/ - Rise of the Data Engineer: https://medium.com/free-code-camp/the-rise-of-the-data-engineer-91be18f1e603 - Downfall of the Data Engineer: https://maximebeauchemin.medium.com/the-downfall-of-the-data-engineer-5bfb701e5d6b - Promptimize: https://github.com/preset-io/promptimize   Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

Out of thousands of engineers at Uber, there’s only a handful of Distinguished Engineers and Joakim was one of them. In this conversation we chat about Why software engineering is a lot like a sausage factory. Considerations for leaving big tech for a startup. “How to beat the promo commitee”. How can one effectively shape engineering culture? “Mentoring two people on the same team is a waste”. Much More. Subscribe now Segments: [0:01:52] The “reverse sausage” architecture [0:07:36] How to get people on board with the new deployment system? [0:13:55] What does it mean to be a distinguished engineer? [0:17:47] Under-appreciated soft skills? [0:21:28] How to improve technical writing [0:24:16] Do all senior engineers need to write and review code every day? [0:30:19] How to search out where to contribute when your time is so constrained? [0:43:10] How to maximize your impact as a mentor [0:48:52] “How to beat the promo committee” [0:52:56] Effective means to influence engineering culture? [0:57:09] Capping the company at 150 employees [1:03:33] Why join a startup instead of moving to another big tech company? [1:11:14] What Joakim is working on now at Beyond Work   Show Notes: Joakim on leaving Uber to start Beyond Work: https://www.linkedin.com/pulse/why-i-left-uber-start-beyond-work-joakim-recht-o63of?trk=public_post_feed-article-content Read Joakim’s other excellent posts here: https://www.linkedin.com/in/recht/recent-activity/all/   Stay in touch: 👋 Make Ronak’s day by leaving us a review and let us know who we should talk to next! hello@softwaremisadventures.com

Experienced guest Kelsey Hightower discusses learning in public and writing 'Kubernetes: Up and Running'. Topics include overcoming barriers, crafting analogies, economics of book writing, cautionary tales, and leveraging technical skills outside of work.

Experienced startup advisor Kelsey Hightower shares insights on: breaking into advising without many connections, influencing without authority, passive vs. active advising, adding value as an advisor, and setting boundaries. He discusses actions engineers can take to prepare for advisory roles, managing the balance between advising and primary job, and cultivating perspectives beyond engineering.