"The Changelog" podcast features Calvin French-Owen, Co-founder and CTO of Segment, along with Alex Noonan, a Software Engineer at Segment. They discuss their transition from a monorepo to microservices and back to a monorepo, highlighting the challenges faced and the benefits of the single service approach. They explore topics such as managing numerous microservices, scaling infrastructure, performance issues, and the design and implementation process of their single service called centrifuge. The speakers emphasize the importance of finding the right fit for your infrastructure and team when considering microservices.
Read more
AI Summary
AI Chapters
Episode notes
auto_awesome
Podcast summary created with Snipd AI
Quick takeaways
Segment transitioned from a monorepo to microservices and back to a monorepo to solve performance issues and improve scalability.
Segment developed Centrifuge, a system that replaced individual queues with virtualized queues per customer per destination, leading to better data delivery insight and improved performance.
Deep dives
Darren Nix running a remote team at Indeed assessments
Darren Nix, the group manager at Indeed assessments, talks about his remote first team that operates like a startup inside Indeed. His team builds tools for job seekers to showcase their skills in a way that a resume cannot. This helps employers find and hire great candidates. The team is currently hiring Java engineers, front-end or react engineers, Ruby on Rails engineers, UX designers, and business intelligence professionals. They operate with a startup mindset, using tools like GitHub, Rails, and Redis, and focusing on frequent deployments to solve big problems and help millions of people find jobs.
Segment's journey from monorepo to microservices and back
Segment, a company that provides a single API for collecting and organizing user data, shares their journey from a monolithic architecture to microservices and back to a monorepo. The decision to change was driven by the need to solve performance issues, reduce maintenance burden, and improve scalability. They developed a system called Centrifuge, which replaced individual queues for each destination with virtualized queues per customer per destination. This change allowed them to have better insight into data delivery, improve performance, and eliminate scalability challenges. Centrifuge is currently a private system, but there are plans to potentially open source it in the future.
Lessons learned from switching to a single service architecture
The blog post by Alex Noonan, an engineer at Segment, highlights the challenges and benefits of transitioning from a microservices architecture to a single service architecture. The decision was motivated by the difficulties in maintaining and scaling the numerous microservices, as well as the desire to provide better visibility into data delivery for customers. The post discusses the process of migrating to the new architecture, the use of Centrifuge as a crucial component, and the positive impact it had on performance and product development. It concludes with the reminder to find the right fit for your team and infrastructure, and not to be afraid to make changes when necessary.
Adam and Jerod talk with two members of Segment’s engineering team: Co-founder and CTO, Calvin French-Owen, as well as Software Engineer, Alex Noonan, about their journey from monorepo to microservices back to monorepo. 100s of problem children to 1 superstar child.
Changelog++ members support our work, get closer to the metal, and make the ads disappear. Join today!
Sponsors:
Indeed – Indeed Assessments is a remote-first team that operates like a startup within Indeed — and they’re hiring. If you’re looking to join a remote-first team working on really big problems that will literally impact 100’s of Millions of people, head to indeed.jobs/changelog
Linode – Our cloud server of choice. Deploy a fast, efficient, native SSD cloud server for only $5/month. Get 4 months free using the code changelog2018. Start your server - head to linode.com/changelog
GoCD – GoCD is an on-premise open source continuous delivery server created by ThoughtWorks that lets you automate and streamline your build-test-release cycle for reliable, continuous delivery of your product.
Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com.