Cloud Engineering Archives - Software Engineering Daily

Every software company backs up critical data sources. Backing up databases is a common procedure, whether a company is in the cloud or on-prem. Backing up virtual machine instances is less common. Rubrik is a company that is known for building backup infrastructure for enterprises. Their main product is an appliance that sits on prem at an enterprise and stores snapshots of virtual machines running within the enterprise. If a virtual machine dies, Rubrik can quickly restore the VM snapshot. The appliance also backs up to the cloud. Kenny To is a founding engineer at Rubrik, and he joins the show to discuss backups and how Rubrik is engineered. Enterprises that start backing up to the cloud through Rubrik start a path towards potentially more cloud services. For enterprises that have not been able to move to the cloud yet, this can be an appealing opportunity. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: jeff@softwareengineeringdaily.com The post Backups with Kenny To appeared first on Software Engineering Daily.

Software deployment evolves over time. In the 90s, a “deployment” might have meant issuing a new edition of your software via CD-ROM. Today, a deployment is often a multi-stage process. A new software build will undergo automated unit tests and integration tests, before being deployed to users.  The deployment might only go out to a small percentage of total users initially, with that percentage going up as the deployment proves not to have bugs. Avi Cavale is the CEO of Shippable, a platform for DevOps. In this episode, we discussed deployments in the context of containers, including a discussion of what has become easier: microservices, feature flagging, and continuous delivery. He also discussed his experience building Shippable. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: jeff@softwareengineeringdaily.com The post Deployment with Avi Cavale appeared first on Software Engineering Daily.

Apache Kafka is an open-source distributed streaming platform. Kafka was originally developed at LinkedIn, and the creators of the project eventually left LinkedIn and started Confluent, a company that is building a streaming platform based on Kafka. Kafka is very popular, but is not easy to deploy and operationalize. That is why Confluent has built a Kafka-as-a-service product, so that managing Kafka is not the job of an on-call DevOps engineer. Neha Narkhede is the CTO of Confluent and she has been on the show twice before to discuss Kafka. In our last episode, we discussed event sourcing and CQRS with Kafka. In this episode, we explore more common enterprise uses for Kafka, and Neha talks about the engineering complexities of building a managed Kafka-as-a-service product. The post Kafka in the Cloud with Neha Narkhede appeared first on Software Engineering Daily.

Modern software applications are often built out of loosely coupled microservices. These services can be written in different languages, by different people, but communication between services needs to be standardized. For this reason, a service proxy is useful. A service proxy is a sidecar container that sits next to a service and facilitates communications with other services. Once every service has a sidecar proxy, that sidecar can be used as a way to communicate with a centralized control plane. The sidecar can report telemetry data to the control plane, and the control plane can be used to set policies across services, such as rules for scaling and load balancing which might vary from service to service. Istio is an open platform to connect, manage, and secure microservices. Istio is a service mesh that uses Envoy service proxies. If all of this sounds confusing–don’t worry, we’ll explain it all in today’s interview with Varun Talwar and Louis Ryan, who work on Istio at Google. Check out our new topic feeds, in iTunes or wherever you find your podcasts. We’ve sorted all 500 of our old episodes into categories like business, blockchain, cloud engineering, JavaScript, machine learning, and greatest hits. Whatever specific area of software you are curious about, we have a feed for you. Check the show notes for more details. The post Istio Service Mesh with Varun Talwar and Louis Ryan appeared first on Software Engineering Daily.

Containers make it easier for engineers to deploy software. Orchestration systems like Kubernetes make it easier to manage and scale the different containers that contain services. The popular container infrastructure powered by Kubernetes is often called “cloud native.” On Software Engineering Daily, we have been exploring cloud native software to get a complete picture of the problems in the space, and the projects that are being worked on as solutions. One area of interest: how should services communicate with each other? What should be standardized? How can you make it easy to identify problems and avoid cascading failures? One solution is the service mesh, a tool that allows services to communicate with each other more safely and effectively. William Morgan was an engineer who helped scale Twitter in the early days when the company was dealing with lots of outages. He was on the show previously to discuss scaling Twitter, and in today’s episode we go into the company that he is running, Buoyant, where he works on building a service mesh called Linkerd. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: jeff@softwareengineeringdaily.com Show Notes Scaling Twitter What’s a Service Mesh and Why Do I Need One? Buoyant is hiring: email william@buoyant.io The post Service Mesh with William Morgan appeared first on Software Engineering Daily.

Software architecture address the challenge of communicating and navigating large, complex systems to stakeholders, both technical and non-technical.  Over the years software architecture has gone in and out of fashion.  Today we discuss why software architecture is important, what it means to have software architecture, and how to properly structure teams and incorporate architecture. Today’s show is guest hosted by David Curry. David sits down with Simon Brown to discuss the importance of having a common language for software systems.  Simon is an independent consultant specializing in software architecture, he is the author of Software Architecture for Developers, and founder of Structurizr. If you are interested in hosting a show, check out softwareengineeringdaily.com/host Show Notes Coding the Architecture Software Architecture for Developers book Structurizr The post Software Architecture with Simon Brown appeared first on Software Engineering Daily.

A self-driving car needs to be able to quickly respond to changes in driving conditions. A factory needs to be able to quickly respond to changes in workplace safety. For these kinds of applications, we need processing power closer to the user of the application. If we put all of our application logic in the cloud, we will have to make a network round trip for every request. Servers in the cloud are powerful, but so are the computers at the edge–smartphones, sensors, drones, cars, and on-prem servers. Edge computing is giving us more computation outside the data center. Olivier Bloch works on Microsoft Azure IoT Edge, a set of services for edge computing. Azure IoT Edge includes on-prem versions of Microsoft Azure technologies. Tools that were previously accessible only in the cloud can be deployed and hosted on premise. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: jeff@softwareengineeringdaily.com The post IoT Edge with Olivier Bloch appeared first on Software Engineering Daily.

John Looney spent more than 10 years at Google. He started with infrastructure, and was part of the team that migrated Google File System to Colossus, the successor to GFS. Imagine migrating every piece of data on Google from one distributed file system to another. In this episode, John sheds light on the engineering culture that has made Google so successful. He has very entertaining stories about clusterops and site-reliability engineering. Google’s success in engineering is due to extremely high standards, and a culture of intellectual honesty. With the volume of data and throughput that Google responds to, 1-in-a-million events are likely to occur. There isn’t room for sloppy practices. John now works at Intercom, where he is adjusting to the modern world of Google infrastructure for everyone. This conversation made me feel quite grateful to be an engineer in a time where everything is so much cheaper, so much easier, and so much more performant than it was in the days when Google first built everything from scratch. I had a great time talking to John, and hope he comes back on the show again in the future because it felt like we were just scratching the surface of his experience. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: jeff@softwareengineeringdaily.com The post Google Early Days with John Looney appeared first on Software Engineering Daily.

Kubernetes makes it easier for engineering teams to manage their distributed systems architecture. But it’s still not simple to deploy and operate a Kubernetes cluster. Google Container Engine (GKE) is a managed control plane for Kubernetes. Just as developers can use Google App Engine to easily deploy monolithic apps against a platform as a service, we can use Google Container Engine to deploy microservices against a platform as a service. David Aronchick and Chen Goldberg work on Google Container Engine, and they join the show to explain why platform as a service container engines are useful. Google is not the only cloud provider with a container engine–Amazon ECS and Azure Container Engine also allow you to run containers in a managed fashion. Software Engineering Daily is looking for sponsors for Q3. If your company has a product or service, or if you are hiring, Software Engineering Daily reaches 23,000 developers listening daily. Send me an email: jeff@softwareengineeringdaily.com The post Container Engines with David Aronchick and Chen Goldberg appeared first on Software Engineering Daily.

DNS stands for domain name system. This is the naming system that maps the entire internet. It associates information with domain names. More specifically, DNS specifies mappings between numerical IP addresses and domain names. Most engineers know these basic facts about DNS, but they may not know how much engineering a complex company like Etsy or Zappos puts into their DNS configuration. Dynamic DNS allows for intelligent response, so that a resource is served from the most efficient place–even in the face of a DDoS attack, or just routine failure of cloud servers. Phil Stanhope is the VP of technology at Oracle Dyn and he joins the show to explain how modern DNS works and the role of a DNS provider. Full disclosure: Dyn is a sponsor of Software Engineering Daily. The post DNS with Phil Stanhope appeared first on Software Engineering Daily.