Cloud Engineering Archives - Software Engineering Daily

On January 31st 2017, GitLab experienced a major outage of their online repository hosting service. The primary database server experienced data loss due to a combination of malicious spam attacks and engineering mistakes that occurred while trying to respond to those spam attacks. GitLab responded to the event transparently. The company put up a postmortem describing the event in detail. In subsequent posts, GitLab expressed sympathy for the employee who made engineering mistakes that led to the deletion of data. The employee was not judged or disciplined for an understandable error. The response from the developer community was very positive. Engineers know that building cloud services is hard. Engineering is as much about avoiding errors as it is about appropriately responding to the inevitable mistakes. GitLab is a developer platform that combines repository hosting with several other features–issue tracking, code review, and CD. Today’s guest is Pablo Carranza, who works on infrastructure at GitLab. In this episode, he walks us through GitLab’s product, the engineering stack, and a postmortem of the outage. We also discuss working at Amazon, and the importance of postmortems, which I first encountered at Amazon. The post GitLab with Pablo Carranza appeared first on Software Engineering Daily.

Healthcare is a complex business. Oscar is a company that wanted to build a new insurance provider–but realized that healthcare is so interconnected that in order to build a new insurance provider, realized it actually needed to build an entire healthcare business too, complete with patient management and facilities. Since Oscar is a modern technology company, the focus on customer service, engineering, and data management offers an optimistic view into what healthcare might look like in the near future. Every time a patient interacts with the healthcare system, their insurance provider collects data on that interaction. Isaac Councill helped architect the infrastructure at Oscar that manages and analyzes the patient data. In this show, we talk about the healthcare system, data engineering, and Apache Mesos, which Oscar uses to manage its applications. The post Healthcare Engineering with Isaac Councill appeared first on Software Engineering Daily.

Many companies are transitioning from a monolith to microservices architecture. Tools for cloud computing, containerization, and continuous delivery are making this easier. But there are still technological and organizational challenges that a company will encounter while making this transition. Cassandra Shum is an engineer with ThoughtWorks. She has worked with major financial institutions and other large companies to architect their migrations from monolith to microservices. Also, she regularly puts on workshops to engineers who are seeking to make this migration at the company they work at. In this episode, she describes some of her experiences and recommendations around transitioning from a monolith to microservices. The post Microservices Transition with Cassandra Shum appeared first on Software Engineering Daily.

Firebase is a backend-as-a-service. The key efficiency of a backend-as-a-service is that it enables developers to go from having a 3-tier architecture (client, server, database) to a 2-tier architecture (client, backend-as-a-service). The team who started Firebase built it as a pivot. They had started a social network, and then they realized there wasn’t a good backend for chat tools. And so they started a chat-as-a-service tool, for people who wanted to include chat in their applications. And that led them to the fundamental realization that chat is actually representative of a broader category of real-time synchronization problems. Firebase was eventually acquired by Google. Doug Stevenson is a senior developer advocate with Google and the host of Meet Firebase, a YouTube talk show about Firebase. It was a pleasure to sit down for a conversation with him, especially because I recently started using Firebase in my own application as a backend for real-time chat.   The post Firebase with Doug Stevenson appeared first on Software Engineering Daily.

Spring Framework is an application framework for Java and JVM languages. Spring was originally built around dependency injection, but grew to become an entire ecosystem of tools and plugins for Java developers. Spring was originally released 15 years ago, and since then a lot has changed around application development. For example, many engineers deploy applications to the cloud in microservices architectures. The expectations around frameworks has also changed, with the rise of Django, Ruby on Rails, and NodeJS. Spring Boot takes an opinionated view of building production-ready Spring applications. By taking an opinionated view, Spring Boot gets engineers up and running faster than the traditional Spring framework. Josh Long is a Spring Developer Advocate at Pivotal and he joins the show to discuss Spring Boot and the history of the Spring Framework. Software Engineering Daily is having our third Meetup, Wednesday May 3rd at Galvanize in San Francisco. The theme of this Meetup is Fraud and Risk in Software. We will have great food, engaging speakers, and a friendly, intellectual atmosphere. To find out more, go to softwareengineeringdaily.com/meetup. We would love to get your feedback on Software Engineering Daily. Please fill out the listener survey, available on softwareengineeringdaily.com/survey. The post Spring Boot with Josh Long appeared first on Software Engineering Daily.

The word “microservices” started getting used after a series of events–companies were moving to cloud virtual machines. Those VMs got broken up into containers, and the containers can fit to the size of the service. Services that are more narrowly defined take up smaller containers, and can be packed more densely into the virtual machines–hence the term “microservices.” As this change to software architecture has occurred, the DevOps movement has encouraged organizations to have better relationships between development and operations. Continuous deployment leads to fewer painful outages. Improved monitoring tools make it easier for developers to take on some of the pain that was previously centralized in operations. Several months ago, I attended the Microservices Practitioner Summit, which brings together engineers who are working with microservices at their companies. The conference was organized by Austin Gunter and Richard Li of Datawire. In this episode, they joined me for a conversation about microservices. Software Engineering Daily is having our third Meetup, Wednesday May 3rd at Galvanize in San Francisco. The theme of this Meetup is Fraud and Risk in Software. We will have great food, engaging speakers, and a friendly, intellectual atmosphere. To find out more, go to softwareengineeringdaily.com/meetup. The post Microservices Practitioners with Austin Gunter and Richard Li appeared first on Software Engineering Daily.

Search is a common building block for applications. Whether we are searching Wikipedia or our log files, the behavior is similar: a query is entered and the most relevant documents are returned. The core data structure for search is an inverted index. Elasticsearch is a scalable, resilient search tool that shards and replicates a search index. Philipp Krenn from Elastic joins the show today to discuss how search works and how Elasticsearch scales. We use Wikipedia as a running example for how a query is processed and how documents are stored. If you’ve ever wondered how search works–or if your company uses Elasticsearch and you want to know more about it–this is a great episode for you. The post Elasticsearch with Philipp Krenn appeared first on Software Engineering Daily.

There are various standards at play when creating and consuming Application Program Interfaces (APIs).  These standards, though, are mostly technical and mostly lower-level than the content of the API. Andy Beier has experienced the broad range of API quality in his role with Domo in creating integrations with other businesses.  He has made standardization of good practices in creating APIs his mission, with an emphasis on making the right information easily accessible without having to download more than necessary.  He has traveled to meet with leaders in the field to promote standards and to make APIs easier to create and to consume. In this episode, Andy joins Dave Rael for a conversation about API design standards, what makes for a good API, and steps in moving the broader technical community toward more useful and secure APIs. The post API Design Standards with Andy Beier appeared first on Software Engineering Daily.

Servers in a data center fail. Sometimes entire data centers have a power outage. Bugs in an application make it into production. Human operators make mistakes and cause data to be deleted. Failure is unavoidable. We make backups and replicate our servers so that when a failure occurs, we can quickly respond to it without making the user feel much pain. But how can we test that our response will work before an actual catastrophe occurs? Kolton Andrus is CEO of Gremlin, a company that works on failure injection as a service. Gremlin is based on ideas around planned failure that Kolton learned from his years at Amazon and Netflix. We ended up talking as much about the culture of Netflix and Amazon as we did about how and why to build failure injection. It’s always nice to share war stories with other people who have worked at Amazon because the culture is so distinct. If you want to know more about Amazon’s culture, check out the episode tomorrow with Brad Stone, author of The Everything Store. The post Failure Injection with Kolton Andrus appeared first on Software Engineering Daily.

Designers and software engineers need to communicate with each other. From Apple to Slack to Uber, the emphasis on visual design within a product is rising in importance. Much like development and operations siloes have been bridged with the DevOps movement, design and engineering teams are working more closely together to align the vision of the designers with the realities of code. InVision is a product for prototyping designs and product workflows. I categorize InVision with high-level productivity tools like Trello, Asana, and github. When teams start using InVision, it often becomes an integral part of the entire product workflow. A designer will mock up their product vision on InVision and share it with the rest of the team for criticism and commentary. Since the product is the perfect bridge between engineers and designers, InVision the company is expanding as a platform. Bjorn Freeman-Benson is the CTO at InVision and his task is to scale the 100% remote engineering team while developing new product features. Bjorn describes himself as a software psychologist. We talked about the psychology of managing engineers and the technical challenges of building a large, popular web app for technology products. The post Software Psychology with Bjorn Freeman Benson appeared first on Software Engineering Daily.