Cloud Engineering Archives - Software Engineering Daily

Back in 2008, the range of tools that engineers could use to connect computer systems together were getting quite good. Cloud computing was democratizing access to servers. But the telephony ecosystem was still inaccessible to the average developer. If you needed your program to make a phone call and connect a user to a customer service representative, there was no easy way to do that. Twilio was started to make it easy for developers to connect to telephone systems using simple API calls. This has unlocked many important use cases: from Uber’s communication systems to the widespread adoption of 2-factor authentication. In this episode, Twilio VP of product management Pat Malatack joins the show to explain how the company builds and scales the telephony systems that underpin applications which we use every day. We also talked about how Twilio’s culture shapes how engineering proceeds at the company. Full disclosure: Twilio is a sponsor of Software Engineering Daily. The post Twilio Engineering with Pat Malatack appeared first on Software Engineering Daily.

A company like Pinterest has millions of transactional emails to send to people. The scalability challenges of sending high volumes of email mean that it makes more sense for most companies to use an email as a service product rather than building their own. Chris McFadden is the VP of engineering and cloud operations at SparkPost and he joins the show to explain the architecture of SparkPost’s email as a service product. SparkPost started as an on-premise email technology for large enterprises, and evolved into a SaaS product. In 2014, the company migrated to the cloud, which has changed its infrastructure as well as its operational model. The post Email Infrastructure with Chris McFadden appeared first on Software Engineering Daily.

Meetup is an online service that allows people to gather into groups and meet in person. Since 2002, the company has been growing and its technology stack has been changing. Today, they are in the process of migrating to the cloud, using both Amazon Web Services and Google Compute Platform. Yvette Pasqua is the CTO of Meetup and she joins the show to explain how Meetup’s technology stack works and how the teams are organized. The discussion of multiple clouds is particularly interesting–Yvette describes GCP and AWS as both having distinct, well-defined use cases at Meetup. The post Meetup Architecture with Yvette Pasqua appeared first on Software Engineering Daily.

When a useful new technology comes out, companies that are in a position to adopt that new technology can gain an edge over competitors. As our industry grows and moves faster, these kinds of changes are coming faster–some recent examples are Docker, ReactJS, and Kubernetes. Evolutionary architecture supports incremental, guided change as a first principle along multiple dimensions. A company with an evolutionary architecture is structured to evolve in response to changes inside the company (such as a decision to change the product) or outside the company (such as the emergence of Docker). Neal Ford is an architect at ThoughtWorks and one the creators of the evolutionary architecture concept. Neal is the co-chair of O’Reilly’s Software Architecture Conference | April 2-5, 2017 | New York, NY. Neal is also giving a 2-day training at the O’Reilly OSCON conference May 8-9| Austin, TX. Save 20% with discount code USRG on both. The post Evolutionary Architecture with Neal Ford appeared first on Software Engineering Daily.

Self-contained systems is an architectural approach that separates the functionality of a system into many independent systems. Each self-contained system is an autonomous web application, and is owned by one team. Communication with other self-contained systems or 3rd party systems is asynchronous where possible. As Eberhard Wolff explains in this episode, self-contained systems is not the same thing as microservices, but they are not mutually exclusive. Organizations often adopt a mix of architectural ideas, and it is worth understanding these different models so you can decide which of them to apply to your own projects. The post Self-Contained Systems with Eberhard Wolff appeared first on Software Engineering Daily.

Application performance monitoring helps an engineer understand what is going on with an application. An application on a single machine is often monitored by inserting bytecode instructions into the application after it has been interpreted. Distributed cloud applications with functionality broken up across multiple servers often use distributed tracing. Andi Grabner from Dynatrace joins today’s show to explain how monitoring software is built, and how engineers use it to solve problems. Monitoring is core to every business, whether the goal is to understand top-level business processes of to dissect and debug a specific engineering problem. And because monitoring is important at every layer of the stack, there is a plethora of monitoring tools for sale. Sponsors SparkPost provides email delivery services for apps and websites with offerings from free, self-service start-up accounts to sophisticated enterprise support and services. Try SparkPost and send 100,000 emails/month for free at http://pages.sparkpost.com/sedaily  MongoDB Atlas is the easiest way to get access to MongoDB without having to run the database yourself. To try MongoDB Atlas today, go to mongodb.com/sedaily and use code goAtlas25 to get $25 in free credit. The post Performance Monitoring with Andi Grabner appeared first on Software Engineering Daily.

Online marketplaces and social networks often have a trust and safety team. The trust and safety team helps protect the platform from scams, fraud, and malicious actors. To detect these bad actors at scale requires building a system that classifies every transaction on the platform as safe or potentially malicious. Since every social platform has to build something like this, Smyte decided to engineer trust and safety as a service. Josh Yudaken joins the show today to discuss how Smyte engineered its platform to provide machine learning models for any organization that wants to take advantage of Smyte for its trust and safety. The tools we discuss include Kubernetes, RocksDB, and Kafka, and Smyte is solving some problems that have not been solved before, so this is a great episode for anyone interested in data engineering or fraud detection–or how to use cloud services and open source tools in unique ways. Sponsors Build apps that communicate with everyone in the world. Voice & Video, Messaging, and Authentication APIs for every application. Start your Free Trial today and get an additional $10 credit account with account upgrade. Learn more at go.twilio.com/podcast MongoDB Atlas is the easiest way to get access to MongoDB without having to run the database yourself. To try MongoDB Atlas today, go to mongodb.com/sedaily and use code goAtlas25 to get $25 in free credit. The post Antifraud Architecture with Josh Yudaken appeared first on Software Engineering Daily.

For many years, software companies have been breaking up their applications into individual services for the purpose of isolation and maintainability. In the early 2000s, we called this pattern “service-oriented architecture”. Today we call it “microservices”. Why did we change that terminology? Did the services get smaller? Not exactly. Jonas Boner suggests that the movement towards cloud and the increased prevalence of mobile changes how we look at these services–so much that we needed to change the terminology necessary to even talk about them. And once the conversation has shifted to “microservices”, what steps do we need to take to implement them properly? The reactive manifesto is a collection of principles for how to build applications. When the reactive manifesto is applied to the idea of microservices, we get reactive microservices, which Jonas and I discuss in today’s episode.   Sponsors   MongoDB Atlas is the easiest way to get access to MongoDB without having to run the database yourself. To try MongoDB Atlas today, go to mongodb.com/sedaily and use code goAtlas25 to get $25 in free credit. Build apps that communicate with everyone in the world. Voice & Video, Messaging, and Authentication APIs for every application. Start your Free Trial today and get an additional $10 credit account with account upgrade. Learn more at go.twilio.com/podcast Jamf Now is a cloud-based Mobile Device Management solution for the iPads, iPhones, and Macs in your workplace. Manage your first 3 devices for free. Add more for just $2/device/month. Create your free account today at jamf.com/SEDaily! The post Reactive Microservices with Jonas Boner appeared first on Software Engineering Daily.

Some tasks are simple, but cannot be performed by a computer. Audio transcription, image recognition, survey completion–these are simple procedures that almost any human could execute, but the machine learning models have not gotten consistent enough to do them accurately. Scale is an API for human labor, created by Lucy Guo and Alexandr Wang. Similar to Amazon Mechanical Turk, Scale sends small, simple tasks to workers who can complete those tasks. Scale provides an interface that is easy for developers to use, unlike Mechanical Turk, which requires a dashboard. Similar to how Stripe allows developers to build software off of payments systems easily, Scale allows developers to build human-driven, manual tasks into their code–which unlocks a wide range of potential applications, which Lucy and Alexandr discussed with me.   Sponsors Build apps that communicate with everyone in the world. Voice & Video, Messaging, and Authentication APIs for every application. Start your Free Trial today and get an additional $10 credit account with account upgrade. Learn more at go.twilio.com/podcast Jamf Now is a cloud-based Mobile Device Management solution for the iPads, iPhones, and Macs in your workplace. Manage your first 3 devices for free. Add more for just $2/device/month. Create your free account today at jamf.com/SEDaily! MongoDB Atlas is the easiest way to get access to MongoDB without having to run the database yourself. To try MongoDB Atlas today, go to mongodb.com/sedaily and use code goAtlas25 to get $25 in free credit. The post Scale API with Lucy Guo and Alexandr Wang appeared first on Software Engineering Daily.

Caching is a fundamental concept of computer science. When data is accessed frequently, we put that data in a place where it can be accessed more quickly–we put the data in a cache. When data is accessed less often, we leave it in a place where the access time is slow or expensive. Netflix has a huge variety of data, and a huge variety of access patterns for how that data gets retrieved from storage. In today’s episode, Scott Mansfield gives an overview of Netflix’s caching architecture, including EVCache, the ephemeral, volatile cache built for Netflix’s cloud architecture. As with other episodes about Netflix architecture, this show is a deeply technical case study. The post Netflix Caching with Scott Mansfield appeared first on Software Engineering Daily.