Cloud Engineering Archives - Software Engineering Daily

Serverless architecture is software that runs without an addressable server. Serverless is made possible by two types of technology: platform as a service providers like Auth0, and functions as a service like AWS Lambda. With both of these technologies, we can program logic that runs without being deployed to a server. Functions as a service are cheap and scalable. Write your code for a Serverless function, and the cloud provider will cheaply deploy and execute that function on some server somewhere. The difficult part is maintaining state. Since Serverless compute instances are ephemeral, you aren’t dealing with a system that will keep track of your state—it is going to disappear eventually. The ephemeral nature of Serverless code requires us to shift our thinking—but the dramatic cost and simplified scalability make it well worth the effort. Serverless functions can add complexity in exchange for lower price. Serverless “platform as a service” often lowers complexity at a slightly higher price. A Serverless database like Firebase handles database scaling and gives you a nice web interface. A Serverless machine learning platform like Google CloudML gives your models scalability and controlled deployment. A Serverless authentication service like Auth0 manages your authentication. In addition to authentication, Auth0 has built a set of tools to allow SaaS companies to extend their platforms into a sandboxed code execution environment. Bobby Johnson is an engineer at Auth0, and he joins the show to describe the toolbox that Auth0 has developed: authentication, webtasks, and extensibility–and how the world of “serverless” architecture is evolving. Full disclosure, Auth0 is a sponsor of Software Engineering Daily. The post Serverless Authentication with Bobby Johnson appeared first on Software Engineering Daily.

Augmented reality applications are slowly making their way into the world of the consumer. Pokemon Go created the magical experience of seeing Pokemon superimposed upon the real world. IKEA’s mobile app lets you see how a couch would fit into your living room, which has a significant improvement on the furniture buying process. Augmented reality applications can have even more dramatic impact on industrial enterprises. Have you ever set up a factory? You might need to build a conveyor belt. You might need to put together the parts of a giant machine that extrudes steel. You might need to fix a silicon wafer fabrication machine. It takes an expert to set up these heavy, complicated machines. ScopeAR is a company that builds augmented reality tools. One of the ScopeAR products allows users to telepresence with each other to collaborate on the construction and maintenance of heavy machinery. Imagine I am setting up my factory, and I have a complicated piece of machinery (let’s say a conveyor belt) in front of me. I have never constructed a conveyor belt before. I put on a HoloLens, and set up a VoIP call with an expert who has experience with that piece of machinery, and they point out what I need to do by superimposing 3-D arrows, text, and other instructions on my field of vision. They can share my experience and help guide me through the process. This is such a flexible tool–you can imagine applications for augmented reality assistance being useful in medicine, construction, education and other fields. Scott Montgomerie is the CEO of ScopeAR and in today’s episode, we talk about the state of AR, how the AR tools from Apple and Google compare, and how the similarity between tools used for mapping the world in AR relate to the tools used to map the world by autonomous cars. Scott was a great guest, and I hope to have him back on in the future. We have done some great shows about how to build augmented reality and virtual reality applications. To find these old episodes, you can download the Software Engineering Daily app for iOS and for Android. In other podcast players, you can only access the most recent 100 episodes. With these apps, we are building a new way to consume content about software engineering. They are open-sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to get involved with, we would love to get your help. Shout out to today’s featured contributor Edgar Pino. He is working on a real-time chat application for Software Engineering Daily, so that we can have chat rooms for people to discuss the episodes easily. Innovative work! The post Augmented Reality with Scott Montgomerie appeared first on Software Engineering Daily.

Computational load is the amount of demand that is being placed on a computer system. “Load” can take the form of memory, CPU, network bandwidth, disk space, and other finite resources. When we design systems, we need to prepare for high-load events. On a social network, people are much more active in the mornings. On an e-commerce site, Black Friday causes many more users to come online for discount shopping. Our distributed application must be able to scale in response to these spikes in traffic. Cloud computing has changed the popular software architecture patterns, and load balancing has changed along with it. With on-demand, infinite infrastructure, we don’t need to worry about ordering servers and provisioning. With infrastructure as code, it becomes simpler to manage lots of deployable units–so we can break up our monolith into microservices, and have hundreds or thousands of virtual machines or containers running. Enterprises that were started before cloud computing have large on-premise server deployments–but today, many of them also use the cloud. The cloud can be used to augment their classic on-prem deployments with cloud platform-as-a-service features. The cloud can also be used as a reliable way to scale during high load events. Today, a common architectural pattern is to have your application broken up into services. Each of those services has multiple instances. When the load on a particular service is under lots of demand, you create more instances to handle the increased load. How do you monitor the load on each service? How do you know when to spin up new instances of the service? Load analysis and load balancing across different services can be implemented by placing “agents” throughout your infrastructure. These agents gather data about services and service instances, and route that data to a centralized place. The centralized “control plane” can be used to make decisions about load-balancing and traffic routing. Ranga Rajagopalan worked on networking at Cisco for a decade before co-founding Avi Networks as CTO. Avi Networks builds modern load balancing software, and in today’s episode, Ranga describes the requirements of load balancing. We talked about the evolution of network infrastructure, the impact of the cloud, and the technical decisions that his team has made when architecting Avi Networks. Full disclosure: Avi Networks is a sponsor of Software Engineering Daily. The post Elastic Load Balancing with Ranga Rajagopalan appeared first on Software Engineering Daily.

It’s 9pm at night, and you are hungry. You order a pizza from Domino’s. You live on a street that’s dark, and so you have installed a smart lightbulb in front of your mailbox that lights up the address. When the pizza at Domino’s is ready, you want the lightbulb on your mailbox to light up so that the delivery person can read your address when they arrive in front of your house with the pizza. The Internet should make it possible to have this kind of event-driven, connected world. Anything that is connected to the Internet should be able to send signals to anything else on the Internet, so that our lives gradually become more automated. This is what IFTTT does. Users of IFTTT can easily create applets to wire different services together. You can use IFTTT to trigger an email whenever three of your friends retweet something on Twitter. You can use IFTTT to flash the lights in your house when Bitcoin hits new market highs. You can use IFTTT to order a pizza whenever Bitcoin crashes. IFTTT makes it easy to connect different services together, and a lot of work goes into the infrastructure that enables these billions of events to process correctly. Nicky Leach from IFTTT’s engineering team joins the show to describe how IFTTT allows for integrations between services that were not built to integrate–and he talks about the scheduling, data engineering, and monitoring of the company’s software stack. The post IFTTT Architecture with Nicky Leach appeared first on Software Engineering Daily.

Computer chips have physical limitations. When transistors get too small, electrons start to behave in ways that make the hardware modules less reliable. Our reliable technological progress has been enabled by Moore’s Law: the idea that the number of components we can fit on a chip doubles roughly every 12-18 months. We can’t keep shrinking the size of these components, because physics is no longer complying. Quantum computing allows us to operate on qubits rather than bits, giving us better parallelism and continued reliable technological progress. Quantum computing is still mostly an area of research rather than production systems–but it is rapidly approaching usability, and Zlatko Minev joins the show to explain how quantum computing works, and why software engineers should care. Zlatko is a PhD candidate at the Yale Quantum Information Lab. Today he describes how qubits work, which algorithms quantum computing impacts, and which parts of modern computer architecture will work on a quantum computer. We may have to throw out the Von Neumann architecture when it comes to quantum! The post Quantum Computing Introduction with Zlatko Minev appeared first on Software Engineering Daily.

How would you build a system for indexing and monitoring the entire Internet? Start by breaking the Internet up into IP address ranges. Give each of those address ranges to servers distributed around the world. On each of those servers, iterate through your list of IP addresses, sending packets to them. Depending on what sorts of packets those IP addresses respond to, and what those responses are, you can build a map of the devices on the Internet: what is running on those devices, and what they respond to. Qadium is a company that indexes and monitors devices on the Internet, to help organizations understand the devices that are within corporate networks. If you are a large corporation, Qadium can probably do a better job of figuring out your Internet footprint than you can. Matt Kraning is the CTO of Qadium, and in today’s show he describes the process by which Qadium maps the Internet. Matt used to work on data infrastructure at DARPA, and has deployed Hadoop in Afghanistan–so the infrastructure of Qadium seems relatively manageable. Our data conversations in this episode spam from talking about Storm and Hadoop to Google BigQuery, BigTable, and DataFlow. The post Internet Monitoring with Matt Kraning appeared first on Software Engineering Daily.

Scala is a functional and object oriented programming language built on the JVM. Scala Native takes this language, loved by many, and brings it to bare metal. Scala Native is an optimizing ahead-of-time compiler and lightweight managed runtime designed specifically for Scala. Denys Shabalin is a Research Assistant at the EPFL and the primary creator of Scala Native. In this episode, Adam Bell interviews Denys about the motivations behind the Scala Native project, how it was implemented and future directions. He also briefly touches on how Scala Native made cold compilation times of Scala code twice as fast. If you are interested in functional programming, compiler design, or want to learn some interesting tidbits about garbage collector design and trade offs you will like this episode. In past shows, we have covered many newer programming languages, and new twists on old programming languages. We’ve talked about ScalaJS, Swift on the Server, and Rust at Mozilla. Download the Software Engineering Daily app for iOS or Android to hear all of our old episodes. They are easily organized by category, and as you listen, the SE Daily app gets smarter, and recommends you content based on the episodes you are hearing. If you don’t like this episode, you can easily find something more interesting by using the recommendation system. The mobile apps are open sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to hack on, we would love to get your help! We are building a new way to consume software engineering content. We have the Android app, the iOS app, a recommendation system, and a web frontend–and more projects are coming soon. If you have ideas for how software engineering media content should be consumed, or if you are interested in contributing code, check out github.com/softwareengineeringdaily, or join our Slack channel (there’s a link on our website)–or send me an email: jeff@softwareengineeringdaily.com Show Notes Scala Native Scala Goes Native Talk Bootstrapping the Web with Scala Native The post Scala Native with Denys Shabalin appeared first on Software Engineering Daily.

Tinder is a rapidly growing social network for meeting people and dating. In the past few years, Tinder’s userbase has grown rapidly, and the engineering team has scaled to meet the demands of increased popularity. On Tinder, you are presented with a queue of suggested people that you might match with, and you swipe left or right to indicate that you like or dislike them. Creating that queue of suggestions is a complex engineering problem. Many factors go into the suggestions that Tinder gives you: geotargeting, food preferences, your favorite band, your photos, and the people you have swiped on in the past. Bryan Li is an engineering manager at Tinder, and he joins the show to describe the interaction between the mobile client, backend servers, and the offline analytics and machine learning. We also talk about managing different teams and how to reorganize smoothly as a company grows. If you like this episode, we have done other shows about scaling companies like Uber, New Relic, and Giphy. Download the Software Engineering Daily app for iOS and Android to hear all of our old episodes, and easily discover new topics that might interest you. If you don’t like this episode, you can easily find something more interesting by looking at the recommendation engine in the app. The mobile apps are open sourced at github.com/softwareengineeringdaily. If you are looking for an open source project to hack on, we would love to get your help! The Software Engineering Daily open source community is building a new way to consume software engineering content. We have the Android app, the iOS app, a recommendation system, and a web frontend. If you are interested in contributing, check out github.com/softwareengineeringdaily–or send me an email: jeff@softwareengineeringdaily.com The post Tinder Engineering Management with Bryan Li appeared first on Software Engineering Daily.

Playing a video on the Internet seems simple. You press play, the video gets delivered, and boom–you are watching Game of Thrones, right? It’s a bit more complicated. Unless you have built an application that involves video, you probably have not dealt with the world of codecs, bitrates, and streaming. Depending on the bandwidth between the user and the server, you might want to use different compression rates. Think about all of the different use cases–different connection speeds, device types, operating systems, video players, cloud providers. As a developer, you just want videos in your application to play quickly and reliably. But it takes a lot of engineering, monitoring, and re-engineering to get it right. Matt McClure and Jon Dahl are the founders of Mux, a company that makes video infrastructure technologies. Previously they built Zencoder, a product for encoding and delivering video. This episode was a fascinating discussion of why building video products for the modern Internet is still so hard. Download the Software Engineering Daily app for iOS to hear all of our old episodes, and easily discover new topics that might interest you. You can upvote the episodes you like and get recommendations based on your listening history. With 600 episodes, it is hard to find the episodes that appeal to you, and we hope the app helps with that. The iOS app is the first project to come out of the Software Engineering Daily Open Source Project. There are more projects on the way, and we are looking for contributors–if you want to help build a better SE Daily experience, check out github.com/softwareengineeringdaily. We are working on an Android app, the iOS app, a recommendation system, and a web frontend. Help us build a new way to consume software engineering content at github.com/softwareengineeringdaily. The post Video Infrastructure with Matt McClure and Jon Dahl appeared first on Software Engineering Daily.

Tinder is a popular dating app where each user swipes through a sequence of other users in order to find a match. Swiping left means you are not interested. Swiping right means you would like to connect with the person. The simple premise of Tinder has led to massive growth, and the app is now also used to discover new friends and create casual meetings. Every social network knows–if you are not growing, then you are dying. Growth is so important to Tinder, they have a large engineering organization devoted to five facets of growth: new users, activation, retention, dropoff, and anti-spam. These five segments cover the entire Tinder user lifecycle, and there is a sub-team in charge of each of the five areas. No matter what kind of Tinder user you are, there are growth engineers focused on your experience. Alex Ross is the director of engineering for the growth team at Tinder. His job requires a mix of data science, data engineering, psychology, and setting proper KPIs (key performance indicators). Each subteam has KPIs that determine how well they are doing with growth–and if the wrong KPI is set, it can create bad incentives. For example, a growth team that is focused only on getting users to spend more time engaging with Tinder would have an incentive to create so-called “dark patterns” that trigger addiction. If you like this episode, we have done many other shows about data science and data engineering. Download the Software Engineering Daily app for iOS to hear all of our old episodes, and easily discover new topics that might interest you. You can upvote the episodes you like and get recommendations based on your listening history. With 600 episodes, it is hard to find the episodes that appeal to you, and we hope the app helps with that. The post Tinder Growth Engineering with Alex Ross appeared first on Software Engineering Daily.