Networking in the Cloud Fundamentals, Part 3

About Corey Quinn
Over the course of my career, I’ve worn many different hats in the tech world: systems administrator, systems engineer, director of technical operations, and director of DevOps, to name a few. Today, I’m a cloud economist at The Duckbill Group, the author of the weekly Last Week in AWS newsletter, and the host of two podcasts: Screaming in the Cloud and, you guessed it, AWS Morning Brief, which you’re about to listen to.

Transcript

This episode of Networking in the Cloud is sponsored by ThousandEyes. Their 2019 Cloud Performance Benchmark Report is now live as of yesterday. Find out which Clouds do what well, AWS, Azure, GCP, Alibaba, and IBM Cloud all have their networking capabilities raced against each other. Oracle was not invited, because we are talking about actual Cloud providers here, not law firms. Get your copy of the report today at Snark.Cloud/realclouds. That's Snark.Cloud/realclouds. That's completely free. Download it, let me know what you think. I'll be cribbing from that in future weeks. Now, for the third week of our AWS Morning Brief Screaming in the Network, or whatever we're calling it, mini-series on how computers talk to one another. Let's talk about the larger internet.

Specifically, we begin with BGP, or Border Gateway Protocol. This matters, because it's how different networks talk to one another. If you have a whole bunch of different computer networks gathered into a super network, or internet as some people like to call it, how do those networks know where each one lives? Now, from a home user perspective, or even in some enterprises, that seems like sort of a silly question, because it is. You have a network that lives on your end of things. You plug a single cable in, and every other network lives through that cable. When you're talking about large disparate networks though, how do they find each other? More to the point, because of how the internet was built, it's designed so that any single failure of another network can now be routed around. There are multiple paths to get to different places. Some biased for cost, some biased for performance, some biased for consistency. And all of those decisions have to be made globally. BGP is the lingua franca of how those networks talk to one another. BGP is also a hot mess.

It's the routing protocol that runs the internet, and it's comprised of different networks in this parlance, autonomous systems, or AS's, and it was originally designed for a time before jerks ruled the internet, and that's jerks in terms of people causing grief for others, as well as shady corporate interests that are publicly traded on NASDAQ. There's no authentication tied to BGP. Effectively, it is trusted to contain correct data. There is no real signing or authentication that someone who announces something through BGP is authorized to do it, and it's sort of amazing the whole thing works in the first place, but what happens is, is when a large network with other networks behind it winds up doing an announcement, it says, oh, I have routes to these following networks. And it passes them on to its peers. They in turn pass those announcements on, oh, behind me. Then this way two hops is this other series of networks, and so on and so forth.

Now this can cause hilariously bad problems that occasionally make the front page of the newspaper when a bad announcement gets out. A few years ago there was an announcement from an ISP that said, oh, all of YouTube lives behind us. That announcement should never have gone out, and their upstream ISP should have quashed it, and they didn't. So suddenly a good swath of the internet was trying to reach YouTube through a relatively small link. As you can imagine, TCP terminated on the floor. Not every link can handle exabytes of traffic. Who knew? That gets us to another interesting point. How do these large networks communicate with each other? You have this idea of one network talks to another network. Does money change hands? Well, in some cases, no. If traffic volumes are roughly equal and desirable on both sides, we'll have our networks talk to one another, and no money changes hands. This is commonly known as peering.

  At that point, everything is mostly grand, because as traffic continues to climb, you increase the links. Both parties generally wind up paying to operate infrastructure on their own side and in between, and traffic continues to grow. Other times it doesn't work that way where you have one network with a lot of traffic, and another network that doesn't really have much of any, and people want to go from one end to the other. Very often this is known as a transit agreement, and money changes hands from usually the smaller network to the bigger network, but occasionally the other direction depending on the specifics of the business model, and at that point, every byte passing through is metered and generally charged for. Usually this is handled by large ISPs and carriers and businesses behind the scenes, but occasionally it spills out into public view. Comcast and Netflix, for example, have been having a fantastic public spat from time to time, and this manifests itself when there's congestion and you're on Comcast.

If so, I'm sorry for you, and your Netflix stream starts degrading into lower picture quality. Occasionally it's skips or whatnot, and strangely whenever Comcast and Netflix come to an agreement, of course under undisclosed terms, magically these problems go away almost instantly. Originally this sort of thing was frowned upon. The FCC got heavily involved, but with the demise in the United States of network neutrality, suddenly it's okay to start preferring some traffic over others through a legalistic framework, and this has led to a whole bunch of either malfeasant behavior or normal behavior that people believe is malfeasant. And that doesn't leave anyone in a terrifically good place. I'm not here to talk about politics, but it does wind up leading to an interesting place, because there's an existential problem to the business model for an awful lot of ISPs out there. Because generally speaking, when you wind up plugging into your upstream provider, maybe it's Comcast, maybe it's AT&T, maybe it doesn't matter, but you're generally trying to use them as a dumb pipe to the internet.

The problem is, is they don't want to be a dumb pipe. There's a finite number of dollars that everyone is going to pay for access to the internet, and that is a naturally self-limiting business model, so they're trying to add value with services that don't really tend to add much value at all. My wireless carrier for example, wants to sell me free storage, and an email address, and a bunch of other things that I just don't care about, because I already have an email solution that works out super well for me. My Cloud storage that I care about is either Dropbox, something in AWS or other nonsense. I don't need to have Verizon's Cloud storage, but they keep trying to find alternative business models. Some of these ways are useful and beneficial to everyone, and others are well to be honest, less so.

Comcast for example, isn't going to build you a search engine that is going to rival Google, which is kind of weird on some level because if you take a look from a customer service perspective, C...