Networking in the Cloud Fundamentals, Part 2

About Corey Quinn
Over the course of my career, I’ve worn many different hats in the tech world: systems administrator, systems engineer, director of technical operations, and director of DevOps, to name a few. Today, I’m a cloud economist at The Duckbill Group, the author of the weekly Last Week in AWS newsletter, and the host of two podcasts: Screaming in the Cloud and, you guessed it, AWS Morning Brief, which you’re about to listen to.

Transcript

An ancient haiku reads, "It's not DNS. There's no way it's DNS. It was DNS." 

Welcome to the Thursday episode of the AWS Morning Brief. What you can also think of as networking in the cloud. This episode is sponsored by ThousandEyes and their Cloud State Live Event Wednesday, November 13th from 11:00 AM until noon, Central Time. There'll be live streaming from Austin, Texas, the live reveal of their latest cloud performance benchmark where they pit AWS, Azure, GCP, IBM, and Alibaba cloud against each other from a variety of networking perspectives. Oracle Cloud is pointedly not invited. If you'd like to follow along, visit snark.cloud/cloudstatelive, that's snark.cloud/cloudstatelive, and thanks to ThousandEyes for their sponsorship of this ridiculous yet educational podcast episode.

DNS, the domain name system, it's how computers translate numbers into something humans can understand when those humans have a first language that is not math. Put more succinctly if I want to translate www.twitterforpets.com into an IP Address of 1.2.3.4, I probably want a computer able to do that because humans find it easier to remember twitterforpets.com. Originally, this was done with a far more manual process. There was a file on every computer on the internet that was kept in sync with each other. The internet was a smaller place back then, a friendlier time and jerks who are trying to monetize everything at the expense of others were no longer lurked behind every shadow, so how does this service work?

Well, let's go back to the beginning. When you look at a typical domain name, let's call it www.twitterforpets.com there's a hierarchy built in and it goes from right to left. In fact, if you pick any domain you'd like that ends .com, .net, .technology, .dev, .anything else you care about there's another dot at the end of it. That's right. You could go to www.google.com., and it works just the same way as you would expect it to. That dot represents the root and there are a number of root servers run by various organizations that no one entity controls scattered around the internet and they have an interesting job where their role is to resolve who is the authoritative responsible DNS server for the top-level domains. That's all that the root servers do.

The top-level domains, in turn, have name servers that refer out to who is responsible for any given domain within that top-level domain and so on and so forth. You can have subdomains running at your own company. You could have twitterforpets.com but all of the engineering.twitterforpets.com domains are delegated to a subname server out and so on and so forth. It can hit ludicrous lengths if you'd like. Now, once upon a time, this was relatively straightforward because there were only so many top-level domains that existed; .com, .net, .org, .edu, .mil and so on and so forth, and the governing body, ICAN, decided, "You know what's great? Money," so they wound up, in turn, going for additional top-level domains that you could grab the .technology, .blog, .underpants for all I know, no one can keep them all in their head anymore and one leaps to mind of an incredibly obnoxious purchase by google.dev.

Now, you can have anything you want .dev exist as a domain because Google has taken responsibility for owning that subdomain. Why is that obnoxious? Well, historically for the longest time on the internet, there were a finite number of top-level domains that people had to worry about. So internally, when people were building out their own environments, they would come up with something that was guaranteed never to resolve, .dev was a popular pick. You could put that to a local name server inside your firewall or you could even hard-code it on your laptop itself and it worked out super-well. Now, anyone who registers whatever domain you picked has the potential to set up a listener on their end. That is not just a theoretical concern. I worked at a company once that had their domain.com as their external domain and domain.net for their internal domain, which is reasonable, except for the part where they didn't own the .net version of their domain.

Someone else did and kept refusing offers to buy it, so periodically, we would try and log into something internal while not being on the VPN, despite thinking that we were, and type a credential into this listener that is set up and immediately have to reset our credentials. It was awful. Try not to do that. If you use a development domain, make sure you own it, it's $12, everyone will be happier with this. Now, a common interview question that people love to ask when it comes to CIS Admins, SRS, DevOps, whatever we're calling them this week, is when I punch www.google.com into my web browser and I hit enter how does it translate that into an IP address?

There're a lot of things you can hit, but by and large, the way that it works is something like this. Oh, and a caveat they love to add in because otherwise, this gets way more complicated, is every server involved has a cold cache, and we'll get to what that means in a bit, but at that point, your browser then says, "Oh, who has www.google.com?" It passes that query to the system resolver on your computer that goes through a series of different resolution techniques. It usually will check the /etc/host's file if it's on a Mac or a Linux style box, and if there isn't anything hardcoded in there, which there is it for purposes of this exercise, it queries the systems external resolver.

  This is usually provided by your ISP, but you can also use Google's public resolvers 8.8.8.8 And 8.8.4.4, Cloudflare's 1.1.1.1, OpenDNSs, which is really weird and no one can remember them off the top of their head, but there're a lot of different options. When that gets queried, it's looks at that www.google.com because it has a cold cache its first question is great, "Who owns .com?" It queries the route name server. The route name server says, "Oh, .com is handled by the .com TLD authoritative servers," and it passes that out. The route name server then returns who's authoritative for.com to the resolver. The resolver says, "Great," and then queries is the authoritative name server for .com, "Who has www.google.com?" and it returns the authoritative name servers for google.com.

Now, something strange if you were to actually try this yourself is that the answer to that question is generally ns1.google.com that sets up the opportunity for an infinite loop where oh, nsi.google.com. Ask .com, "Who has nsi.google.com?" except for the part that when it returns with that result specifically, it includes an IP address. That IP address is known as a glue record to break that circular dependency. Glue records are often one of those things that pop up in CIS Admin type interviews to prove the interviewer thinks they're smarter th...