Oxide and Friends

Bryan and Adam are joined by a number of members of the Oxide networking team to talk about the networking software that drives the Oxide rack. It turns out that rack-scale networking is hard... and has enormous benefits!We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from February 27th, 2023.In addition to Bryan Cantrill and Adam Leventhal, speakers included Ryan Goodfellow, Levon Tarver, Ben Naecker, and Arjen Roodselaar.LinksIntel Tofino SeriesP4 (programming language) - Wikipediap4lang/p4c: P4_16 reference compileroxidecomputer/p4: A P4 compilerThe quote crate: Rust quasi-quotingRIFT WG - Routing In Fat Trees | IETF Community WikiHere's (much of) the live chat from the show:ahl https://github.com/oxidecomputer/oxide-and-friends/blob/master/2021_11_29.mdahl That's the Sidecar switch episodebcantrill https://p4.org/admchl What does "at line rate" mean?Riking Line rate = As fast as the packets could possibly come. 1Gbit, 10Gbit, 100Gbit, etcadmchl Do you need ASICs to hit that speed? I assume x86_64 is not going to be fast enough for these specialised operations?levon Yes, the Tofino 2 is the ASICbcantrill You need ASICsbnaecker Yes, you really can't do these kinds of operations on a general purpose CPU.rng_drizzt Yeah, you need specialized silicon here.JustinAzoff Right, also often across all ports at the same time in both direction. a 48 port 10gbps switch will have a line rate of 960gbps (10 ** 48 ** 2)duckman So the advantage is being able to offload compute to the switch?bnaecker Yes, and specifically that you can separate the data plane (operations on the packets) from the control plane (decisions about what operations to allow or make).tahnok What's TCAM?levon Ternary Content Addressable Memorybnaecker https://en.wikipedia.org/wiki/Content-addressable_memory#Ternary_CAMsryaeng Sure beats logging into a number of Cisco switches and making changes at the console.admchl This is my favourite episode in a long time, this is all really fascinating.rng_drizzt the first Sidecar episode was nearly 1.5 years ago 🤯 , right after we cut the first revlevon That episode blew my mindduckman This sounds like a big deal on the scale of ebpfduckman Or biggerbnaecker It is extremely useful for understanding the processing pipelines. As long as you only run single-packet integration tests 🙂od0 just want to go out and find things to write P4 code forJustinAzoff <@354365572554948608> yeah one way to think about that sort of thing is that xdp can be used to run little programs on a nic, where p4 is kind of like that, but running on effectively a nic with 48+ portsbcantrill https://github.com/oxidecomputer/p4SyntheticGate sidecar is the "codename" of our switch boxSyntheticGate "gimlet" is our server sledbcantrill https://github.com/oxidecomputer/propoliswmf So you have P4 and OPTE in the hypervisor at the same time?bnaecker OPTE is in the host kernel.arjenroodselaar The P4 runtime Ry described only exists in the test bed, where it high level simulates the switches. OPTE is part of the production environment.arjenroodselaar The rough difference between P4 and OPTE is that P4 works on individual packets without much concept of a session (so it can't reason about TCP streams, packet order etc, so no firewall like functionality), while OPTE aims to operate on streams of packets.JustinAzoff So you can run 100 VMs on a test system and wire them up to your virtual switch compiled by x4c?arjenroodselaar Correct.bcantrill OPTE == Oxide Packet Transformation Engineadmchl Gimlet?rng_drizzt Compute serverrng_drizzt The Sidecar switch is actually just a PCIe peripheral to a Gimlet.bnaecker The Gimlet managing the Sidecar is often called a "Scrimlet" for "Sidecar attached Gimlet"Riking and "how do i reconfigure this giant network without hosing my ability to reconfigure this giant network"ShaunO can identify with that - we seriously struggle to keep our own products inter-operating, let alone anyone else'slevon It can feel like a Sisyphean task.a172 Setup a much smaller/simpler network in parallel that is accessible from "not your network" that gets you to the management interface.levon It's a whole new world when you can look at the actual table definitions in P4rng_drizzt Owning all the layers here is immensely beneficiallevon Those DTrace probes have been very helpfulbnaecker Those probes turned out to be everywhere. They are are in: SQL queries, HTTP queries, log messages, Propolis hypervisor state, virtual storage system, networking protocol messages, the P4 emulator, and probably more that I'm forgetting about.levon For those unfamiliar with the DTrace tool, or the rationale behind leveraging DTrace over other tracing / debugging tools: https://www.cs.princeton.edu/courses/archive/fall05/cos518/papers/dtrace.pdfbcantrill https://github.com/oxidecomputer/progenitorahl some notes on rust codegen: https://github.com/ahl/codegen-templatearjenroodselaar DDM! Bring us home!a172 it astonishes me how many "cloud" type architectures are built on v4 only or v4 first.a172 IPv6 is older than Wi-Fia172 It solves real problems. PLEASE use it.nyanotech yessss fina...

Yael Grauer joined Bryan, Adam, Steve Klabnik, and the Oxide Friends to talk about her recent Consumer Reports article on memory safety and memory safe languages. How do we inform the general public? How do we persuade practitioners and companies? Thanks for joining us, Yael!In addition to Bryan Cantrill and Adam Leventhal, we were joined by special guest Yael Grauer, and Steve Klabnik.Some of the topics we hit on, in the order that we hit them (experiment in turning the show live-chat into notes):Nahum: https://www.backblaze.com/blog/the-3-2-1-backup-strategy/ if anyone wants to read up on the 3-2-1 Backup strategy. 👅Cyborus: can we get a link to the talk?Nahum: https://www.youtube.com/watch?v=Q9s2NxILBK8Nahum: https://digital-lab-wp.consumerreports.org/wp-content/uploads/2023/01/Memory-Safety-Convening-Report-.pdf via https://digital-lab-wp.consumerreports.org/2023/01/23/new-report-future-of-memory-safety/Nahum: https://en.wikipedia.org/wiki/Pegasus_(spyware)Cyborus: "can we talk" => "hey. you. have a panic attack. anyways i got a cool sandwich"AaronW: "of course we should have seatbelts" 😄MattCampbell: but then you've got the C die-hards who say that Rust itself is too complexAaronW: https://twitter.com/markrussinovich/status/1571995117233504257?s=46DanCrossNYC: People used to say the same thing about PL/I and recently the COBOL people have been saying the same thing. Nothing new under the sun.statuscalamitous: https://blog.yossarian.net/2023/02/11/The-unsafe-language-doom-principleDanCrossNYC: People who still want to treat C as a high-level assembler are saying the same stuff the PL/I people were saying when I was young.Eric Likness - carpetbomberz.com: In support of Yael, Ralph Nader wasn't/isn't an automotive engineer and he could still argue for lowering safety risks to car buyers. It's advocacy.cdaringe: As an ocaml user, i was hoping revery would take off https://github.com/revery-ui/reverystatuscalamitous: https://press.princeton.edu/books/hardcover/9780691174952/the-tyranny-of-metricsSaethlin: Wake up babe, new 0xide reading assignment droppedAaronW: Labelled like a can of pringles -- "20% more malloc() free()!"Nahum: Relevant to rules based accounting: https://www.schneier.com/blog/archives/2023/02/hacking-the-tax-code.htmldrew: Rigorous definitions of “unsafe code” just wont cut it igig: 40% less direct pointer arithmetic than the leading brand of operating systemsa172: How does principle based accounting even work? Like, how do you define if something violates the principle or not, without just turning it back into rules based?Eden: Checkboxes are meaningful for operational checklists. Aviation and medicine use them pretty heavily. Not so meaningful for systemic work like developing a new aircraft or a new surgery.Eden: So I guess a rules-based approach works for lines of code, but breaks down for project-level decisions such as which language to use.Saethlin: The S in IoT is for securitybenstoltz: ifixit repairability score for HW should have an analog for SW/FW.DanCrossNYC: That's precisely what the pl/i folks acted like 25 years ago.sam801: c++ will live on thru carbon, cppfront, and val.DanCrossNYC: Prediction: carbon is doa.Saethlin: I'll believe it once anyone uses thoseig: I think the other part is there's some really important pieces of software that everyone uses daily which use memory unsafe languages. Our web browsers, and our operating systems.AaronW: I live in a condo and I still unplug expensive electronics during a thunderstorm. Maybe it's because I had many electronics fried when I was young, and my first language was C++.Eric Likness - carpetbomberz.com: Same with answering a landline during a thunderstorm.DanCrossNYC: Had to stop training during thunderstorms in the Marines.Eden: My day job is security. 😉 I rail against compliance checklists on a regular basis because a lot of auditors insist on the checkbox rather than proper security consideration. For example, PCI-DSS requires password rotation, which everyone has known for decades leads to users picking worse passwords.alilleybrinker: https://www.usenix.org/system/files/sec22summer_alexopoulos.pdfstatuscalamitous: https://security.googleblog.com/2022/12/memory-safe-languages-in-android-13.htmla172: Google and Mozilla are making pretty good strides in migrating their browser to Rust. Still a ton of work to go, but entire systems have been moved to Rust.JamesBrock: "Lindy" https://en.wikipedia.org/wiki/Lindy_effectstatuscalamitous: https://security.googleblog.com/2021/04/rust-in-android-platform.htmlDanCrossNYC: Another issue with C/C++ in particular is that UB causes latent bugs to surface years later.alilleybrinker: In the paper linked above, the average lifetime looks to have been about 3.5 years.Saethlin: I learned Rust faster than C++alilleybrinker: Related, you might be interested in EPSS: https://www.first.org/epss/DanCrossNYC: Rust requires a bit of humility. For veteran C programmers, that can be a gut punch.srockets: “Compiler says no” is something that Haskell was proud of, but Rust is the first language I’ve seen that managed to get popular despite of italilleybrinker: Humility also requires a lot of Rust https://github.com/oxidecomputer/humilityEden: I do like the checklist item that every change must be...

Members of the Oxide team join Bryan and Adam to talk about our journey through compliance (spoiler: we passed!). Oxide and Friends: February 6th, 2023 We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from February 6th, 2023.In addition to Bryan Cantrill and Adam Leventhal, speakers on February 6th included Arjen Roodselaar, Nathanael Huffman, Robert Keith, Eric Aasen, and Josh Clulow,Some of the topics we hit on, in the order that we hit them:If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!

Oxide and Friends: January 23rd, 2023Revisiting UnikernelsWe've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from January 23rd, 2023.In addition to Bryan Cantrill and Adam Leventhal, speakers on January 23rd included Steve Klabnik, Dan Cross, and others.Some of the topics we hit on, in the order that we hit them:Bryan's 2016 blog post Unikernels are unfit for productionIf we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!Give feedback

We've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from January 16th, 2023.In addition to Bryan Cantrill and Adam Leventhal, we were joined by Nathanael Huffman, Cliff Biffle, Rick Altherr, Matt Keeter, Eric Aasen, and Dan Cross.Check out the show notes on github to browse the images.(00:00) - Intro (11:42) - Gemini (18:33) - Root of Trust (RoT) carrier (20:53) - Power (23:41) - Trimmed Power (28:11) - SPI MUX (29:38) - SPI MUX rework (33:14) - Gimletlet (41:10) - Gimletlet NIC (46:28) - DIMMlet (56:39) - Gimletlet mk2 (58:27) - Adapters (59:54) - Adapters zoom (01:01:47) - Ignition (FPGA) (01:04:40) - Gimletlet peripherals (01:06:12) - Gimletlet with management switch (1/2) (01:07:22) - Gimletlet with management switch (2/2) (01:09:21) - Kludge.2 (K.2) (01:16:23) - Donglet (01:25:49) - RoT carrier carrier (01:26:17) - Tranceiver load tester (01:29:06) - Load slammer for Tofino 2 (01:31:30) - Power (improved) (01:32:08) - Part Toaster (01:33:28) - K.2r2 Images of each proto board:@11:42 Gemini @18:33 Root of Trust (RoT) carrier @20:53 Power @23:41 Trimmed Power @28:11 SPI MUX

See github for the list of predictions (and add your own!)

Break it down with Ian BrownWe've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from December 26th, 2022.In addition to Bryan Cantrill and Adam Leventhal, our special guest was Ian Brown.

A Debugging OdysseyWe've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from December 19th, 2022.In addition to Bryan Cantrill and Adam Leventhal, our special guest was Dave Pacheco. 

Oxide and Friends: December 12th, 2022Podcasts for Podcast-LoversWe've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from December 12th, 2022.In addition to Bryan Cantrill and Adam Leventhal, speakers on MM DD included XXX, and YY. (Did we miss your name and/or get it wrong? Drop a PR!)Podcasts mentioned on the showZiade&Ford AdvisorsBrady HeywoodEconTalkLamorna Ash on Dark, Salt, ClearThe Amp Hour Electronics PodcastTools & CraftResilient web designSoftware Defined TalkKubernetes: The Documentary part 1 and part 2devtools FMScience and Futurism with Isaac ArthurAs the Ice Cream ChurnsLost TerminalLet's Make A Sci-FiInvest Like the Best: Shane BattierNew York Times: The No-Stats All-StarMark and CarrieNerdOut@Spotify: Open Source Work Is WorkThe Flop HouseThe MothIt Could Happen HerePlaydate PodcastThe Chernobyl PodcastBehind the BastardsBehind the PoliceTerrorism BadGuys We F****dHow Did This Get Made: Holy MatrimonyStartup: How Not to Pitch a BillionaireBoom / Bust HQ TriviaAcquired: TSMCThe Pitch ShowBad BetsToolsEmbedded.fmHuff DufferOther links from the audienceBSD NowHardcore HistoryEMCrit Podcasthttps://podcasts.google.com/feed/aHR0cHM6Ly9lbWNyaXQub3JnL2ZlZWQvcG9kY2FzdC8?sa=X&ved=0CAMQ4aUDahcKEwiIuNvvuPX7AhUAAAAAHQAAAAAQAQ&hl=enhttps://podcasts.apple.com/us/podcast/if-books-could-kill/id1651876897The Weirdest Thing I Learned This Weekhttps://99percentinvisible.org/about/the-show/https://intel.com/aipodcasthttps://nextcloud.com/podcast/https://gzmshows.com/shows/listing/the-big-fib/http://wandb.com/podcasthttps://feeds.captivate.fm/gradient-dissent/https://blart.libsyn.com/https://darknetdiaries.com/https://theretrohour.com/https://www.youtube.com/channel/UCPxHg4192hLDpTI2w7F9rPghttps://reasonablysound.comhttps://www.youtube.com/@NoBoilerplatehttps://www.youtube.com/@Namtaohttps://signalsandthreads.com/https://www.bbc.co.uk/programmes/w13xtvg9/episodes/downloadshttps://darknetdiaries.com/https://blog.mainframe.dev/https://podcasts.apple.com/us/podcast/revolutions/id703889772https://gimletmedia.com/shows/mystery-showhttps://mast.hpc.social/@freemin7https://feeds.megaphone.fm/uncivilhttps://open.spotify.com/episode/4cJ7NqRen0OSJ2a4Wg4uaO?si=XYNwryI0Sc6MeYRboeUcgAhttps://mango.pdf.zone/finding-former-australian-prime-minister-tony-abbotts-passport-number-on-instagramhttps://open.spotify.com/episode/5EUBBoQVZtQMkhTjfSIvzu?si=j9wcHSKGSfCiP2khHsWBughttp://www.autonocast.comhttps://risky.bizhttps://share.transistor.fm/s/7809611e

Oxide and Friends: November 28th, 2022Leaving Twitter with Tim BrayWe've been hosting a live show weekly on Mondays at 5p for about an hour, and recording them all; here is the recording from November 28th, 2022.In addition to Bryan Cantrill and Adam Leventhal, our special guest was Tim Bray. Other speakers on November 28th included Adam Jacob, Toasterson, and raggi. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:Bye, Twitter by Tim Brayjwz: PSA: Do Not Use Services That Hate The Internetjwz: Mastodon stampede "Federation" now apparently means "DDoS yourself."Tim Bray On AlgorithmsOn terrible Twitter ads: @intelnews: "Moore’s Law only stops when innovation stops.”PRs needed!If we got something wrong or missed something, please file a PR! Our next show will likely be on Monday at 5p Pacific Time on our Discord server; stay tuned to our Mastodon feeds for details, or subscribe to this calendar. We'd love to have you join us, as we always love to hear from new speakers!