Oxide and Friends Twitter Space: September 20th, 2021Theranos, Silicon Valley, and the March Madness of Tech FraudWe’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for September 20th, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on September 20th included Land Belenky, Toasterson, Cole Frederick, and Simeon Miteff. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:John Carreyrou on Theranos “Bad Blood: Secrets and Lies in a Silicon Valley Startup” 2018 book“Bad Blood the Final Chapter” podcast as the trial proceeds (announcement), on apple, spotifyCole’s tweet linking to a ~5min video of a would-be Theranos competitor commenting on its collapse > The lone inventor is a dangerous impression to give people.Related: Brian Fitzpatrick and Ben Collins-Sussman “The Myth of the Genius Programmer” 2009 talk ~55mins[@9:47](https://youtu.be/YWdk9CKML2g?t=587) Companies that drive scientific people nuts uBeam “claims to be developing a wireless charging system to work via ultrasound. Scientists have strongly criticised the plausibility under physics of this proposal.”uBiome > To innovate, you have to balance the world as it is with the world as it isn’t.[@13:44](https://youtu.be/YWdk9CKML2g?t=824) Theranos’ fantastical vision. European attitudes around business and innovation. PCR Polymerase chain reaction invented 1983 by Kary Mullis.[@18:39](https://youtu.be/YWdk9CKML2g?t=1119) Fake it till you make it? Optative voice > The secrecy of Theranos should have been a red flag[@23:57](https://youtu.be/YWdk9CKML2g?t=1437) Whistleblower Avie Tevanian. Smoke and mirrors, giving the board the run around.[@29:05](https://youtu.be/YWdk9CKML2g?t=1745) “Everyone was relying on someone else to do their due diligence” Tech risk, venture capitalCerebras Systems wafer scale processorsEllen Pao NYT editorial “The Elizabeth Holmes Trial is a Wake-up Call for Sexism in Tech”[@35:20](https://youtu.be/YWdk9CKML2g?t=2120) Software cure-all 737 MAX failures[@40:14](https://youtu.be/YWdk9CKML2g?t=2414) Founding myths Jean-Louis Gassée2015 “Theranos Trouble: A First Person Account” blog2018 “Theranos Could Have Been Stopped” blog[@44:06](https://youtu.be/YWdk9CKML2g?t=2646) Tesla “Autopilot”, Uber self driving Anthony Levandowski > Judge Alsup: This is the biggest trade secret crime I have ever seen. > This was not small. This was massive in scale.[@48:21](https://youtu.be/YWdk9CKML2g?t=2901) March Madness of Silicon Valley Fraudsters Solyndra bankrupt 2011Tether[@59:02](https://youtu.be/YWdk9CKML2g?t=3542) Levandowski jeopardizes employee Better PlaceThe Economist ObituaryJuiceroFlip Video bought by Cisco 2009[@1:04:35](https://youtu.be/YWdk9CKML2g?t=3875) Warning signs of fraudulent companies Transparency, celebrity boardsOptaneInconsistency between board and leadership on what the coming milestones areIf we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We’d love to have you join us, as we always love to hear from new speakers!

Oxide and Friends Twitter Space: September 13th, 2021Docker, Inc., an Early EpitaphWe’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for September 13th, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on September 13th included Steve Tuck, Tom Lyon, Dan Cross, Josh Clulow, Ian, Nick Gerace, Aaron Goldman, Drew Vogel, and vint serp. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:Topic: Scott Carey’s article How Docker broke in halfMore by Carey on Docker: Docker Desktop is no longer free for enterprise usersWhat is Docker? The spark for the container revolutionAndrej Karpathy’s tweet showing InfoWorld.com spamming adsCarey talked to:Solomon Hykes (Docker cofounder with Sebastien Pahl)Ben Golub (Docker CEO 2013-2017)Craig McLuckie (Kubernetes cofounder)Nick Stinemates (early employee and former VP of Business Development)[@5:21](https://youtu.be/l9LTJdT0sZ8?t=321) Akira Kurosawa’s 1950 Rashomon ~90mins. Watch a 2min trailerBox office bomb “The Hottie and the Nottie” movie. Other stinkers: Gigli, Gotti[@9:31](https://youtu.be/l9LTJdT0sZ8?t=571) Jerry Kaplan’s 1996 book Startup: A Silicon Valley AdventureSteve’s take on commercialization > Bryan: There’s no question that they hit on something very big. > We saw a container as an operational vessel, but we failed to see > a container as a development vessel.[@14:36](https://youtu.be/l9LTJdT0sZ8?t=876) dotCloud (PaaS) struggles to find a buyer; ultimately open sources as last resort > All of a sudden a company that nobody had heard of, > was a company that everybody had heard of.They took too much money.[@17:40](https://youtu.be/l9LTJdT0sZ8?t=1060) Pitfalls in raising money and scaling sales by imitating big companiesHBO’s Silicon ValleyClip ~1min with Jan the Man, Keith, and Doug (I’m shadowing Keith) > Everybody should be spending time arm in arm with customers understanding > how is this technology going to solve a problem > which they’ll want to pay to have a solution.Tom: Was there actually a business anyways? Or was it just technology?What if developers are attracted to those things they know cannot be monetized?There was this belief that if a technology is this ubiquitous, it will be readily monetizable.[@27:26](https://youtu.be/l9LTJdT0sZ8?t=1646) Docker Swarm and Kubernetes > Hykes: We didn’t work at Google, we didn’t go to Stanford, > we didn’t have a PhD in computer science.Stinemates: (The Kubernetes team) had strong opinions about the need for a service level API and Docker technically had its own opinion about a single API from a simplicity standpoint. We couldn’t agree.DockerCon 2015: No mentioning Kubernetes! Brendan Burns’ talk “The distributed system toolkit: Container patterns for modular distributed system design” was unfortunately made private by Docker sometime in the last two years. The internet archive only has this. Burns wrote a blog post about the topics from his talk.rkt (“Rocket”), CoreOS[@36:11](https://youtu.be/l9LTJdT0sZ8?t=2171) Docker coming to marketEnterprise teams wanted supportInitial support offerings were expensive and limited (no after hours, no weekends) > Bryan: I floated to Solomon in 2014: run container management as a service.Rancher Labs, K3s (lightweight kubernetes)People care about GitHub stars (for better or worse)[@48:02](https://youtu.be/l9LTJdT0sZ8?t=2882) Monetizing open source technologiesTriton implementing the Docker APIThe support relationships are the foothold to figure out the product.[@54:36](https://youtu.be/l9LTJdT0sZ8?t=3276) Venture capital going into DockerDocker acquires TutumProduct market fitAcquisitions[@1:04:42](https://youtu.be/l9LTJdT0sZ8?t=3882) Could the outcome have been materially different?Who made money on Docker? Cloud companies? Developers?VMware acquires HeptioWho invented containers? BSD Jails, Plan9 namespaces?Tyler Tringas’ post about how small teams can create value with little outside investment, as a result of the Peace Dividend of the SaaS Wars.If we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We’d love to have you join us, as we always love to hear from new speakers!

Oxide and Friends Twitter Space: September 6th, 2021Put the OS back in OSDIWe’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for September 6th, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on September 6th included Dan Cross, Josh Clulow, Tom Lyon, Simeon Miteff, Daniel Maslowski, Matt Campbell and Moritz. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:Adam’s tweets on recording Twitter Spaces.Tweet on recovering a recording Timothy Roscoe’s Keynote Screenshots teasing his slidesConf videoComplicated relationship with academia and industry [@8:09](https://youtu.be/PVJfqjJJCkg?t=489) Adam’s MS graphics experienceBryan’s USENIX 2016 keynote ~1hr: A Wardrobe for the Emperor – Stitching Practical Bias into Systems Software Research Conferences as the publishing vector for CS research[@13:47](https://youtu.be/PVJfqjJJCkg?t=827) What a modern OS does > … accreted and not designed. > They were not designed, they congealed.[@17:10](https://youtu.be/PVJfqjJJCkg?t=1030) Rob Pike’s 2000 “Systems Software Research is Irrelevant” paperThe value of incremental improvements[@21:47](https://youtu.be/PVJfqjJJCkg?t=1307) Building on extant working components and interfaces Opaque, proprietary hardwareAMD Platform Security Processor > Artifacts of the OS implementation tend to have outsized impact > on overall system performance[@26:27](https://youtu.be/PVJfqjJJCkg?t=1587) Performance is not the only axis of a system Security, malleability, convenience, reliability[@31:12](https://youtu.be/PVJfqjJJCkg?t=1872) Specialization HarmonyOS, FuchsiaDifferent chips performing different tasksFirmware everywhereIntel OptaneIntel 8051[@37:02](https://youtu.be/PVJfqjJJCkg?t=2222) Open hardware and firmware ARM Cortex-M0 > That’s why we land at incrementalism, we ossify at some boundary. > And it’s very hard to change things on either side without moving in lockstep.Tom: The PC architecture was a great thing, but now the OS vendors have abdicated any knowledge of the hardware. Give us UEFI and we don’t care what happens beneath that.Should ARM have UEFI? (or something like it)[@45:29](https://youtu.be/PVJfqjJJCkg?t=2729) Developing hardware is still challenging, but has never been easier than today (especially low-speed) Tom’s tweet about parallels with homebrew computing in the 70’sPrecursor and Xous[@50:58](https://youtu.be/PVJfqjJJCkg?t=3058) Where will new systems development fit in with our existing (working) systems? Low-speed is an opportunity areaRISC-V for peripherals[@56:37](https://youtu.be/PVJfqjJJCkg?t=3397) Backwards compatibility seems to be more important than marginal gains: Shingled magnetic recording offered <25% density gain at the cost of compatibilityOptane: gains didn’t justify the costSmart NICs only made sense in hyperscale server fleets > Josh: If you’re going to change the programming model, you have to blow the doors off on at least one axis[@1:00:45] Moving management plane to a NIC. AWS Nitro implements this with a series of PCIe offload cards.[@1:01:22](https://youtu.be/PVJfqjJJCkg?t=3682) Abstraction boundaries not designed for the current circumstances Coordination problems between vendorsVestigial componentsAMI, AST2500Arcane boot processes and shortcuts available for cloud compute xhyve[@1:08:57](https://youtu.be/PVJfqjJJCkg?t=4137) Removing things is so hard Things change given enough timeGraham Lee’s essay on legacy and software dependencies …and in the end will be the command lineIf we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We’d love to have you join us, as we always love to hear from new speakers!

Oxide and Friends Twitter Space: August 30th, 2021A brief history of talking computersWe’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for August 30, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on August 30th included special guest Matt Campbell, as well as MattSci, TVRaman, Jessamyn West and Dan Cross. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:Brian Dear’s The Friendly Orange GlowBrodie Lockard created amazing software on PLATOControl Data Corp Homework[@2:47](https://youtu.be/b9GVJg0LRX4?t=167) Matt’s intro Deane Blazie created TotalTalk, a speaking terminal. See his 2004 interview.Apple IIe computer and the Echo II speech synthesizer card.[@4:15](https://youtu.be/b9GVJg0LRX4?t=255) The Echo ][ sound sample Wargames computer: GREETINGS PROFESSOR FALKEN. Listen > SHALL WE PLAY A GAME? > Love to. How about Global Thermonuclear War? > … > Is this a game or is it real? > WHAT'S THE DIFFERENCE? > … > What’s it doing? > It’s learning… > … > A STRANGE GAME. > THE ONLY WINNING MOVE IS > NOT TO PLAY.[@7:46](https://youtu.be/b9GVJg0LRX4?t=466) Prose 2000 sampleDECtalk audio sample[@12:14](https://youtu.be/b9GVJg0LRX4?t=734) Apple to PC Keynote Gold, Master Touch, Zoom Text[@14:53](https://youtu.be/b9GVJg0LRX4?t=893) Keynote Gold sampleTalking Moose. Watch a sample.[@17:17](https://youtu.be/b9GVJg0LRX4?t=1037) GUI screen readers outSPOKEN used QuickDrawWindow Bridge 1992[@21:58](https://youtu.be/b9GVJg0LRX4?t=1318) Meeting another sight impaired person on a MUDpwWebSpeakEmacspeak[@26:44](https://youtu.be/b9GVJg0LRX4?t=1604) Early programming experiences Apple IIGS[@28:47](https://youtu.be/b9GVJg0LRX4?t=1727) Emacspeak user base[@31:34](https://youtu.be/b9GVJg0LRX4?t=1894) Things were getting better on the Windows side.. JAWS, patch parody sampleMicrosoft Narrator[@36:12](https://youtu.be/b9GVJg0LRX4?t=2172) Linux SpeakupMixing multiple sound streams, hardware limitationsSlackwareZipSpeak by Matthew Campbell[@44:53](https://youtu.be/b9GVJg0LRX4?t=2693) Editors for the visually impaired? ed text editorEdbrowse[@49:36](https://youtu.be/b9GVJg0LRX4?t=2976) Working on accessibility (a11y) for pay FreedomBoxGNOME EsounDKDE aRtsGnopernicusOrca[@57:46](https://youtu.be/b9GVJg0LRX4?t=3466) Microsoft Active AccessibilityAT-SPICORBA, D-Bus[@1:03:11](https://youtu.be/b9GVJg0LRX4?t=3791) Handheld devices Apple VoiceOverGoogle TalkBackiPhone Screen Recognition article[@1:08:09](https://youtu.be/b9GVJg0LRX4?t=4089) What should software engineers know about accessibility? Use a mature UI framework!Microsoft UI Automation is the successor to MSAA.AccessKit by today’s speaker Matt Campbell DECtalk samples One of the most important settings a blind person will want to change in their speech synthesizer is how fast it talks. JAWS parody clipAlt text image captionsTopical recent conference presentation: - Emily Shea (2019) Voice Driven Development videoIf we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We’d love to have you join us, as we always love to hear from new speakers!

Oxide and Friends Twitter Space: August 23rd, 2021The episode formerly known as ℔We’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for August 23rd, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on August 23rd included Neal Gompa, Tom Lyon, Laura Abbott, Jeremy Tanner, Matt Campbell, Simeon Miteff and others. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:Last week’s recording on “Showstopper” with author G. Pascal Zachary, and Jessamyn West.Ashton-Tate history (there never was any Ashton, and dBASE II was the first version) dBASE IV was “slow, buggy” and didn’t get fixed in a timely mannerLast week, Pascal mentioned that CEO Ed Esber “in a fit of insanity admitted to me (a journalist) he didn’t know how to use his company’s own product!”Friday! personal information manager, and Sidekick from Borland (like Google calendar for DOS)[@3:01](https://youtu.be/-ZRv6EHaQYM?t=181) Phrasing: operating program (vs operating system) Steve Jobs 1992 MIT Sloan talk ~72mins on consultants, hiring people and leaving Apple (see mit.edu summary) > Jobs: NeXTSTEP is not an operating system, it’s an operating environmentJuly 5th recording discussing NeXT. Randall Stross book: Steve Jobs and the NeXT Big Thing (1993) > Mac OSX focused on user capabilities of the desktop environment, but they considered it one and the same with the operating system[@7:42](https://youtu.be/-ZRv6EHaQYM?t=462) Windows NT had “multiple personalities” > Adam: I was instantly transported to the 90’s. > Bryan: I could hear Smashing Pumpkins playing on the radio. Sun’s Spring OS was the ne plus ultra of this approachMach microkernel, GNU Hurd, Apple M1,Windows Subsystem for Linux WSL > Adam: Docker takes static linking to the extreme and just ships everything[@12:40](https://youtu.be/-ZRv6EHaQYM?t=760) Microkernels > Simeon: (Oxide) is working on a microkernel for Hubis, tell us about that Minix, and the Tanenbaum-Torvalds 1992 microkernel vs monolithic debateQNX Unix-like real-time OS See ACM ByteCast interview with Rashmi Mohan, Bryan tells the story ~3mins of coming to QNX after reading about it in the “Operating Systems Roundup” of Byte Magazine 1993 (see also Bryan’s blog post and remembering Dan Hildebrand)L4 microkernelThe QNX 1.44M demo diskThe GUI was called Photon. > Bryan: why would we not run this (QNX) absolutely everywhere?Oberon OS. Photon microGUI[@15:49](https://youtu.be/-ZRv6EHaQYM?t=949) Laura on writing a microcontroller operating system Cliff Biffle’s websiteMicrokernels, root of trust, embedded systemsThere is very little (or no) dynamic memory allocation in Hubris.Tock multitasking embedded OS, and Bryan’s “Tockilator: Deducing Tock execution flows from Ibex Verilator traces” video ~12minsIn Tock, dynamic program loading is central. Hubris functions as a security-minded service processor. The programs it will use are all known in advance; so dynamic loading (and the accompanying security concerns) can be left out.Fit-to-purpose OSs[@24:19](https://youtu.be/-ZRv6EHaQYM?t=1459) ROPI/RWPI (aka “Ropy Rippy”) and the growing pains of RISC-V GitHub issue ROPI/RWPI Specification (Embedded PIC)OpenTitan, ARM Cortex-M > When we set out to write Hubris, we spent a lot of time reading > and learning what’s out there.QNX vs monolithic systems. QNX was robust against module failure, so bugs in modules were tolerable. At Sun, faults in a module were system faults, so bugs were unacceptable.Memory protection. Stack growing into (and corrupting) data segment, hard to debug.Stack corruption, a hit and run.[@32:39](https://youtu.be/-ZRv6EHaQYM?t=1959) Humor: Oxide rustfmt bot is named Ozymandias Percy Bysshe Shelley’s “Ozymandias” poem > LOOK UPON MY REFORMATTING YE MIGHTY AND DESPAIR!stale bot, open source maintainers, communicating bugs and issues[@39:54](https://youtu.be/-ZRv6EHaQYM?t=2394) Fun QNX bug story QNX wrote their own POSIX utilities, they wrote their own AWKQNX developers, incl. Peter van der Veen[@43:00](https://youtu.be/-ZRv6EHaQYM?t=2580) How do you say… vi, ed > Tom: Off with their eds!sed, ps, kubectl, /etc/passwd, QNX (...

Oxide and Friends Twitter Space: August 16th, 2021The Showstopper ShowWe’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for August 16th, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on August 16th included special guests G. Pascal Zachary (see gpascalzachary.com), and Jessamyn West (see jessamyn.medium.com), as well as Dan Cross, Tom Lyon, Josh Clulow, and others. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:G. Pascal Zachary’s “Showstopper! The Breakneck Race to Create Windows NT and the Next Generation at Microsoft” bookTracy Kidder’s “The Soul of a New Machine” book[@0:46](https://youtu.be/hlQuF75L4TE?t=46) “The endless debate of NT vs Unix.” Bryan: My whole career was kind of defined by going where Windows wasn’t. I don’t know what I was expecting, but what I found was a real time capsule from software development in the 90’s.[@2:46](https://youtu.be/hlQuF75L4TE?t=166) Jessamyn: There was some familial impact (from developing DG Eclipse) that wasn’t mentioned in the book. “O, Engineers!” retrospective from wired[@6:30](https://youtu.be/hlQuF75L4TE?t=390) What was Kidder’s process? “He lived in my house!”[@8:32](https://youtu.be/hlQuF75L4TE?t=512) Zachary interviewed family members extensively. > People couldn’t leave, they were staying at the office all the time.[@14:23](https://youtu.be/hlQuF75L4TE?t=863) I do feel this is a time capsule. A time before two mega-trends hit: the Internet and open source.[@17:33](https://youtu.be/hlQuF75L4TE?t=1053) Microsoft was kind of a joke software company in the early 90’s. > Dave Cutler was a force of nature.[@19:59](https://youtu.be/hlQuF75L4TE?t=1199) No one understood why someone was good at coding. It was a mystery to everyone, why there was such a wide stratification of coders. > There were projects that never saw the light of day. Ashton-Tate, dBase > There was a sense from Cutler and Perazzoli, that leadership of the team, > that these guys at Microsoft really didn’t get how serious the process > of building this battleship was.I think the level of anguish did surprise me.[@23:59](https://youtu.be/hlQuF75L4TE?t=1439) In “Soul of the New Machine,” the machine was the star, and people served it. East Coast vs West Coast attitudes. > On the West Coast, the personal computer were supposed to help you > actualize your counter-cultural values. Ken Olsen of DEC > Computing is equivalent with IBM. There was no software industry > so long as IBM gave all the software away for free.[@26:09](https://youtu.be/hlQuF75L4TE?t=1569) Crashes. > Wozniak dreamed of owning > his own PDP > computer, which cost as much as a house. So he was aware of the robustness > of the minicomputer, and by contrast, the puny power of a personal computer. Thirtysomething > Dave Cutler was not cuddly. He was menacing, he could lose his temper. > And I tried not to get to close to him physically for that reason. > There were two looming father figures in Cutler and Gates. > And I think it created a lot of anxiety.[@29:52](https://youtu.be/hlQuF75L4TE?t=1792) The stakes for NT at Microsoft were high. Fred Brooks’ “The Mythical Man-Month” book > It was a watershed moment in the history of computing. > It was more like the last battleship, rather than the next frontier.Bryan: I didn’t realize this, that Gates was arguing against memory protection with Cutler. From our perspective, shipping an operating system without memory protection, in an era when microprocessors supported it, is malpractice.[@33:14](https://youtu.be/hlQuF75L4TE?t=1994) Cutler’s vendetta against Unix. > Conflict was at the heart of innovation at Microsoft at that time. Mitch Kapor of Lotus. > These early personal computer innovators were dismissed and sometimes > humiliated by mainstream big iron people of the 60’s and 70’s.Bill Gates’ “The Road Ahead” book doesn’t mention the internet.Zachary’s “Endless Frontier: Vannevar Bush, Engineer of the American Century” book > Computers on the West Coast were seen as extensions of your creativity, > and a tool for liberation. And for a long time that dominated the horizons.In 2005 Gates and Ballmer don’t want to do cloud computing. “Who’s gonna want to put their stuff in the cloud?” We’ve found that computing is a collective experience.[@38:28](https://youtu.be/hlQuF75L4TE?t=2308) Email and personal messaging Sun Ray thin client computerDennis Ritchie and Ken Thompson’s “The UNIX time-sharing system” paper > Unix was an experiment in collaboration.RSX-11 for the PDP-11. And VMS for the VAX. > The attitude of looking down on Unix (as undesigned, academic) is > carried forward by Microsofties today.Tom: You can forgive Cutler’s misgivings, because Unix pretty much stole the thunder out of VMS.[@42:24](https://youtu.be/hlQuF75L4TE?t=2544) Interviews for the book. Family members perspective on workplace behavior. Betty Shanahan, Society of Women Engineers. Brief Q&AEAGLE (Eclipse Appreciation and Gratitude for Lonely Evenings) award > Betty’s husband got an award for having to do his own laundry…Jessamyn’s “Women in Early Tech” blog entry about Shanahan

Oxide and Friends Twitter Space: July 26, 2021Agile + 20We’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for July 26, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on July 26 included Tom Lyon, Tom Killalea, Dan Cross, Aaron Goldman, and others. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:Al Tenhundfeld’s Agile at 20: The Failed RebellionThe Agile Manifesto[@0:55](https://youtu.be/3tp5EtPdPwY?t=55) Adam’s experiencesFrom the Agile Manifesto history > The only concern with the term agile came from Martin Fowler > (a Brit for those who don’t know him) who allowed that > most Americans didn’t know how to pronounce the word ‘agile’.[@6:25](https://youtu.be/3tp5EtPdPwY?t=385) > The problem with agile is when it became so prescriptive that it > lost a lot of its agility.[@8:06](https://youtu.be/3tp5EtPdPwY?t=486) > There’s so much that is unstructured in the way we develop software, > that we are constantly seeking people to tell us how to do it. > The answer is it’s complicated.Steve Yegge’s Good Agile, Bad Agile > So the consultants, now having lost their primary customer, were at > a bar one day, and one of them (named L. Ron Hubbard) said: > “This nickel-a-line-of-code gig is lame. You know where > the real money is at? You start your own religion.” > And that’s how both Extreme Programming and Scientology were born.[@9:15](https://youtu.be/3tp5EtPdPwY?t=555) Edward Yourdon“Decline and Fall of the American Programmer” book[@10:26](https://youtu.be/3tp5EtPdPwY?t=626) “The principles are not all wrong. Some today even feel obvious.” > There’s also a lack of specificity, which gives one lots of opportunity > for faith healers to come in.[@14:43](https://youtu.be/3tp5EtPdPwY?t=883) “Something I found surprising about Agile was how rigid it became.” Dan’s perils of personal tracking methodologySun’s engineers connecting directly with customersThe Agile Ceremonies. (an ultimate guide) Sprint Planning, Daily Stand-Up, Sprint Review, Sprint Retrospective[@20:48](https://youtu.be/3tp5EtPdPwY?t=1248) “I think we overly enshrine schedule estimation. If there are any unknowns it becomes really hard.” > I think there’s a Heisenberg principle at work with software: > you can tell what’s in a release or when it ships, but not both.[@23:25](https://youtu.be/3tp5EtPdPwY?t=1405) Tom Killalea talks to success stories he’s seen with Agile Building S3 at AWS[@28:31](https://youtu.be/3tp5EtPdPwY?t=1711) Sprint planning and backlogs Big work chunks, responding to changing priorities[@33:39](https://youtu.be/3tp5EtPdPwY?t=2019) Success or failure of an Agile team? “Do demos and retrospectives”Unknowns in software development make estimation hard[@39:11](https://youtu.be/3tp5EtPdPwY?t=2351) Dan’s experiences Personal Software Process, Team software process, Software Engineering Institute > Some people really benefit from the level of rigidity that is set out > by these processes. Prior to that, they just weren’t having > these conversations with their sales team, product owners, etc.Construction analogies, repeatability.Self-anchored suspension bridge[@46:40](https://youtu.be/3tp5EtPdPwY?t=2800) Software as both information and machine. Consultancies, repeatability, incremental results.“For each success story, there are many failures.”Manifesto as a compromise between different methodologiesSilver Bullet solutions, cure-alls. See Fred Brooks’ (1987) “No Silver Bullet” paper[@51:18](https://youtu.be/3tp5EtPdPwY?t=3078) Demos: “Working software is the primary measure of progress.” Experimentation and iterationNo true Scotsman fallacyWhat does Agile even mean anymore?“Letting people pretend to agree while actually disagreeing, but then going off and building working software anyway.”[@59:45](https://youtu.be/3tp5EtPdPwY?t=3585) Ed Yourdon and the Y2K problemMaybe there are too many Agile books already.Tom Killalea conversation with Werner Vogels AWS developmentAgile is more like a guideline than a target to hit.Consistent team composition over time“Soul of a New Machine”: trust is riskThe answer can’t be “you’re doing it wrong.”How do you know if it’s working for your team?(Did we miss anything? PRs always welcome!)If we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We’d love to have you join us, as we always love to hear from new speakers!

Oxide and Friends Twitter Space: July 5, 2021NeXT, Objective-C, and contrasting historiesWe’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for July 5, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on July 5th included Tom Lyon, Ian, bch, Theo Schlossnagle, Rick Altherr, and Nate. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:First Twitter Space, May 3rdthe lost recording (~31mins)(possible?) genesis of the idea to record spacesAdam’s process for recording spacesSomeone (Sid?) mentioned NeXT’s transparent compensation modelOxide: Compensation as a Reflection of Values[@2:28](https://youtu.be/2H9XQBdLB0Y?t=148) Randall Stross book: Steve Jobs and the NeXT Big Thing (1993)[@4:42](https://youtu.be/2H9XQBdLB0Y?t=282) The SPARCstation 1 and the Sun-4c (campus) architecture > The hardware was not competitive, but dammit they sure looked good!NeXTcube[@9:15](https://youtu.be/2H9XQBdLB0Y?t=555) It’s nuts how much time and energy they spent on the look of it. > They were building a huge factory, just about the time people were > starting to outsource everything.Sun was doing incremental things, and Steve was going for the 100 yard pass.Apple Lisa computer > NeXT refused to interoperate with anything. > They had this idea that a NeXT customer is going to buy all NeXT machines.[@13:20](https://youtu.be/2H9XQBdLB0Y?t=800) NeXT was a really proprietary company, contrasted with Sun, a really open company. > Bill Gates volunteers that he would gladly urinate on a NeXT machine.They are attempting to reinvent absolutely everything, so they need all software to be written from scratch, effectively.Jobs does this over and over again at NeXT. He does things to make NeXT look bigger than it is.[@16:23](https://youtu.be/2H9XQBdLB0Y?t=983) Jobs blows off important meeting with IBM[@18:56](https://youtu.be/2H9XQBdLB0Y?t=1136) Mathematica went whole hog on NeXT[@20:55](https://youtu.be/2H9XQBdLB0Y?t=1255) “Steve Jobs yells at your dad a lot?”Quark Software Inc, QuarkXPress[@22:22](https://youtu.be/2H9XQBdLB0Y?t=1342) Story of Jobs trying to sell NeXT machines to Brown’s CS dept > “Your product looks great, I’m just not sure your company is > going to be around for as long as we need it to be.” > Then Steve Jobs calls him an a**hole and storms out.[@23:35](https://youtu.be/2H9XQBdLB0Y?t=1415) NeXT spent very freely. Lavish offices, catering, etc > He did not take VC money. He had weird money from beginning to end. > Ross Perot thought Jobs was a total genius. Then realized that whether > he was a genius or not, he wasn’t selling any computers.The 80’s were all about fear of Japan.Ultimately they had to pivot away from hardware.[@26:38](https://youtu.be/2H9XQBdLB0Y?t=1598) In contrast to SunMetaphor Computer SystemsBryan’s tweet from July 3 > Measured by most any yardstick one could choose, Sun was one of > the most successful stories of the 1980’s for all of industrial America.John Gage[@32:43](https://youtu.be/2H9XQBdLB0Y?t=1963)the NeXTSTEP operating system, based on the Mach microkernelObjective-C HOPL paperWalter Isaacson biography on Steve JobsBe Inc, computer company. Jean-Louis GasséeStepstone (originally PPI) > Not that I’ve read a ton of HOPL papers, but I don’t think HOPL papers > spill the tea quite this much..[@39:53](https://youtu.be/2H9XQBdLB0Y?t=2393) Named parameters in programming languagesThe software crisis, Object Orientation, “Software ICs”[@44:40](https://youtu.be/2H9XQBdLB0Y?t=2680) NeXT was building real things with Objective-C, PPI wasn’t.[@45:54](https://youtu.be/2H9XQBdLB0Y?t=2754) Rick’s experience with Objective-C at AppleObjective-C, Objective-C++, and Swift[@54:08](https://youtu.be/2H9XQBdLB0Y?t=3248) Objective-C and Swift are mandated. If it were an open ecosystem, would they be significant? > There was a feeling that the hardware didn’t matter. > You shouldn’t trouble yourself with any details.[@57:46](https://youtu.be/2H9XQBdLB0Y?t=3466) Secrecy at NeXT and AppleNDAs signed per project > Secrecy is a lot of work.It was all about being able to walk on stage, and dramatically drop something that was going to be life changing.It seems like the secrecy was being used to manipulate people.[@1:03:13](https://youtu.be/2H9XQBdLB0Y?t=3793) x86 port at Apple[@1:05:34](https://youtu.be/2H9XQBdLB0Y?t=3934) Jobs tells them to make it great, because it’s currently sh*t.[@1:08:04](https://youtu.be/2H9XQBdLB0Y?t=4084) Is Objective-C being used anywhere today outside the Apple ecosystem?GNUstep, Agent-based modelingIf we got something wrong or missed something, please file a PR! Our next Twitter space will likely be on Monday at 5p Pacific Time; stay tuned to our Twitter feeds for details. We’d love to have you join us, as we always love to hear from new speakers!

Oxide and Friends Twitter Space: June 21, 2021What’s a bug? What’s a debugger?We’ve been holding a Twitter Space weekly on Mondays at 5p for about an hour. Even though it’s not (yet?) a feature of Twitter Spaces, we have been recording them all; here is the recording for our Twitter Space for June 21, 2021.In addition to Bryan Cantrill and Adam Leventhal, speakers on June 21st included Dan Cross, Sean Klein, Aram Hăvărneanu, and the mononymous Nate. (Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:Adam’s toddler (being chased by a rooster) > Don’t get me wrong, some of my best friends are three-year-olds.[@3:12](https://youtu.be/UOucW3F7nCg?t=192) Sy Brand’s tutorial Writing a DebuggerLobsters – when HN isn’t enough Bryan’s debuggersMDB Modular Debugger > Adam: I think people are using cargo-cult debugging, rather than getting to the root cause > of these things, or being satisfied until they get to the root cause. > Bryan: I think with software systems, it’s really hard to know what they’re actually doing.Procedure Linkage Table aka “the plits”“Runtime Performance Analysis of the M-to-N Scheduling Model” (pdf) 1996 undergrad thesis (Brown CS dept website)[@6:29](https://youtu.be/UOucW3F7nCg?t=389) Threadmon website and 1997 paper (a retooling of the ’96 paper) > When I built that tooling, it revealed this thing > is not doing at all what anyone thought it was doing.TNF Trace Normal Form > Part of the problem with debuggers… debuggers are historically written by compiler folks, > and not system folks. As a result, debuggers are designed to debug the problem that > compiler folks have the most familiarity with, and that’s a compiler. > Debuggers are designed for reproducible problems, way too frequently.I view in situ breakpoint debugging as one sliver of debugging that’s useful for one particular and somewhat unusual class of bugs. That’s actually not the kind of debugger I want to use most of the time.Software breakpoints[@11:59](https://youtu.be/UOucW3F7nCg?t=719) > libdis was my intern project in 2000. The idea was to take the program text, > and interpret it in some structural form, and try to infer different things about the program.Ghidra: software reverse engineering toolLaura Abbott’s Exploiting Undocumented Hardware Blocks in the LPC55S69Volatility: the memory forensics framework Adam couldn’t quite remember.[@14:59](https://youtu.be/UOucW3F7nCg?t=899) I meant this question earnestly, what is a debugger?The first bug > The term is somewhat regrettable… It implies a problem, when there may not be a problem. > It may just be I want to understand how the system is operating, independent of whether > it’s doing it badly.Wikipedia on Observability (control theory)Oxide’s embedded OS and companion debugger: Hubris and Humility[@19:01](https://youtu.be/UOucW3F7nCg?t=1141) Using DTrace to help customers understand their systems. > If you strings the DTrace binary, > you’re not gonna find any mention of raincoats.Cliff Moon on Boundary[@22:13](https://youtu.be/UOucW3F7nCg?t=1333) Cardinal rule of debuggers: Don’t kill the patient! (see also: Do No Harm) > Not killing the patient is really important, > this was always an Ur principle for us.The notion that the debugger has now become load bearing in the execution of the program, is a pretty grave responsibility.[@26:54](https://youtu.be/UOucW3F7nCg?t=1614) Post-mortem debugging > It is a tragedy of our domain that we do not debug post-mortem, routinely.Heisenbug (when the act of observing the problem, hides the problem)[@31:11](https://youtu.be/UOucW3F7nCg?t=1871) > What’s going on in the system? It’s not crashing, there’s no core dump. > But the system is behaving in a way I didn’t expect it to, and I want to know why.[@33:51](https://youtu.be/UOucW3F7nCg?t=2031) Pre-production reliability techniques > All of our pre-production work has gotten way better than it was, and I think that’s > compensation for the fact we can’t understand these systems when we deploy them.[@37:58](https://youtu.be/UOucW3F7nCg?t=2278) > The move to testing has in fact obviated some of the need for > what we consider traditional debuggers. > (Bryan audibly cringes)[@39:08](https://youtu.be/UOucW3F7nCg?t=2348) Automated and Algorithmic Debugging conference AADEBUG 2003HOPL History of Programming Languages > There was a test suite of excellence when it comes to automated program debugging. > And it was some pile of C programs with known bugs, and you would throw your new > paper at it, and it would find 84% of the bugs, and there would be a lot of > slapping each other on the back on that. Really focused on the simplest of simple bugs.[@43:15](https://youtu.be/UOucW3F7nCg?t=2595) Bryan’s Postmortem Object Type Identification paper > Who is my neighbor in memory? Because my neighbor just burned down my house basically.mdb’s ::kgrep > I need to pause you there because it’s so crazy, and I want to emphasize that > he means what he’s saying. We look for the 64 bit value, and see where we find it. > This is a game of bingo across the entire address space.We can follow the pointers and propagate types.[@48:49](https://youtu.be/UOucW3F7nCg?t=2929) printf/println debugging – everyone’s doing it > I think it’s a mistake for people to denigrate printf debugging. > If you’ve got a situation that you ca...

Oxide and Friends Twitter Space: June 7, 2021Barracuda 7200.11: broken firmware is broken software!We’ve been holding a Twitter Space weekly on Mondays at 5p (PT) for about an hour. In addition to Bryan Cantrill and Adam Leventhal, speakers included Laura Abbott, Joshua Clulow, Dan Cross, Bill Blum, Rick Altherr, Tom Lyon, and others. The recording is here.(Did we miss your name and/or get it wrong? Drop a PR!)Some of the topics we hit on, in the order that we hit them:The Seagate ST3000DM001class action[@2:01](https://youtu.be/qisoAIx8EE8?t=121) Bryan and Adam’s experience FishworksHGSTBryan is unable to forget SU0D > This thing damn near ruined our livesBroad InstituteThe Seagate Barracuda product line. 7200.10, 7200.11[@8:10](https://youtu.be/qisoAIx8EE8?t=490) Tough customers[@10:17](https://youtu.be/qisoAIx8EE8?t=617) Cargo cultism and bad interview questions What is a Good New Englander? We’re not a hugging people.[@12:35](https://youtu.be/qisoAIx8EE8?t=755) Adam and Bryan after Sean Manaea’s 2018 no-noThe Gift of the Magi, LBA[@15:11](https://youtu.be/qisoAIx8EE8?t=911) Adam torments the interns[@16:41](https://youtu.be/qisoAIx8EE8?t=1001) Bill and the HP Z620sThe Wisdom of James Mickens[@19:21](https://youtu.be/qisoAIx8EE8?t=1161) Rick’s story Fast and loose firmware source controlWestern Digital’s Sparta drive, flying too low[@25:34](https://youtu.be/qisoAIx8EE8?t=1534) Need for open source firmware (see also: Bryan explains why proprietary firmware is a problem ~3mins) Vendor gaslighting[@27:48](https://youtu.be/qisoAIx8EE8?t=1668) Tom on custom firmware Rent seekingS.M.A.R.T.ADM-3A “dumb terminal”[@32:08](https://youtu.be/qisoAIx8EE8?t=1928) Adam’s firmware horror story flashbacks HBAWhen turning it off and on again isn’t enough: unplug and replugSun’s ILOM bugSun’s embarrassing ticker symbol change[@38:10](https://youtu.be/qisoAIx8EE8?t=2290) After Sun > Stay the hell away from hardware[@39:55](https://youtu.be/qisoAIx8EE8?t=2395) Hard drive API wish list? Adam’s series on APFS > There is no bit rot here..Networking vs Storage. Intermittent, transient failure[@44:40](https://youtu.be/qisoAIx8EE8?t=2680) Firmware as differentiator Heat-assisted magnetic recording (HAMR)Microwave-assisted magentic recording (MAMR)see also: Jessie’s Life of a Data Byte surveys storage media tech through historyAmazing physics, mediocre firmware. Firmware is software[@48:23](https://youtu.be/qisoAIx8EE8?t=2903) The only firmware that didn’t give us problems.. Adam on flash: A File System All Its Own, Flash Storage Today, History of SSDs blog entry mentioning sTec and GnuteksTec aquires Gnutek LtdThe SEC’s complaint against Manouchehr Moshayedi of sTecChannel stuffingSee also: Bryan mentions sTec misconduct on the Data Center Podcast[@54:04](https://youtu.be/qisoAIx8EE8?t=3244) Sans firmware? FPGA to ASIC transition article 2011. (aside: treat yourself to this amazing vintage mouse-themed site announcing the same) > It’s when microprocessors show up that all the trouble starts.(Did we miss anything? PRs always welcome!)Our next Twitter Space will be on Monday at 5p Pacific Time. Join us; we always love to hear from new speakers!