Failure as steady state and its consequences

In this episode of Elixir Wizards, Charles Suggs and Emma Whamond are joined by Flora Petterson, a self-taught Elixir developer, speaker, and maker, to talk about what happens when software systems forget and how developers can design for recovery.
 
Flora shares her unconventional path to programming, from puppetry and creative work to discovering Elixir through Nerves and hardware projects. The conversation centers on her talk, “After the Crash,” which uses forgetting as a way to think about failure in software systems. Flora and the hosts explore process crashes, lost state, GenServer lifecycle, supervision, “let it crash,” and the difference between preventing failure and building systems that know how to recover.
 
The episode also expands from software memory to team memory. Flora discusses documentation, institutional knowledge, mentorship, storytelling, and the challenge of preserving both what a system does and why decisions were made. Along the way, she shares how creative, low-stakes projects can help developers build deeper technical understanding and how the Elixir community can continue supporting new and self-taught engineers.

Key topics discussed in this episode:

• Learning through experimentation with Nerves, hardware, and animatronics
• What it means for software systems to “forget”
• Failure, recovery, and lost state in the BEAM
• GenServer lifecycle and process recovery
• The nuance behind “let it crash”
• What supervision trees can and cannot protect
• Designing systems to recover from failure
• Deciding when state should be recovered or abandoned
• Documentation as organizational memory
• Capturing the “why” behind technical decisions
• Tests, docs, and examples as living knowledge
• Mentorship for self-taught and junior developers
• Storytelling as technical communication
• Creative projects as a path to deeper learning
• AI’s impact on new developers
• Community, curiosity, and growth in Elixir
• Join us and the broader Elixir community at ElixirConf in Chicago this September 10-11. Use promo code Elixirwizards for 10% off in-person or virtual tickets at https://elixirconf.com/

Links mentioned:

Embedded Elixir Nerve https://nerves-project.org/
Gig City Elixir Conference https://www.gigcityelixir.com/
Strange Loop Conference https://www.thestrangeloop.com/
Elixir Wizards S15E05 The State of Code Quality with Saša Jurić http://bit.ly/EWS15E05
One Art by Elizabeth Bishop https://www.poetryfoundation.org/poems/47536/one-art
"Systems that run forever self-heal and scale" by Joe Armstrong (2013) https://www.youtube.com/watch?v=cNICGEwmXLU
Conway's Law https://en.wikipedia.org/wiki/Conway%27s_law
Dunning–Kruger effect https://en.wikipedia.org/wiki/Dunning%E2%80%93Kruger_effect
Elixir Wizards S15E10 Bruce Tate http://bit.ly/EWS15E10
Designing Elixir Systems with OTP by James Gray & Bruce Tate https://pragprog.com/titles/jgotp/designing-elixir-systems-with-otp/
GenServer Behavior https://elixir.hexdocs.pm/main/GenServer.html
IEx https://iex.hexdocs.pm/IEx.html
Domain Driven Design by Eric Evans https://www.domainlanguage.com/
GDPR General Data Protection Regulation https://gdpr-info.eu/
https://blog.cleancoder.com/uncle-bob/2017/02/23/NecessaryComments.html
Living Documentation by Cyrille Matraire https://www.oreilly.com/library/view/living-documentation-continuous/9780134689418/
Exciting ExDoc talk by Flora Petterson 2025 CodeSync https://www.youtube.com/watch?v=tLr9FOARwzQ
ExDoc https://ex-doc.hexdocs.pm/
The AI Coding Crisis Blog Series https://grox.io/blog/series/the-ai-coding-crisis/
Build Animatronics with Nerves talk https://www.youtube.com/watch?v=4Ulurej4NTg
Sam Aaron Sonic Pi https://sonic-pi.net/
https://scratch.mit.edu/
https://www.scratchfoundation.org/learn
Nerves LiveBook https://github.com/nerves-livebook/
Hardware for Embedded Systems https://www.raspberrypi.com/
Goatmire Conference https://goatmire.com/