Oxide and Friends

A Debugging Odyssey

Dec 19, 2022
Ask episode
AI Snips
Chapters
Transcript
Episode notes
ANECDOTE

Initial Runtime Panic Discovery

  • Dave discovered unexpected memory corruption errors while running the test suite with CockroachDB on Lumos.
  • He encountered multiple go runtime panics that seemed unreproducible and mysterious at first.
INSIGHT

Track Failures Rigorously

  • Collecting detailed data on failure modes helps distinguish between similar but different bugs.
  • Without rigorous tracking, it’s easy to lose clarity on the nature of intermittent failures.
ANECDOTE

Echoes of Past Debugging Battles

  • The team compared this issue to a previous sporadic econ reset problem encountered on Fishworks.
  • That bug was also difficult to reproduce and required dedicated debugging efforts over months.
Get the Snipd Podcast app to discover more snips from this episode
Get the app