Having Breadcrumbs for Debugging

Joël is joined by Amanda Beiner, a Senior Software Engineer at GitHub, who is known for her legendary well-organized notes. They talk about various types of notes: debugging, todos, mental stack, Zetelkasten/evergreen notes, notetaking apps and systems, and visual note-taking and diagramming too!

---

• @amandabeiner
• Mermaid.live
• Monodraw
• Zettlekasten
• Evergreen Notes
• Notion
• Obsidian

Transcript:

JOËL: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Joël Quenneville. And today, I'm joined by Amanda Beiner, a Senior Software Engineer at GitHub.

AMANDA: Hey, Joël. Great to see you.

JOËL: And together, we're here to share a little bit of what we've learned along the way. So, Amanda, what is new in your world?

AMANDA: Well, one thing I'm really excited about is that my team at GitHub is experimenting with how we're going to incorporate learning and sharing what we've learned with each other in new ways, and I'm really excited to see where people take that.

So, one of the things that we're thinking of is that we all get really busy, and we all have exciting projects that we're working on in the day-to-day, and sometimes it can be really hard to pull yourself away from them to do some learning that would be something that will probably help you in the long run. But every time we do do projects like that, people are really excited about it, and people like to collaborate. So we're just trying to figure out how we can make that a more regular thing because it's great for our whole team.

JOËL: I love that. Do you have a project or something that you've been getting into recently to learn?

AMANDA: Yeah. One of the things that I have been working on is that this is the first backend-focused role that I've had in my entire career. So I feel like I just kind of keep pulling back layers on how different forms of magic work. And I'm just trying to get closer to the metal of what is powering our databases. And that's something that I've been really excited to learn some more about.

JOËL: So it's digging into a lot of, like, Postgres and just general database theory.

AMANDA: Yeah. So for me, I've spent a lot of time at the Active Record layer as I have been settling into my role and figuring out what our domain models are that we care about. And I'm trying to get a little bit more into the questions of why did these tables end up looking the way that they do? Why are they normalized or denormalized where they are? And trying to get a better idea of the theory behind those decisions.

JOËL: And this is a new team that you've joined.

AMANDA: This is an existing team that I've joined a year ago now.

JOËL: So it sounds like you're dealing with a somewhat unfamiliar codebase. You're looking at a bunch of existing models and database tables. That can be a lot to process and understand when you first join a team. Do you have an approach that you like to use when you're looking at unknown code for the first time?

AMANDA: Yeah. I usually like to dive right in as much as I can, even if it's with a very small bug fix or something like that, something that allows me to just get my hands dirty from the beginning and poke around what models I'm dealing with, and maybe some of the adjacent ones that I don't need to know about now but might want to come back to later.

JOËL: One thing that I find is really helpful for me are diagramming and note-taking. So if it's something like a database table or ActiveRecord models that I'm not familiar with, if it's more than maybe two or three, which is probably the most I can keep in my head, I have to start drawing some kind of like an entity-relationship diagram or maybe even just a bulleted list somewhere where it's like here are the things and how they connect to each other. Otherwise, I’m like, I don't know, I don't have enough RAM in my brain for that.

AMANDA: That sounds like a really helpful approach. How do you approach creating these diagrams?

JOËL: Occasionally, I will just draw it out by hand with pen and paper. But more recently, I've been using tools like Mermaid.js and specifically the website mermaid.live that allows you to just put in some names and arrows, and it will build out a diagram for you. And that's been really helpful to explore and understand what is going on with different entities that relate to each other.

AMANDA: I've used Mermaid.js recently, and I really enjoyed it as well. I found that writing something that lets me write words or something somewhat like words and takes care of the drawing for me is probably best for everyone involved.

JOËL: Yeah, that's a good point. It's kind of like Markdown, the ability to just write a little bit of text and move on and not worry about the size of boxes or the shape of the arrows or whatever. It helps you to really stay in that flow and keep moving.

AMANDA: I definitely agree. I feel like I can't have been the only person that somehow ended up very deep into the Figma documentation because I didn't quite know how to do what I was supposed to do, and I forgot what I was trying to draw in the first place.

JOËL: Right. It's really easy to put your designer hat on and want to make something like a beautiful diagram when this is really more of a capturing your state of mind. It's a rough note, not something you're necessarily going to publish. So, in addition to visuals, do you find yourself taking a lot of notes when you're exploring code or debugging code?

AMANDA: Yeah. I feel like I tend to jot a lot of things down, maybe class names, maybe some links to PRs or issues, or anywhere that might have context about what I'm looking at and how it got in that way. At this point in the process, it feels my notes usually feel like a bit of a bullet point list that doesn't quite make sense to me yet but maybe will get some shaping later.

JOËL: What kind of things do you tend to record in those notes?

AMANDA: I think one of the things that I'm usually trying to get out of those notes is just a snapshot of what I'm trying to accomplish at the time that I'm creating them. What's the bug that I'm trying to solve, and how did I get into this rabbit hole? So that if it ends up being the wrong one, I can follow my breadcrumbs back out and start a different way.

JOËL: Oh, that is really powerful. I love the imagery you used there of following breadcrumbs. And I feel like that's sometimes something I wish I had when I'm either exploring a particular code path or trying to find a bug. And at some point, I've gone a pretty long path, and I need to back up. And I don't remember exactly where I was or how I got to this point, especially if I've gone down a path, backtracked a little bit, gone down a different path, backtracked, gone further down a third path. And so having breadcrumbs, I think, is a really valuable thing that I wish I did more when I was debugging.

AMANDA: Yeah. And one of the most helpful breadcrumbs that I found is just a list of questions. What was the question that I was trying to answer when I opened this file or looked at this method, and did it help me solve that question or answer that question? And if the answer is no, then I can refer back to what the question was and try to think about what else might help me solve that question.

JOËL: I also love that. It's really easy to get sidetracked by other questions or other ideas when exploring or debugging. And sometimes I find that half hour later, I haven't answered the original question I came here to answer, and I kind of haven't even tried. And so, maybe writing down my questions before I go down a path would help me stay more focused during a debugging session rather than just trying to keep it all in my head.

AMANDA: I very much relate to getting nerd sniped by something that looks interesting but ultimately doesn't solve the original problem that you were trying to.

JOËL: This even happens to me when I'm pair programming. And so we'll say out loud the question we're trying to answer is this; let's open this file. And then you go into it, and you're like, oh, now that is an unusual line of code right there.

AMANDA: [laughs]

JOËL: I wonder why they're doing that. Let me check the git blame on this line. Oh, it's from 2015?

AMANDA: [laughs]

JOËL: I wonder what was happening there. Was that part of a Rails upgrade? And then, at some point, the other person has to interject and be like, "That's all fascinating, but I think the question we're actually trying to answer is..." and we get back on track.

AMANDA: I feel like that's a really good opportunity, maybe for a different kind of note of just interesting curiosities in a given codebase. I find that one of the skills that I'm trying to get better at is, rather than building a repository of information or answers to questions, just building a mental map of where the information I'm trying to find lives so that when someone asks me a question or when I have to solve something I don't necessarily know the answer, but I just know the resource to find that will point me in the direction of that answer.

And I feel like those kinds of explorations are really helpful for building out that mental model, even if it may be at the time seems like an unrelated rabbit hole.

JOËL: So this kind of note is a bit more permanent than a bread crumb style note would be.

AMANDA: Yeah, maybe. And I guess maybe it's less of a note, and it feels kind of like an index.

JOËL: Hmmm.

AMANDA: Like something that's connecting other pieces of information.

JOËL: That's really interesting. It's got me thinking about the fact that note-taking can be very different in different situations and for different purposes. So we've talked a little bit about debugging. I think we've mixed debugging and exploration. Maybe those two are not the same, and you treat notes differently. Actually, do you treat those two as different, or do you have different approaches to note-taking when you're exploring a new codebase versus debugging a particular problem?

AMANDA: I think that those kinds of notes could probably be a little bit different because I think when I'm onboarding onto a new codebase, I'm trying to cast a pretty wide net and give some overall information about what these things do that by the time I'm very deep in debugging, it might be information that I already know very well. So I feel like maybe debugging notes are a little bit more procedural. They are a little bit more I did X, and I did Y, and I did Z, and these were the questions.

And the introductory notes to a new codebase might be more along the lines of this is what this model does, and stuff that will eventually become second nature and might be useful to pass off to someone else who's onboarding but which I might myself not refer back to after a certain amount of time.

JOËL: I see. That's an interesting point because not only might the type of notes you take be different in different scenarios, but even their lifespan could be different. The value of a debugging note, that sort of breadcrumbs, might really only be that useful for a few hours or a couple of days. I can imagine notes you're taking while you're exploring a codebase those might be helpful for a much longer period and, as you said, maybe in passing them on to someone else when they're joining a team.

AMANDA: So that makes me think of whether the debugging notes should be as short-lived as I'm making them sound because I feel like there are times where you know you've debugged something previously, but you didn't keep the notes. Maybe they were just on a scrap of paper, and now they're gone. And I feel like I'd like to do a better job of digesting those notes a little bit better and eventually turning them into something that can be a little bit longer-lived.

JOËL: That's fair. I find that, especially for debugging, I like to capture a lot of what was in my notes in the eventual commit message for the fix. Of course, my random breadcrumbs probably don't make sense in the commit message, but a lot of what I have learned along the way often is helpful.

AMANDA: That's a really good point. I hadn't thought of commit messages as notes, but you're right; they totally are.

JOËL: One thing I've done is I've sort of taken this idea to the extreme. I was debugging some weird database table ActiveRecord model interactions, and the modeling was just a little bit unusual. There were multiple sources of truth in the relationships. And there were enough models that I struggled to really understand what was going on.

And so I drew an entity-relationship diagram. And I felt that that was important to understand for people reviewing the code but also anybody looking back at the commit later on. So I used a tool called Monodraw, which allows you to draw simple diagrams as ASCII art. And so, I have a little ASCII art ERD in my commit message.

AMANDA: That's incredible. I feel like if I were a developer git logging and I saw that commit message, I would be both thrilled and terrified of what exactly I was diving into in the git blame. [laughs]

JOËL: Definitely both, definitely both. But I have referred back maybe a few months later. Like you said, I had to refer back to that commit because a similar bug had cropped up somewhere else. And I knew that that commit had information that I had gathered that would make the debugging experience easier.

AMANDA: I guess the commit message is a really good example of having a note that's very closely tied to its context. Like, it's in the context of like a commit, which is a set of changes at a point in time, and it's really well situated in there. What do you think about the trade-offs of having that as part of a commit message versus something like some other sort of documentation where something like that could live?

JOËL: I guess it depends on how you think you're going to use it in the future. Again, for debugging things, it feels like you don't often need to refer back to them, so I don't think you would want to just dump that on a wiki somewhere. It probably makes sense to have that either in just a collection of debugging notes that you have or that you could then dig into if you needed or in a commit message, something like that. But maybe some of the things that you learned along the way could be pulled out and turned into something that lives somewhere else that's maybe less of a note at that point and more of a publication.

AMANDA: That sounds like a fine line between note and publication.

JOËL: Perhaps it's an artificial line that I'm making.

AMANDA: [laughs]

JOËL: But yeah, I guess the idea is that sometimes I will look at my own debugging notes and try to turn them into something like either a wiki page for a particular codebase or potentially even a blog post on the thoughtbot blog, something that I've been able to synthesize out of the notes there. But now you've kind of gone a few steps beyond the underlying raw notes.

AMANDA: I'm interested in your thoughts on that synthesis of notes into how does something go from a commit message into a blog? What does that process look like for you?

JOËL: I have a personal note-taking system that's loosely inspired by a system called Zettelkasten and also another similar system called evergreen notes. The idea is that when you learn things, you capture small atomic notes, so they are an idea in your note-taking application, and then you connect them. You create links between notes. And the idea is that there's a lot of value in making connections between notes that's almost as much part of the knowledge-creating experience as capturing single notes on their own.

And as you capture a bunch of these little, tiny notes over time and they become very interconnected, then you can start seeing, oh, this note from this one experience, this note from this conference talk, and this note from this book all connect together. And they maybe even make connections I hadn't seen, or I hadn't thought of individually in those moments. But now I see that they all kind of come together with a theme. And I might then combine those together to make a blog post or to use as the foundation for a conference talk.

AMANDA: That's really interesting. I like the concept of being able to capture bits of information at the time that they felt relevant without having to have an entire thesis in this note. Or that idea doesn't have to be fully fleshed out; it can become fleshed out later when you connect the dots.

JOËL: That's a really powerful concept. One of the big ideas that I picked up through this was that there are always byproducts of knowledge creation. So if I'm writing a blog post, there are always some things that I cut that didn't make it into the blog post because I'm trying to keep it focused. But those are still things that I learned, things that are valuable, that could be used for something else.

And so anytime I'm writing a blog post, preparing for a conference talk, learning some things in a debugging session by reading a book, there are always some things that I don't use necessarily immediately. But I can capture those little chunks, and eventually, I have enough of them that I can combine them together to make some kind of other work.

AMANDA: I'm really curious about your process of creating those notes. If you're reading a blog post, say, to learn a new topic and you're taking notes on that, how do you go from this concept that you're learning in the blog post to these really focused notes that can be combined in other ways?

JOËL: So the Zettelkasten approach suggests that you have two forms of notes, one it calls literature notes which are just sort of ideas you jot down as you are reading some work. You're reading a book or a blog post or watching a talk, and then, later on, you go and turn those into those atomic-separated, linked notes together, what Zettelkasten calls permanent notes. And so I'll often do that is just focus on the work itself and jot down some notes and then convert those later on into these smaller atomic chunks.

AMANDA: That concept of taking a larger theme and then actually spending the energy to distill that into a different kind of artifact that might be helpful later on is really interesting. And I don't do Zettelkasten note-taking, but I've also found that to be useful in other contexts as well.

JOËL: One thing that I sort of hold myself to is when I am writing those atomic notes is, I don't write them as bullet points. They're always written in prose and complete sentences. The title is usually a sort of thesis statement, a thing that I think is true or at least a thing that posits that could be true, and then a short paragraph expanding on that idea which I think helps cement a lot of information in my mind but also helps to give me little chunks of things that I can more or less copy-paste into an article and already have almost a rough draft of something I want to say.

Do you find that when you synthesize ideas into notes that you do something similar, or do you stick mostly to bullet points?

AMANDA: I think I might do a mixture of the two. I think procedurally, I use bullet points a lot, but I think those bullet points tend to be full sentences or several sentences together. I've definitely run up against some of the drawbacks of terseness, where they're less helpful when you refer back to it later. But I do like the visual cues that come with things like bullet points, or numbered lists, or even emoji and note-taking to be a visual cue of what I was thinking of or where I can find this later on.

JOËL: I love emoji; emoji is great.

AMANDA: I guess actually I've started using emoji as bullet points. That's something that I've found even to be helpful just with remembering or with grouping things thematically in my mind. And when I'm going back through my notes, I find it easier to find the information that I was looking for because it had a list, or an emoji, or an image, or something like that.

JOËL: Yeah, that makes it really easy to scan and pick out the things that you're looking for. It's almost like adding metadata to your notes.

AMANDA: Totally.

JOËL: That's a great tip. I should do that.

AMANDA: You can definitely run into the Figma problem of you then spend so much time finding the right emoji to be the bullet point that you forgot what you were doing, [laughs] but that's a problem for a different day.

JOËL: So this sort of Zettelkasten evergreen notes approach is a system that I use specifically to help me capture long-term thoughts about software that could eventually turn into content. So this is very much not a debugging note. It's not an exploratory note. It has a very particular purpose, which is why I write it in this particular way.

I'm curious; I know you have a lot of different systems that you use for your notes, Amanda. Is there one that you'd like to share with the audience? Maybe tell us a little bit about what the system is and why it's a good fit for the type of scenario that you'd like to use it in.

AMANDA: Sure. One situation that I found myself in recently is I have started taking classes on things that I'm interested in, development-related and non-development-related. And that's a formal structure that requires some note-taking that I haven't really done since I was in school. And the tools were very different back then as to what we had available to us for note-taking. It was basically Microsoft Word or bust.

So I have found myself having to develop a new note system for that kind of content delivery method, basically of watching a video and taking notes and having something that then makes sense outside of the context of sitting down and watching a video. So that has been a little bit of a process journey that I've been on the last couple of months.

JOËL: And what does your note-taking system look like?

AMANDA: So it's been a mix of things, actually. I started out just kind of brain-dumping as I went along with the instructor talking trying to type and keep up. And I found that very not scannable to look back on. I was looking for some more visual cues, and I didn't really have time to insert those visual cues as I was trying to keep up with a lecture essentially.

I then transitioned back to old school pen and paper, like, got myself a notebook and started writing in it. And obviously, that has some benefits of the free-formness, like, I'm not constrained by the offerings of any specific tool. But the trade-off for that is always that you have different notebooks for everything. And it's like, where's my X class notebook?

And so I've been trying to bring those two methods together into something that makes a little bit more sense for me and also bring in some of that synthesis process that you were talking about with your note-taking method of doing the full literature notes and then synthesizing them down into something a little bit more well-scoped for the particular piece of information.

JOËL: So you have like a two-step process then.

AMANDA: It did end up being a two-step process because one of the things that I found was the grouping of ideas that make sense when you're first learning a concept and the grouping of ideas that make sense when I'm revisiting that concept, later on, aren't necessarily the same. And so, keeping it in the original context doesn't necessarily help me recall the information when I'm referring back to my notes.

JOËL: That's really interesting. When you're writing it, it's going to be different than when you're reading it.

So we've been talking a lot about the purpose of different notes along the way, and you mentioned the word recall here. Do you use these notes mostly as a way to recall things that you would look back at them and try to remember, or are they more of a way to digest the material as you're going through it?

AMANDA: I think at the time that I'm writing them, they definitely served the purpose of helping me digest the information. But at some point, I probably want to be able to look back at them and remember the things that I learned and see if maybe they have new salience now that I have sat on them for a little bit.

JOËL: Hmmm, that's good. So it's valuable for both in different contexts.

AMANDA: Yeah, definitely. And one of the more surprising things that I've learned through that process has been that when I'm learning something, I really like a chronological kind of step-by-step through that process and building blocks of complexity that basically go one on top of the other. But then, once I've kind of made it to the end when I look back on it, I look back on those notes, and they're usually pretty thorough. They probably have a lot of details that aren't going to be top-level priority at the end.

But I didn't really have that concept of priority when I was first learning it. I was kind of grasping onto each bit of information, saying, "I'm going to scroll this away in case I need it later." And then when you get a better understanding of the full picture, you realize, okay, I'm glad that I know that, but it's not necessarily something that I'd want to look back on. So I really like having systems that then allow me to regroup that information once I have built out a fuller picture of what it is I'm trying to learn.

JOËL: Interesting. So the sort of digesting step that happens afterwards or the synthesis step, a lot of the value that you're adding there is by putting structure on a lot of the information you captured.

AMANDA: Yeah, I think putting structure and changing the structure, and not being afraid to change that structure to fit my new understanding in how I see this concept now instead of just how this concept was explained to me.

JOËL: So you mentioned that you'd initially used notebooks and paper and that that felt a little bit constraining in terms of organization. Is there any kind of software or apps that you like to use to organize your notes, and how do they fit in with your approach to note-taking?

AMANDA: I've been using Notion for the last few years. I found that that application works well with my visual preferences for note-taking. I think there's a lot of opportunity for visual cues that help me recall things, such as emoji and bullet points. And I like that I can do all of that by writing Markdown without then also having to read Markdown.

JOËL: Yeah, I definitely agree that a little visual change there where you can actually see the rendered Markdown is a nice quality-of-life improvement.

AMANDA: Absolutely. And I also think that the way that it turns Markdown into blocks that then you can rearrange has served me really well for that synthesis process of maybe this bullet point makes sense, and I want to keep it as is. But I want to rearrange it into these new themes that I'm seeing as I'm reviewing these things that I've learned.

JOËL: That's fascinating. So it has some really good tools for evolving your notes and reorganizing them, it sounds like.

AMANDA: I like that I can group my notes by concept, and notes can be subsets or sub-notes of other notes. And you can kind of move the individual notes in between those blocks pretty easily, which helps me rearrange things when I see different themes evolving.

JOËL: I've heard really good things about Notion, but I've not tried it myself. My app of choice so far has been Obsidian, which I really appreciate its focus on linking between notes. It doesn't have this concept of blocks where you can embed parts of notes as notes into other notes and things like that. But that has been okay for me because I keep my notes very small and atomic. But the focus on hyperlinking between notes has been really useful for me because, in my approach, it's all about the connections.

AMANDA: So, what does that process look like when you are referring back to all of these notes that are hyperlinked together?

JOËL: That's actually really important because the recall aspect is a big part of how you would use a note-taking system. For me, it's sort of like walking the graph. So I'll use search, or maybe I know a note that's in the general theme of what I care about, and then I'll just follow the links to other related articles or notes that talk about things that are related to it. And I might walk that graph three, four steps out in a few different directions. It's kind of like surfing Wikipedia. You find some entry point, and then you just follow the links until you have the material that you're interested in.

AMANDA: It sounds like creating a Wikipedia wormhole of your own.

JOËL: It kind of is. I guess, in a way, it's kind of like a little mini personal wiki where the articles are very, very condensed because I have that limitation that everything must be atomic.

Wow. So this has been a really fascinating conversation. I feel like one of the big takeaways that I have is that types of notes matter. Note-taking can take very different forms in different contexts. And the way you organize them would be vastly different; how long you care about them is also going to be different.

So going into a particular situation, knowing what sort of situation is this that I'm using notes and what is their purpose is going to be really helpful to think in terms of how I want to do my note-keeping. Whereas I think previously, I probably was just like, yeah, notes. You open a document, and you put in some bullet points.

AMANDA: I am definitely guilty of doing that as well. And I like the idea of having a purpose for your notes. You mentioned your purpose was ultimately to build a map that would produce content. And I really like how you have found a system that works really well for that purpose. And I'm going to keep thinking about how to be more intentional in what is the purpose of the notes that I'm taking in the future.

JOËL: Well, thank you so much for joining the conversation today. Where can people find you on the web?

AMANDA: Thanks so much for having me, Joël. You can find me @amandabeiner on Twitter.

JOËL: And we'll link to that in the show notes. And with that, let's wrap up.

The show notes for this episode can be found at bikeshed.fm.

This show is produced and edited by Mandy Moore.

If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show.

If you have any feedback, you can reach us at @_bikeshed, or reach me at @joelquen on Twitter, or at hosts@bikeshed.fm via email. Thank you so much for listening to The Bike Shed, and we'll see you next week. Byeeeeeeee!!!!!!!

ANNOUNCER: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.

Support The Bike Shed