Chris gives a DB sessions update and talks bifunctors & command objects. Steph shares the coolness of a gem she's been using called after_party, and excitedly gushes about her new laptop. (Chris is hoping to hold off on replacing his until the end of the year and then they can compare!)
The two then answer a listener question on retrospectives and how they've seen productive ones run, while giving some of their own helpful opinions on dos and don'ts. They're talking to you, Grumpy Goose!
- dry-monads gem
- attr_extras gem
- after_party gem
- What Went Well? - Bike Shed 123
- What I Believe About Software - Bike Shed 172
- Is Agile Over? - Bike Shed 299
- Running a Retrospective - Upcase
Transcript:
STEPH: Cool.
[laughter]
CHRIS: Good. No, I like what you did there.
STEPH: Yeah, I feel like we can get rambling on that one.
CHRIS: It's been great. This is what the Bike Shed is at its best. It's the two of us just rambling and being like, well, what about this? And if it's this, then that, then these, and it depends. And it's complicated and it's nuanced. And what about the humans? That's the story of The Bike Shed right there. [laughs]
STEPH: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Steph Viccari.
CHRIS: And I'm CHRIS Toomey.
STEPH: And together, we're here to share a bit of what we've learned along the way. Hey, Chris, how's your week?
CHRIS: My week has been good. I have some updates actually on some topics from previous episodes. One of the things that I can update on is the discussion around the cookie versus the database store. So I had posed this as a thing that I was going to be doing in the app for a handful of reasons. Most notably, I wanted the ability to invalidate sessions from the server-side, wanted to have a little more control over that. And so that's a dream that the database-backed session store can do. Eventually, I have to make that actually work in the way that I want.
But I was asking the question in that episode, which we can include a link to the specific episode, but I was asking the question of why don't we just do this all the time? The database-backed sessions seem better in all these ways. It's a lower overhead per request because you're just sending the session ID and the cookie instead of the whole payload of the session. You actually can have more data stored in it, a bunch of things that seemed really great. And then right after I introduced it, I figured out the thing. I figured out the secret. It's not a big issue, and we're going to stick with database session stores. But we have to be purposeful because it turns out they are essentially plain text in the database.
And so if there's anything that you are putting into the session like say a social security number or an authentication token or other things which naturally I might have done if it was in a cookie that lives on the user's browser and never actually lives on the server, persists on the server, that seems fine to me. But now these things are getting stored in the database and that really changes the calculus, especially because if I'm not purposeful, they'll just stick around for forever. So social security is probably the most pointed example of this. If you happen to have a form in the app that accepts a social security number and you want that to persist through some number of other steps, not actually going to store the social security number in the database because that's a thing that I have actively chosen not to do. I need to send it off to some other system, but I do need to hold onto it for a few minutes. The session is a perfect place to put that unless the session gets stored in my database.
STEPH: That's such a great point. I'm so glad you discovered that. And in our recent conversation, we were trying to think of the reasons why this isn't the default case. You may be headed in this direction, but this may also be timely, the fact that you're discovering this issue but also the fact that Rail 6.0 now has encrypted columns. Is that where you're headed with the fact that you can still keep session data in a database?
CHRIS: That is a great question, and it is an intriguing option. But it's not the one that I'm going with here. I think broadly, my hope is to completely avoid ever persisting this data in the database, this truly sensitive user-specific PII or PCI or social security numbers or any of these other fancy acronyms that get collected together under the umbrella of I probably don't want that on my server. For those, I'm just opting to push them back into a cookie. So I'm using particularly a...In rails, it's fun because they have a fluent interface where you can just chain together things, so it's cookies.signed.encrypted.whatever, and then you go from there. But I'm using signed, encrypted cookie, which is essentially what the session store views; the cookie session store uses itself.
So I'm basically reverting to the old session store behavior for specific values. So anything that is truly sensitive like that, I'm just saying, cool, that's actually just going to live in a cookie, and that will be it, but not leaning on the ability to encrypt the database sessions. There's just enough subtlety around that. There's so much volume of data if I do allow that sensitive data into the system that any failure, any exploit that happens, would be somewhat catastrophic. So, in my mind, this lowers the surface area and says, yeah, this data really never lives on the server. It comes with a request, and then it's gone after the fact. And that's the world I want to live in.
STEPH: Yeah, that's super interesting. You're also raising questions for me that I hadn't considered when we originally had this conversation where there's necessity or that you're looking to store form data or sensitive data in that session as well. So that makes a lot of sense to me that for that type of behavior, we're going to separate that from the idea of authentication and the user session and still use encrypted cookies for those details. So it only stays with the user's browser, but then the actual authentication of a user that part could still live in the database.
CHRIS: It is ending up being a weird Venn diagram of; this is data that I want to stick around but only sometimes and particular to the machine that the user is interacting with because the session is still associated with a cookie at the end of the day. So a user may have multiple sessions in the database-backed session version. It is somewhat interesting, and I'm going to see how it develops over time. But yeah, at a minimum, I have now found this edge case of like, ooh, okay, sensitive data, that's a thing, which is one of the reasons that I would reach for the session inherently. So it turns out, as always, it depends. Things are complicated.
STEPH: It's a nice update. I like when we have closure to a question like that, especially so quickly.
CHRIS: Love to provide that continuity. It's what I'm all about. But yeah, what's up in your world?
STEPH: What's up in my world? I am excited that I have a new laptop. So I have been using a MacBook Pro for about the last...it's lasted me for a while. I think I've had it for a good four, four, and a half years, but that's on the fritz. The keyboard, in particular, the keys are popping off, and that's something that I could go get fixed. But I'm at the point that I need a new laptop. So I have a brand new shiny laptop. And I had the option to either go with a 16-inch MacBook Pro or to go with one of the new, fancy laptops that has the M1 chip. And I was torn for a little while because having the M1 chip sounds really cool and novel, and there's a lot of speed improvements that come with that.
But I ended up going with just the 16-inch MacBook Pro; specifically, one, it's still very fast. It's very reliable. I can use this as my work machine and just know everything's going to work. That part feels really important to me. And then also the screen size is important. So any Mac laptop that is using the new M1 chip, I think they only go up to a 13-inch, right now, screen size. And I really want the 16-inch in case I am traveling, so I have that larger desktop. But I did do some research into the M1 just because I know about it. I know it's out there. I know it's hot. People are interested in it, but I didn't know a whole lot about it. So for anyone else that's like me and is curious about what the heck this M1 chip is, it's essentially Apple's foray into making their own processors.
So traditionally, their machines have used Intel CPUs and third-party graphic processors, and other parts. And this introduction of the M1 chip really represents Apple's switch to having their own internal architecture rather than relying on those third-party parts. And it also means that all those features that were sourced from other parties like the CPU and its security are now being combined into a single chip, which has also led to some performance improvements. And while I was reading about the M1, there's a lot to go through, but the thing that stood out to me was this idea of Apple's Neural Engine. And I thought, well, that sounds super fancy. What is that? Are you familiar with Apple's Neural Engine? Have you read about that?
CHRIS: I don't think I am. What all is that?
STEPH: Yeah, good question. That was a question I was asking myself just recently. So essentially, their neural engine it's a microprocessor that specializes in the acceleration of machine learning algorithms. So it's really similar to how a GPU will focus on accelerating graphics rendering. And their neural engine or neural engines in general then focus on accelerating neural network operations. And the inclusion of a neural engine isn't something that's new because Apple introduced this into iPhones and iPads back in 2017 to support their features like Face ID and emoji, searching for photos with dog pictures. Siri speech recognition is also one that's using this engine and other machine learning tasks.
But the sparkly stat that Apple is sharing with this new design is it's a 16-core design that can perform 11 trillion operations per second, which sounds very fancy, very fast. But it really got me thinking about how companies are working to improve, not just laptops but also our mobile devices to run machine learning software more efficiently, and then how that's just going to evolve and change all the different features that we use, and then how developers can integrate with this engine. I think currently, Apple hasn't shared much information about how this engine works, but I think they've exposed a few developer tools so people can still build features that will then use the power of this faster, improved neural engine.
CHRIS: Oh, that's super interesting. I have still not really delved into machine learning or artificial intelligence, or any of that stuff in any real way. But it's one of those things like the number of mentions is ticking up. And at some point, I'm like; I probably have to pay attention to this, don't I? I'm still in the not paying attention to it camp. So if I'm understanding, though, you just described this wonderful feature, but you opted for the machine that does not have all the fancy stuff. You did not...
STEPH: Exactly. [laughs]
CHRIS: Okay, yeah.
STEPH: Yes, it would have been nice. That would have been neat, but yeah, I needed a machine with a larger screen, all those good things. And that's still really fast, for the record.
CHRIS: Oh yeah. I'm desperately hoping to make it to the end of this year. This is going to be a bit of a rumor mill here, but my understanding is the expectation is that Apple is going to release a 14-inch MacBook Pro with the M1 and the return of MagSafe, and the removal of the touch bar. And that sounds like my dream machine right there. I want that piece of hardware.
I also seem to care a little bit less about the size of the laptop screen. I'm so often working at my desk with a large-format monitor that I'm connected to. And so, when I'm on the road, I want to optimize for portability when I'm traveling because I do it so rarely, and then I'm hopefully focusing on travel at that point. But we'll see if that remains true as the shape of my work changes and I start to not only work from home. And maybe I'll actually change my tune on that. But for now, that's my hope is to make it to that machine and then get one, and that it exists because right now, those are all rumors.
STEPH: Well, I totally support this goal of yours. So that way, you can have that new-new, and then you can report back on what it's like, and then we can compare. Because I'll have the other version, the Intel CPU, and then you'll have the M1 chip, and we can see how our lives are different.
CHRIS: You'll have the new, and I'll have the new-new, and that's how we'll categorize them.
STEPH: [laughs] But yeah, I'm very much looking forward. Having a new laptop is always just such a fun feeling. It's just a clean space that I get to rebuild. It's like going through and prioritizing; what are the things that still spark joy? And then I get to only port over the stuff that I still really use all the time and want to keep. So I'm looking forward to getting it set up.
CHRIS: I need to do that sometime soon. I'm like five years deep, at least on this machine. So I've been dragging along. Also, the hard drive is just completely full, and I regularly have to go through and delete things before we start recording because it turns out these audio recordings start as very large files. [chuckle] So it's almost a weekly thing where I'm just like, got to throw something out today. I don't know what. It's fine. I'm going to be fine. [laughter] I'm going to make it to the end of the year, and it's going to be great.
STEPH: What else is going on in your world?
CHRIS: Well, I wrote some fancy code, and I use fancy not necessarily as a good word. [chuckle] So I'm intrigued that the code could be described perhaps as clever or other words like that, which I think are very complicated words in the coding space. I tend to try and avoid this type of coding where I'm trying to introduce abstractions and clean things up, and remove duplication because I've been burned by that so many times in the past. But this time, I think maybe this time it'll work.
So, in particular, there are two different areas of the application. There were two sets of refactorings, but they really went together. One is we have the idea of command objects within the application that we're working on. So there are a lot of cases where we need to save something to the database and then communicate something to an external API. And then presuming the results of that is a successful response from them, then unpack some data, make sure it's in the right shape, and then save something else to the database. And ideally, wrap that all in a transaction and keep everything together and then return some data at the end of it.
So that whole sequential operation, I've been using dry-monads to model that. I've talked about this on a few previous episodes. I'm really enjoying it. The more I lean into it, the more I find that it is just a really great way to wrap up that very procedural code. But ideally, do it in almost a functional way so that we've got these sequential operations that feed into each other. There's the railway-oriented programming stuff, which is associated with this idea.
But there is a lot of boilerplate to these objects. So the way we've defined them is they have a class method called run that takes whatever the arguments are, and then it needs to pass those arguments into the initialize and then call run on the instance. So in order to define one of these objects, what we had been doing was def self.run and then all the arguments. And then inside of the body of that, it's new, and then pass forward all the arguments .run, and then define initialize to capture all of those and set all the instance variables, and then define the run method, which actually does stuff. Also need to define an Adder reader for all of those instance variables, which is a thing that I enjoy doing. So that's the interface I want, or that's the way that I want this class to work. I know other folks in the Ruby world feel differently. But that's the shape of the thing that I want, but that's a lot.
And there's also I regularly would find myself forgetting to duplicate something that we put into the class method run interface into the initialize method. And it was just like, this is all just wiring up and plumbing. There's also the binding of the dry-monads do notation for the run method as well as the inclusion of the results type within dry-monads. Type is a strong word, but that gives us the success or the failure objects that we can create. So ideally, all of these command objects either return a success object or return a failure object. It's one of the two. And that's one of the things that I really like about them. But yeah, so much plumbing.
So we define a base command, and the base command has the self.run method, the class method, and that method is defined very abstractly. So it's just args * keyword args. So we're capturing all of the arguments and then forwarding them on to new. So that way, I don't have to think about that interface. It basically just says, "Give me anything, and I'll forward it onto new." And the new or initialize is in charge of actually defining things. It also includes the result type. It includes the macro annotation for the run method, which is how dry-monads does its magic, that actually I had to include inline within the self.run, just because of the sequence of definition and the metaprogramming that's going on there. As I said, that sentence terrifies me a little bit, but hopefully, no one ever needs to look at this magic base class [chuckles] and figure anything out. So that was one part of it. That cleaned a lot of things up, so that meant I didn't have to write a ton of the wiring up code.
Then there was still the noise of actually defining all of the arguments to these classes. They often take a handful of arguments because that's their job is to grab a bunch of things and do some work with those things. So for that, I have brought Adder Extras, which is a gem that I've talked about probably in previous episodes, I think so. But this is the first time that I've really leaned into it and used it. And it gives some very high level what look like macros are just class methods. But the one that I'm using is Adder private initialize, and that you can then pass a variety of values too. And it will then say, okay, this method accepts a required keyword arg, a defaulted keyword arg, and a positional argument or something to that effect. But it's a very, very concise way to express that and then also get the private Adder readers, which again is the direction that I want to go with all of this.
So that’s a bunch of things that I have said. But all total, it cleaned up these command objects very nicely. And now, when you look at one of these command objects, all you see is the run method that does the work. And the plumbing and the wiring up behind the scenes should just happen. I am concerned about the day that someone forgets to inherit from the space command, and then it's like, why does nothing work? I thought command objects just worked in the system. But we're going to deal with that when we get there, which is hopefully a while down the road.
STEPH: I like how you're pushing at the boundaries of our comfort zone. I say our comfort zone because I imagine we feel similar.
CHRIS: It is. We definitely got a shared comfort zone. [laughter]
STEPH: Yeah, we have a shared comfort zone with inheritance, but you're pushing at that boundary of that comfort with inheritance because it is something that can be so painful. But you've identified an area where inheritance feels useful. And then it also sounds like a very meaningful...you're introducing this pattern and then trying to make it easier for others to follow this pattern. So it's a very intentional design decision of where we want to group these behaviors together and then make it very easy for other developers to then pick up this pattern and run with it, and then also work with these classes. So I am intrigued to hear how it goes and how others feel about the pattern as well.
I also wonder, this is one of those areas where it feels like this very intentional design decision. Is it something that you think in the base class would be worth highlighting? Like, hey, here are the things that we are using in this base class. This is the intention of this base class. I don't know if that's maybe a comment or if that's something that's documented in the README. I know; I see your eyebrows went up when I said comment. But it does feel like one of those areas where it's like, hey, we have introduced this new concept. We want you to follow along. Here are some helpful guidelines.
CHRIS: Those were mostly joking eyebrow raises because I have thought of that. I haven't actually gone to that level. But in the back of my mind, there's this pattern that we have within this application. Ideally, we're going to lean into it more and more so that A, we have a clear way that we do things within the app but also make that as understandable and discoverable as possible. I'm not sure if a comment in the class is the right thing or...so I'm deferring what I want to do on that for now because right now, it's myself and one other developer. We sort of developed this in tandem. So we were working together on it. We would pair in a bunch of the features. And what we have now is the crystallization of what we found useful. And we're both very comfortable with it. So there isn't the need to explain it.
I'm almost thinking about it as just-in-time educational content around this piece of our application. I don't actually trust that I would do a good job describing it in the abstract because I know it. Like, to me, this thing makes sense right now. But I've been on the other side of stuff where someone was like, "Hey, this totally makes sense." And I'm like, "I don't know any of the words you just said," and so I felt that pain being on the other side. You could say I'm just being lazy, but I do think this is a purposeful delaying of that where I want to wait until I actually have someone to teach this to. And at that moment, I want to see what that conversation looks like. And I'll try and explain it to the best of my ability, but I'm sure they're going to ask questions, and I'll be like, "Oh, wow. Yeah. I hadn't even thought of that. But now that you ask the question, totally let me explain this part that I was going to gloss over and forget to mention."
And so, ideally, that is what will happen down the road. And then from that, hopefully, some artifact becomes clear, whether it's a documentation page in the repo or a comment in the class if it's simple enough or maybe even it's a recording of a pairing session. And that's the artifact that we keep around that explains this piece of the application. So I definitely think a version of that makes sense, but I am not doing it yet.
STEPH: It's funny; you’re saying so many good things that I agree with. I love the just-in-time education; that part is fun. And yeah, there's a part of me that definitely still leans into the idea because we've talked about this in the past too, where we write down, in the moment, the things. Having that context when we're implementing it is really important and helpful. So even if it's not this grand explanation…which I really like what you said around having someone to explain it to or have that conversation with so that way you're documenting the useful bits, that part I like very much, but capturing the intent as soon as the change was introduced. So even if it is a very high level like, hey, we are using dry-monads and Adder Extras, even if it's just links to those things, that's something that I think I would still favor just to go ahead and start surfacing this is a pattern. This is a choice. And then, as you continue to work with the pattern, if you change your mind, it's still very easy to scrap that documentation.
So I think if it were me, I would still go ahead and document it. I think it's that piece about discoverability that's calling to me so strongly where that's where I want to then highlight the things that are in use. So even if there's not an explanation, people can find the resources very easily. Because you're right, you did say a lot of interesting bits in describing this pattern. And the fact that we're talking about it now also just deepens my suspicion that it would be nice to comment somewhere, and perhaps a repo is a perfect place for it and just get it out there, and then it can always be revisited later and improved.
CHRIS: Okay. I like that you are keeping me honest on this because I do think there's a certain amount that I'm just being lazy here and not wanting to do that because it is actually really hard to try and document something like this. Like, what are the important pieces versus what are the extraneous details that people don't actually need?
I do wonder, so the pull request that did this refactoring and introduced this base command object that does have the explanation captures the point in time and whatnot. And so I wonder, is there a version of tagging important pull requests that tell the story of the application? A lot of things are just going to be like; this is adding a feature. It's the same as the other 30 pull requests recently that added a new feature. But this one is special from an architecture perspective. And so let's tag this, let's add a label. I don't know what it is but something that allows for discoverability of the story of how this application became what it is today because anything else I worry will go out of date almost instantly. But this pull request is true fundamentally in that same way that we say commit messages should capture as much of that detail.
So I did do that writing for the pull requests/the commit message. And I wonder if maybe that's the best artifact for this moment but then the question of surfacing it and making it discoverable because otherwise, it's just lost in the sea of other pull requests. So I don't know. But I do like the slight push back that you're giving me here of like, yeah, but what if you did something though? And I'm like, yeah, that's fair. I should probably do something.
STEPH: Being able to pin those specific PRs that have significant architecture changes sounds really novel, but I'm going to take this opportunity for me to be lazy. And if I'm joining a project, I don't want to read through what has happened. I just want to know what's true now. And if I go back and look at those PRs, I won't know if all of that is still relevant and how it's changed. So it sounds neat from telling the story of how an application has evolved. I like that sort of developer lens, and what are the things that we have tried and then changed over the years? But from I am onboarding to this application, I just want to know what's true today? What are the things that you want me to follow? What are the patterns that are going to be really helpful for me to look at? And so then, I don't know if I would use it in that context.
And this may be one of those areas where I'm feeling overly skittish in response to the number of things that you said and the use of inheritance. Because I have felt so much pain of where I'm going up the tree to figure out what the heck is happening in the world and then to understand all of those pieces, and then swimming all the way back down to the class that I'm actually working in. So it could just be past experiences that are now influencing how I want to document or work with inheritance. It probably is. [chuckles] That's probably a big factor of it. It doesn't mean I disagree with it because those painful experiences are meaningful. [chuckles]
CHRIS: Yeah. I think the foundational thing...I tried to start this with the context of like; I did a thing. This is another example of good idea, terrible idea; my favorite segment on The Bike Shed. I stand by it. I think it was useful. It does use things that we have traditionally moved away from. I say we because, again, I think we have a shared approach to development at this point. But I think it's worth it. I hear everything that you're saying about the documentation, but I've been burned by that so many times where the documentation is like, here's what's true now. And you're like, no, there isn't even a class called that anymore, and no less does it work that way. And so, my inclination is not to go that way.
The solution that I have in mind is when someone is onboarding into the application, I don't expect there to be documentation and other things that I can hand them so that they can run. I expect to sit down with them and work with them. I'm going to pair with people when they join a team for a long time. There's a period where that's true, I think, and then you get to a certain size of an organization, and you're onboarding enough people regularly enough that that's a thing that you should get better at.
But for I think a surprisingly long time, my answer I'm more than happy for it to be, yeah, we're bringing someone new into the team. Let's sit down with them. Let's spend the time. Let's tell them what's true because I know currently, and I can give them an up-to-date version of that. And ideally, as part of that, then update the static documentation, the repo, the README, the other things based on the conversation that we have and recognize oh, that that link is very out of date. Let me change that one real quick. But I'm not expecting to have comprehensive documentation for that. I'm expecting to use real human interaction to fill that gap.
STEPH: Yeah, I really like that you're also calling out how fallible documentation is and how it has misled us so many times. I also love what you highlighted where when somebody new is joining the team, we are more than often going to sit with them and then explore the app together. And it just made me revisit that phrase that you used earlier about the just-in-time education. Because for this command object, you may join the project and not need to interact with this design pattern for your first couple of weeks, first couple months, who knows?
So then it comes back to the idea of how when someone is in the space of where using a command object feels like the right approach, then how do we introduce them to this pattern and then make sure that they have the tools that they need? And if someone is accessible to then sit down and go with them, that's great. But if someone is not accessible, then I still want them to have at least a few of the resources that they need to dive into some of the more complex things that are being included. So, yeah, it's a tricky one. I like this thought experiment.
CHRIS: But yeah, overall, I'm happy with it for now. I'm hopeful it will work out for us moving forward, and I'm hopeful that it will also be a sufficiently discoverable or teachable thing within the application. But again, I will certainly report back and see how that one plays out for us. But yeah, that's what's up in my world. What else is going on in your world?
STEPH: Something else that's up in my world is I have pulled in a tool that I've used in the past, and I really like it. So I'd really like to talk about it here for a bit because I just find it so useful. And now that I've added it to this new project, it's just really top of mind for me. So I found that when working on a project, there are often times where I want to run something right after a deploy has happened, and I want that to be automated. I can do it manually. I can hop in, but then perhaps if you're deploying across many environments or many systems, you don't want to have to do all that manual work, or you also just want the convenience of you can set it and forget it. And that way, you know something's going to happen. So perhaps it's something where you want to change some data, or if you want to enable a feature flag, then this is really helpful.
So the gem I've been using for this is called after_party, where you can write automated deploy tasks that essentially behave very similar to migrations. So you can write a Rake task. It has a timestamp. You can implement the logic that you want to be run right after your code has deployed, and then after_party itself, we'll check the timestamp. It will see if it has been run. If it's already been run, it won't run it again. Or if you like, you can set it up so that way, you can tell after_party to say, "Hey, after every deploy, I want you to run this particular task," but it's such a nice improvement to the workflow.
And the other thing that I really like about this that I feel is a bit contentious is separating changing data outside of migrations. So I am a big fan of migrations are focused on changing your schema itself. But if there's actual data that you need to change, I really like when that is separated outside of the migration. There are definitely times that I understand it's really nice to just do it all at once, and it's easier. But anytime it starts to get even a little complex, I immediately want to write tests for it. And I can't test my migration. But if I'm changing some meaningful data on production, I want tests to back it up to make sure that I'm scoping correctly, that the outcome is exactly what I expect. It also makes it easier for other people to review. And after_party gives me that functionality so then I can have my migration.
But then I'm like, oh yeah, but I still want to automate changing this data because that's often one of the complaints that I hear from people when I do ask them to separate into a Rake task, changing the data. They're like, "But I don't want to have to then follow up and then run this task later." And I'm like, that's cool. After_party has you, and you can automate it and not worry about it. So after_party has been one of my favorite gems to add to applications.
CHRIS: That's interesting. There's a bunch of layers to everything that you just said. I think I've worked with after_party on a project. I think we were working together on that project, if I'm remembering correctly. I have no bad memories of it, which given the nature of the tool, makes me think it did its job very well because its whole point is just like, oh cool, now you can just do this thing, and you don't even really have to think about it. Because there are plenty of other times where I've had to orchestrate or do a deploy. And then I SSH tunnel into production, which is a bad idea, and then I'm running Rake tasks manually. And so, I think the fact that I don't have any pointed memories of this is a really good sign for a tool like this. So that's a weird vote in its corner for me.
You did say something that was interesting that I want to poke at a tiny bit which was you can't test migrations, and I think that's true. Like, I don't know of any way. And it feels like a thing that is sort of fundamentally deeply true. But I do wonder, is there any gem out there? Has anyone done a weird science experiment to figure out like, I would actually really like to be able to test my migrations? So I think the idea of having to pull data change out of migrations for the reasons that you said totally makes sense.
But there are often times where I want to convert from non-nullable to nullable. And in the process, I want to backfill with a given value or something to that effect. And I like to encapsulate that altogether such that if it fails or succeeds, it's transactionally consistent. And I do wonder, could I wrap a test around that? I don't know of a way, and I think it may actually be the Rails testing infrastructure is just like no, we prepare your schema for you in the background, so it's just up to date. And therefore, you don't even have a way to be in a state where the migration hasn't run. But it's an intriguing one.
STEPH: Yeah, that's probably a hard absolute that I said where you can't test it, and I'm sure there is a way to test it. How friendly or how easy that is to do, I'm really not sure of. It also feels like one of those areas where it feels like I'm testing this other service that I should trust fully, so then I'm not necessarily testing the migration itself. I'm testing some logic that I've added inside of the migration where I'm changing some data. And the example that you provided is perfect because that's one of those that I'm inclined to include in a migration. It's more like where we want specific users who have this value or in this category. And then, we want to migrate them from this data to the other data. And when we start getting complicated in our migrations, that's when I'm like, this is a bit much, and I'd really like a test that documents that we're doing this correctly. That's where I get squeamish about having data changes in migrations. But you do raise a good point. I don't know; I’ve never tried to test one. I've just always reached for this other approach, but that is more the pain point of if I could test this data change inside of migration, then that would work for me. That would solve my problem.
CHRIS: I wonder if an alternative approach would be to just introduce an object or a class that does this work. So like a command object as it were, to do a call back to earlier in the episode, that does that data transformation because it’s exactly what you're describing, for this subset of users do this. But if they're in this state, then do these things and create two new records for any user like this. That sort of stuff is really complicated. Definitely want to have some tests around it. But you're talking about a gem that allows you to extract it into a Rake task-like situation. But I wonder, could we just have a class for that?
And I used to be a big believer in your migration should live forever, and they should always be runnable from the beginning of time. I've given up on that belief. That's one of the things that I've been like; I don't know. It turns out I've never done that. It's not an important thing. Just DB schema load is going to be fine most of the time. It's great for the past ten migrations to be around just to tell a little bit of a story. But I'm not tied to migrations being runnable forever. So the idea of you introduce this class, it encapsulates that data transformation. You can test it because it's its own thing. It will still be run within the context of the transaction of the migration. And then you throw it away down the road along with the migration, and you do that migration roll-up thing. It's just a different thought there, although I do like the...well, I guess that would also run automatically, but that runs as part of the deploy as opposed to after the deploy, which is meaningfully different than what after_party does because there might be one of these migrations that takes a long, long time to run because you've got a ton of data. And you want to decouple it from the true deploy release sequence that happens and the time limits that are there. So I think I've now talked myself in three circles, and I'm going to stop.
STEPH: I like how you highlighted that part where it does decouple you from the deploy process where it's still automated; it runs afterwards. But say if it's something that doesn't need to hold up the deploy, you don't need to wait for this data to be migrated before the deploy can go out. Then that's a nice separation because then it can happen afterwards. Or if you do need it to happen part of the deploy, yeah, there's lots of interesting bits there. I feel like you and I could talk about it for a while.
But we have a listener question that I'm really excited for us to talk about. So I'm going to hard pivot over to our listener question. This question comes from Jonathan. And Jonathan wrote in, "Hey, gang, longtime listener, first-time emailer. I've heard you reference retrospectives a few times as part of your normal development practice. In my limited experience with them, I often find retrospectives don't feel productive because team members are reluctant to raise issues without seeming critical or blaming another team member. I would love to hear you describe how you typically run retrospectives to foster open discussion and make it a productive use of time. Bonus points," oh, I love bonus points "if either of you have experienced rescuing an existing team that was not having productive retrospectives. P.S. Thank you for ongoing participation in the Ruby and Rails communities. I look forward to seeing a new episode pop into my podcatcher each week."
All right, retrospectives. I love this question because I've definitely been part of teams that are really struggling to have a productive retro. So I think it would be helpful, as Jonathan highlighted, to go ahead and share how thoughtbot runs a retro. And then I'd also love to touch on some of the areas where I have seen teams really struggle to have a productive retro. So with the thoughtbot format, there are really two questions that we focus on. The first question is, what went well? And this starts the meeting on a positive note, which can help people get engaged before then we move on to heavier topics like concerns and issues.
When we run a retro, we ask each person these two questions. So that first question, we go around to the room, and we say, "Hey, what went well for your week or for your last two weeks?" And then we document all of those positive things that people say. The next question is, "What concerns do you have, or what are you worried about?"And the goal here is to highlight issues early, which then gives us the chance to address them as they come up rather than waiting till an issue has grown out of control. And it's usually during the concerns portion that I often see retrospectives fall apart. The reason for that is hearing someone describing a concern is often something that can stir up a lot of emotions. And I know for me, it certainly triggers my instinct to where I really want to dive into that issue, and then I want to solve it.
But by reacting to a specific issue and then trying to solve that issue, I'm interrupting that retrospective flow to then focus on that issue. And we may not get to a bunch of other important issues that people had. So that's often where I see retrospectives fall apart. And the way to fix that is to then have the team consensus that hey, this is a space where everybody gets to air concern. We're going to go around the room, so everybody has a chance to speak. We're going to document it, but then we're going to move on and then come back to this later.
So when do we talk about concerns? So once everybody's had a chance to share their concern and that's been documented, during that process, you're often upvoting other concerns. So someone may bring up a concern that I also have as well. So when it's my turn to speak, I'll say, "I'd like to plus-one that particular concern," and then maybe add my own or just plus-one some of the others. So then, by the time that everybody's had a chance to speak, you already have an idea on…whoever's taking notes or if it's being ideally shared so the whole team can see. You can already see the concerns that most of the team is identifying with or that are the more popular concerns. So then, as a team, you can say, "Hey, we're going to focus on the top two concerns because that's really the amount of time that we have," and that way, we're focusing on concerns that impact the majority of the team. So at that point, then we can start talking about those specific issues and how we'd like to address them.
And then out of that conversation is then the next part of the retro format, our action items. And then action items are where we can capture the things that we would like to try during our next iteration of work until our next retro. This is our experiment area. So then we can say, "Yes, we'd like to try something different, or we'd really like to monitor how this goes."
And then one other fun thing that I typically include in retros are housekeeping. So then we can talk about time off, team celebratory events, anything like that that's helpful to highlight to the team.
That's a quick overview of how typically I myself run a retro. Chris, do you have anything you'd like to add or anything that I've missed?
CHRIS: No. I think that that mirrors pretty well the best retros that I've been a part of. There are a couple of things that I think I would add or emphasize in that. So one is foundationally, with a retro, what are we doing? What's the goal? And the goal with a retro is to identify and evolve our process. So identify where there are any bottlenecks or things that aren't working, and then ideally change things over time. I've been on many teams where just the same issues get brought up over and over in retro, and nothing changes. And that will just completely deflate the team. And so, if that is happening, that's a fundamental thing that we need to fix.
And I can totally understand folks being like, "Retro is awful. We just sit down and say the same things, and then nothing ever changes." If that's happening, we have to fix that at a more fundamental level. That is going to be more than a retro’s worth of effort. But ideally, retro is now this structured space each week, each iteration, whatever it is where we are discussing what's going on and ideally, slowly, incrementally making the process slightly better. In my experience, it's something that I really love because I come to associate it with stuff is going to get better now. That's what retro means. If that's not the feeling you have, then I totally get why you wouldn't want retro. But I promise that that can be a reality.
And then to touch on some of the particular procedural points, everything you said definitely maps. And I've found that structure works really well, but there's a lot of subtle things in that structure that I think are important to highlight. So one, going around the room and actually asking everyone individually for their thoughts, I find to be so useful because it's very easy for one or two more vocal individuals to just dominate the conversation. So particularly by starting with what went well and then also by actually going around the room and requesting "Everyone reply to this question please," even if it's just like, "Yeah, you know what? It just felt like a good week." That's an answer we'll accept but ideally, a little more structure or a little more meat to it. But I find that to be really important.
Likewise, I have found that having a facilitator, so someone who is guiding the retro but not actually a part of it. They're not going to be saying what went well or what didn't go well. They are just directing the conversation and somewhat critical as you're going around and asking for concerns. They are the person whose job it is to prevent the team from starting to try and address the concern when it's first voiced. So ideally, we're just collecting the concerns. We're collecting the plus-ones so that we know which are the more prominent ones, and then we can focus on those.
And I think that idea of the plus-oneing of concerns and then really focusing on the ones that have more folks that are concerned about it feels really critical in my mind. So ideally, we are a team. We're working as a team, and if one person has this gripe that they really feel deeply, but nobody else really cares about it; ideally, we find a way to help that person not feel that way. But that's not necessarily where the team collectively should put all of their energy. So yeah, that's a bunch of little pieces.
Also, just as a note, we'll include these in the show notes, but there are a couple of previous episodes, so Episode 132: “What Went Well?” is a discussion between Derek and Sage, previous hosts of the show, talking about retros. Episode 172: “What I Believe About Software” was the first guest visit by a certain Steph Viccari. And so that is a wonderful episode in which we dug into retro because it's one of our favorite topics. Also, Episode 299: “Is Agile Over?” We definitely touched on...that was a pretty recent one, but we touched on retro. Then there's also a video on Upcase called “Running a Retrospective” that basically describes exactly this process and shows actually an example retro and running through it. So there are lots of other things that we can point out here. But again, I think fundamentally, what are we doing, and how are we doing it? If we can answer those questions well, retro is going to be great. If not, it's probably not going to be that great.
STEPH: I appreciate you calling out all of those important nuances because those nuances are what lead to then a retro feeling more productive. And to address Jonathan's other question around if people are feeling timid to bring up an issue because they don't want to blame anyone, then I think to address that one; specifically, you have to come to retro with a WE mindset. And I think HBO accidentally sending a test email is a really good example of that. Because in the Twitter thread, a bunch of other I presume developers were commenting and responding in support of the person that sent that out to say, "Hey, you discovered a missing safety net in the system," or the fact that it was fairly easy to make this mistake and send it out. So if you come to retro with this mindset of if a mistake was made, how can we as a team improve this so then it's less easy to make that mistake? Then you won't have the sense of we're blaming this on one person, but instead, we as a team are responsible for helping each other out.
CHRIS: It's interesting to have that conversation in the context of retro because I don't necessarily think of retro in exactly this way. But there is the idea of blameless postmortems, which come out of the Google Site Reliability Engineering; I think it's a book, maybe it's a website. We can include a link regardless. But that idea of blameless postmortems of collectively as a team, this thing made it out into the world, this bug, this problem. So we need to own that as a team, and we need to have a blameless conversation around that, just talking about what happened. And there are subtleties there. And that's a nuanced idea that needs to be evolved, but that is at least some writing that exists in the world that talks specifically to that part of it. That said, I wonder if a true postmortem, so a distinct meeting just dedicated to those more pointed issues, might be more relevant, and then retro is more of a shared overall conversation. But if there are smaller versions of that, then I think using that framing could be really helpful in retro.
STEPH: Yeah, I think you said that perfectly where there needs to be team ownership over all of the issues that are being discussed. And I think there is one other very tricky area to navigate with having a productive retro. And I don't know of a better way to say this. But you have a grumpy goose on your team. You have someone who doesn't like retros, and they're going to be negative, and they're going to be vocal. And that is a hard one. I have been there before. And I often approach that situation by speaking with them specifically around what are your concerns with retro? Are you willing to at least buy-in and give this new format a chance? But you essentially need them to buy in or have leadership buy-in so then they know to follow suit as well that this is a team process that we're going to improve and work on together. And if you don't like it, then that's what retro is for. So how we can make this a better, more productive meeting? But just showing up and being grumpy isn't helpful. And then helping people who have been burned by retros overcome that negative reaction to retros is something that takes time.
CHRIS: Oh yeah. The grumpy goose just affects everything on the team. But definitely retro is one where I've seen that particularly pointed. I think in those cases, the best luck I've ever had is to, like you said, have a separate conversation but have the conversation at a higher level. So the question isn't about do we have retro or do we have it in this shape? The question is, do we think we are operating at our best? Do we think everything is going perfectly? And almost never will the answer be "Yeah, this is great. We have no bugs. We're moving as fast as we possibly can. Everyone is happy. No one is burnt out." And so if we get to an agreement that is like, well, yeah, sure, there are things that we could improve, then I think that's a toehold that we can then build on and say, "Okay, so how do you want to go about that?
I am fine to explore a different approach than retro as a meeting to continually improving and evolving our process. I'd love to know what thoughts you have, Mr. Goose." But if they don't have an alternative, retro is the most effective structure that I've found for this continuous feedback loop around the process. I'm very happy to find an alternative, but I critically think we need something like that. And so if they're going to be pushing back on retro specifically, then I'll bump up to the higher level and say, "Okay, how do you want to be improving our process? Let's try something else, but let's make sure we are asking the question of how do we improve our process and is that succeeding?" And also, stop being so grumpy. Come on, what are you doing?
STEPH: [chuckles] I recognize that approach so much because then it really gets to the heart of the purpose of retro whether it's actually called retro or how we handle it is not significant, but the fact that we together as a team can get together and discuss how to improve. That's really the important thing that we're after. And retro just happens to be the format that I use and really enjoy. But like you said, it's always open to each team's interpretation.
On that note, Jonathan, I hope this quick overview of the thoughtbot retro has been helpful. And we will also include some other links that also highlight how thoughtbot runs retros and some other discussions that we've had about retrospectives. But on that note, shall we wrap up?
CHRIS: Let's wrap up. The show notes for this episode can be found at bikeshed.fm.
STEPH: This show is produced and edited by Mandy Moore.
CHRIS: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review on iTunes, as it really helps other folks find the show.
STEPH: If you have any feedback for this or any of our other episodes, you can reach us @_bikeshed or reach me on Twitter @SViccari.
CHRIS: And I'm @christoomey.
STEPH: Or you can reach us at hosts@bikeshed.fm via email.
CHRIS: Thanks so much for listening to The Bike Shed, and we'll see you next week.
All: Bye.
Announcer: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.