Exploring the Purpose and Effectiveness of Spiking Tasks in Software Development

Joël shares his recent experience with Turbo, a JavaScript framework that simplifies adding interactivity to websites without extensive JavaScript coding. Stephanie gives an update on her quest to work from her office more, and the birds have arrived—most notably, chickadees.

Stephanie and Joël address a listener question from Edward about the concept of a "spike" in software development. They discuss the nature of spikes, emphasizing that they are typically throwaway work aimed at learning and de-risking rather than producing final code, and explore how spikes can lead to better decision-making and prioritization in software development, especially in complex codebases.

Transcript:

STEPHANIE:  Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Stephanie Minn.

JOËL: And I'm Joël Quenneville. And together, we're here to share a bit of what we've learned along the way.

STEPHANIE: So, Joël, what's new in your world?

JOËL: I'm pretty excited because this week, I actually got to use a little bit of Turbo for the first time. Turbo is Rails'...I guess it's not technically just for Rails. It's a sort of unobtrusive JavaScript framework that allows you to build a lot of interactive functionality without actually having to write a lot of JavaScript yourself, just by writing some HTML in a certain way. And you can add a lot of functionality and interactivity to your site without having to drop to custom writing some JavaScript.

STEPHANIE: Cool. Yeah, that is exciting. I personally have not gotten to use too much of it in a production/client setting; only played around with it a little bit on my own to keep up with what's new and just kind of reading about how other people are excited to use it. So, what are your first impressions so far?

JOËL: It's pretty nice. It, you know, works as advertised. My situation, I was rendering a calendar view of a lot of events, and this is completely server-rendered. And I realized, wait a minute, there are some days where I've got, like, 20 events, and I really, like, I want my calendar squares to say sort of equally sized. So, I wanted to limit myself to only showing four or five events per calendar day.

And so, I added a little link at the bottom of the calendar day that says, you know, "See more." And when you click that link, it does some Turbo stuff, and it pulls in other events so that you can now sort of expand it to get the whole day. So, it's just a little bit of interactivity that you kind of get for free with Turbo just by wrapping a particular HTML tag around it and having the Turbo library loaded.

STEPHANIE: That's cool. I'm excited to try it out next time I'm working on a Rails project that just needs a little bit of that interactivity, you know, just to make that experience a little bit richer. And it seems like a really good, like, low-effort way to add some of those enhancements. Based on what you described, it sounds really easy.

JOËL: Yeah, I was impressed with just how low effort it all was, which is what you want, right? It works out of the box. So, for anyone who's kind of curious about it, Turbo Frames is the little bit that I used, and it worked really well.

Oh, something I'm actually excited about it as well; it plays nicely with clients that have disabled JavaScript. So, this link that I click to pull in the rest of the events, if somebody has JavaScript disabled, or if they command-click or control-click to open in a new tab, it doesn't just do nothing like it would often do in many sort of front-end framework-y places that have hijacked the URL click handler. Instead, it actually opens up the full list of items in a new page, just as if you'd clicked a normal link.

So, it really gives you that progressive enhancement feel where I can click a link, and it goes to another page with a list of all the 30 events if I don't have JavaScript. But if I do, maybe I get a slightly better experience where, instead of taking me to a new page, it just expands the list, and I get to see the full list. So, it plays nicely on both sides.

STEPHANIE: That's really cool. As someone who's just starting to dabble in some alternative browsers outside of the main popular ones [chuckles], I have noticed how many websites do not work for me anymore [laughs]. And that sounds, like, nice from a user perspective.

JOËL: So, other than dabbling with the new browsers, what's new in your world?

STEPHANIE: A few weeks ago, I talked about [laughs] sitting more at my desk and, you know, various incentives that I gave myself to do that. And I'd like to say that I've been doing a pretty good job [laughs]. So, what's new in my world is that I've followed up on my commitment to sit at my desk more, feel a little bit more organized in my workday. And that's especially true because the birds have finally discovered my bird feeder [laughs].

JOËL: Oh, that's really cool.

STEPHANIE: There were a few weeks where I was not really getting any visitors, and, you know, I was just like, when are they going to come and eat this delicious birdseed that I've [laughs] put out for them? And it seems like a flock of chickadees that normally like to hang out on the apple tree in my backyard have figured out this new source of food, and they'll sometimes, five of them at a time, will come, and sometimes they even fight [laughs] to get on the ledge to hang out at the bird feeder.

And yeah, it turns out that the six pounds of bird feed that I bought, I'll start to turn through [laughs] that a little bit quicker now, so I'm excited about that and just to also see other birds and species come and go as time goes on. So, that's been an exciting new development.

JOËL: So, the six pounds of birdseed might not last you through the winter.

STEPHANIE: I was debating between six pounds and, like, a 20-pound bag [laughs], which that would have been a lot. And so far, I think the six pounds has been serving me well. We'll see how long it lasts, but yeah, it's finally starting. I might have to refill it soon, so, you know, I was hopefully not going to have to store all that bird feed [laughs] just, like, in my house for a long time.

JOËL: Any birds that have shown up that have been particularly fun to watch or that are maybe your favorites?

STEPHANIE: I mentioned the chickadees because they seem to come as a group, and I really like watching them interact with each other. It's just kind of like bird TV, you know, it's not just a single bird. It's just watching these animals that are a collective do their thing. And I've been enjoying that a lot.

JOËL: Now I'm just imagining a reality TV but the Chickadee edition.

STEPHANIE: Oh yeah, definitely. I know some people put, like, cameras at their bird feeders to either live stream, which is funny because most of the time, there's nothing happening [laughs]. Usually, the birds are really in and out. Or they'll have, like, a really fancy camera to take, like, really beautiful up-close photos.

There's a blog that I discovered recently where someone posts about the birds that visit them at their place in Michigan. I'll link to it in the show notes, but it's really cool to see these, like, up-close and personal photos of basically the bird's mouth. Sometimes, they're open [laughs], so you can see right in them. I don't know; maybe there's a time where I'll get so into it that I'll create my own bird feeder blog.

JOËL: Well, if you do, you should definitely share it with the listeners on the podcast. Speaking of listeners on the podcast, we've recently had a listener question from Edward that I thought was a really interesting topic, and I wanted to take a whole episode to dig into.

And Edward asks about the concept of a spike. Sometimes, we're asked to investigate a complex new feature, and you might want to do some evaluation on the feasibility and complexity and build out just enough of it to make a well-informed opinion. And ideally, you're doing that in a way that reduces risk of spending too much time with unproven impact.

The problem is that in any reasonably complex codebase, that investigation work can be most of the work needed to build the feature. And Edward gives an example: if you're adding a system admin role, the core of the work is adding a new role with all of the abilities, but the real work is ensuring that it interacts with the entire system in the appropriate way. So, how do you manage making sure that you're doing spikes well?

And Edward asks if this is something that we've experienced a sort of feeling that we're doing 90% of the work in the spike. He also asks, does this say something about the codebase that you're working on? If it's hard to spike in it, does that say something about the underlying codebase, or are we just all doing spikes wrong? So yeah, I'm curious, Stephanie: do you occasionally spike things out in code on your projects?

STEPHANIE: Yeah, I do. I think one piece that was left a little bit unsaid is that I think spiking usually comes up when the team can't really estimate how long a task will take, you know, assuming that you use estimates on your team [chuckles]. That calls for a spike ticket, right? And someone will spend some time. And I think on some teams, this is usually time-boxed as well to maybe do a proof of concept or, yeah, do some of that initial exploration.

JOËL: Before we go too deep, I think it's probably useful to define spike in that I think it's a little bit easy and probably varies from team to team and even from a developer to developer. I think, for me, when I think of a spike, it's throwaway work. The code that I write will not get shipped, and this is not code that will just get improved later. It is entirely throwaway work. And the purpose of it is to learn something about the project that's being done.

Typically, it's in a sort of de-risking fashion, so to say, look, we've got a feature that's got a lot of unknowns in it. And if we commit to it right now or we start investing time into it, it could become a bit of a time pit. Let's try to answer some questions about it. Let's try to resolve some of those unknowns so that we can better make decisions around maybe estimation, but maybe even just prioritization. If this seems like something that would be really challenging to do, maybe we don't want to prioritize it this quarter. Is that similar to how you think of spikes, or do you have a different sort of definition of it?

STEPHANIE: Yeah, I am glad you mentioned that it's throwaway work. I think I was a little hesitant to commit to that definition with conviction because even based off of what Edward was saying, there's kind of, like, maybe different ideas about that or different expectations. But I sometimes think that, depending, spiking doesn't even necessarily need to lead to code. Like, it could just be answering questions. And so, at the same time, I think it is, I like what you said, work that helps you learn more about the system, whether or not there's some code written as, like, a potential path at the end of it.

JOËL: Interesting. So, you would put some things that don't involve code at all in the spike bucket.

STEPHANIE: I think there have been times where I've done a spike, and I've not coded out anything, but I've answered some questions, and I've left comments about unearthing some of the uncertainty that led us to want to explore the idea in the first place. Then, again, I also have gone down the path of, like, trying out a solution and maybe even multiple and then evaluating afterwards which ones I think were more suitable. So, it could mean both. I think that is actually something that's within the power of whoever is assigned this work to determine whatever is valuable to them in order to get enough information to figure out how you want to move forward.

JOËL: Another element of spikes that I think is often implied is that because this is throwaway work, you're not necessarily putting in all the work to make everything sort of clean, or well-structured, or reusable, or anything like that. So, it's quite possible that you would not even test this. You might not break this out into objects in the way that you would if this had to be reused. You might have duplication all over, and that's okay because the purpose of this code is not to be sort of production-grade; it's to answer some questions, and then you're going to throw it away and, using those answers, build something correctly.

STEPHANIE: Yeah, I think that's true. And it's kind of an interesting distinction from, you know, what you might consider your regular work in which the expectation is that it will be shipped [chuckles]. And there's also some amount of conflating the two, I think because if, you know, you and I are saying like, yeah, like, this exploration should be standalone, and it is not going to be used to be built on top of necessarily, there is some amount of revisiting. And you're not starting from scratch because you have an idea, but you are starting fresh if you will.

And so, you know, when you are doing that spiking, I think it allows you to move a little bit faster, but that doesn't mean that the work is, like, any X percent [laughs] done at the end of it.

JOËL: The work is still kind of, I guess, 0% done, again, because this is throwaway code, in our definition of a spike anyway. Would you distinguish between the terms spike and prototype?

STEPHANIE: Oh, interesting. My initial reaction is that a prototype would then be user-tested [laughs] in some way. Like, the point is to then show someone and then get them interacting with it, any initial reactions from that. Whereas a spike is really for the developer and maybe the team to discuss.

JOËL: I like that distinction. I definitely think that a spike, for me, is purely technical. We're not spiking out a feature by putting a thing live in production behind a feature flag, showing it to 10% of users, and seeing how they respond to that. That's not a spike. So, I think something a little bit more like that, or where you're showing things maybe to users, or you're wanting to do maybe some user testing with something. And it can be throwaway code still. I think now you're starting to get something more that you would call a prototype. So, I like that distinction of, is this sort of internal or external?

But in the way they're used, they can often be similar, and that oftentimes both will sort of...they're built to be as cheap as possible to answer the questions you're trying to get answered, whether that's from a user or just technical reasons. And so, the whole thing can be a little bit of smoke and mirrors, a little bit of duct tape and toothpicks, as long as you only have...like, the only solid parts you need are the parts that are going to help you answer your question. And so, any hack or cheat you can get to to bypass everything else is time you've saved, and that's a good thing.

STEPHANIE: Oh, I'm very curious about this idea of time saved because I think sometimes an underappreciated outcome of a spike is what not to do or is choosing not to do something. And it can feel not great to have spent hours or even days exploring a path just to realize that it's not worth it. I'm curious, like, when you know to stop and also, how you get other people kind of onboard that even just figuring out an initial idea was not a viable solution, how that could be a valuable insight to the rest of the team.

JOËL: Something that I think can be really useful is before you even start spiking out something, write a list of questions that you're trying to answer with this code, and then don't let yourself get distracted. Write the minimum amount of code that will allow you to answer those questions. So, maybe that is a question around, is it possible to connect this external API to our systems? There are some questions around, like, how credentials and things will work or how complex that will be.

It might be a question around, like, maybe there's even, like, a performance thing. We want to talk to an external system and, you know, the responses back need to be within a certain amount of time. Otherwise, this whole approach where we're going to try to fetch data live is not feasible. So, the answer we need there is, can we do it live, or do we need to consider some sort of background fetching, or caching approach, or something like that? So, write the minimum amount of code that it would take to do that.

And maybe the minimum amount of code, like you said, is not even really code. Maybe it's a script or even just trying out some curl commands and timing them at the command line. It could be a lot of things. But I think having a list of questions up front really helps you focus on the purpose of the spike.

And I think it helps me a little bit as well with emotional attachment in that success is not necessarily coming to a yes on all of those questions. It is having an answer, going from question mark to some answer. So, if I can answer that question, if I can find even a clever way to answer that question faster, that is success. I have done a good job with my spike.

STEPHANIE: I like that a lot. I think some people might struggle with spikes because they're so ambiguous. And if it's just, like, explore this potential feature, or, like, maybe not even that, but even saying, like, we want to build this admin role, to use Edward's example. And to constrain it to how should we do that, it already kind of guides the spike in a certain direction that may or may not be exactly what you're looking for. And so, there's some value in figuring out what questions to ask with the product team, even to get alignment on what the purpose of this task is.

And, you know, this is true of regular feature work, too. When those decisions have kind of already been made about what we're working on without a lot of input from developers who will be working on it, it can be really hard to, like, go back and be like, "Oh, actually, that's not really possible." But if the questions are like, "Is this possible?" or like, what it costs to do this, I think it prevents some of that friction and misalignment that might be had when the outcome of a spike turns out to be maybe not what someone wants to hear.

JOËL: And I think the questions you ask don't necessarily have to be yes or no questions. They could be some sort of list, right? It could be, look, we're looking at two different implementations or two general approaches, families of solutions for our super admin role. What are the trade-offs of each?

And so, a spike might be exploring. Can we come up with a list of pros and cons for each approach? And maybe some of them we just know from experience at developing, but maybe some of them might involve actually doing a little bit of work to play out the pros and cons. Maybe that's in our app. Maybe that's even spinning up a little app on the side, right? If we're comparing maybe two gems or something like that to see how we feel about throwing a few different scenarios and exploring edge cases. So, the questions don't need to be straight-up yes or no.

So, you mentioned earlier the idea that sometimes one developer might do the spike, and then another one might do the actual work, maybe inspired by the answers that were on that spike. And I think that can lead to some really interesting dynamics, especially if the developer who did the first spike has done kind of, like, what Edward describes, what feels like 90% of the feature.

It may be not so great code quality. And then this is a branch on GitHub, and they're like, "Okay, do the rest. Make it good. I've already explored the possibilities here," and then you're the developer who has to pick that up. Have you ever experienced that? And if so, how do you feel picking up a ticket like that?

STEPHANIE: Yeah, I have experienced it, and I think there is always something lost when that happens when you are not the person who did the research. And then having to just go from whatever was left in the notes or from the code and, you know, I don't know how feasible it is for whoever spiked to always be doing the implementing, but I certainly end up having a lot of questions, I think. Like, you can't document or even code out, like, every single thing you learned in that process, right? There's always from big to small decisions or alternatives considered that won't make it into however that communication or expression or knowledge transfer happens.

And I think the two choices that I have as a developer picking that up is either to just trust [laughs] that the work the other person did is taking me down a good path or to spend more time rebuilding some of that context and making some of my own evaluations along the way and deciding for myself whether I'm like, oh yeah, this is a good idea, or maybe, like, I might change something here. So, I think that there is some time lost, too. And I think that's a really good thing to point out when someone might think like, oh, this is mostly done. That's kind of my first reaction in terms of the context loss in an exchange like that.

JOËL: Do you feel like this is a situation where you would want to have the same developer do both the spike and the final implementation? Or is this maybe a situation where spikes aren't being done correctly, and maybe a branch with some code that's kind of half-written is maybe the wrong artifact to hand off from one developer to the other?

STEPHANIE: Oh, that's really interesting about if that's the wrong artifact to hand off because it could be misleading. Maybe it's not always, and maybe there's some really great code that comes out of it if someone builds on top of a work-in-progress branch or a spike branch.

Honestly, I think, and I haven't even really gotten to experience this all too much because maybe there is some perception that it's backtracking or, you know, it's more work or more time, but it would be really cool for whoever had spiked it to then bring someone along to pair on it and start fresh, like we mentioned, where they're kind of coming to each decision to be made with an idea, but it's not necessarily set in stone, right? There could be that discussion. It could be, like, a generative experience to either refine that code that had initially been spiked out or discover new things along the way. It's not like the outcome has already been decided because of the spike. It is information, and that's that.

JOËL: And we on this podcast are very pro-discovering new things along the way. I think sometimes as a developer, if I get sort of a, you know, maybe a 90% branch done that's get passed on to me from somebody else who did a spike, it feels a little bit like the finish the rest of the owl meme, except that now I'm not even, like, just trying to follow a tutorial. Just somebody did the first couple of circles and then is like, "Oh yeah, you finish the rest of the owl. I did the hard work. You just need to polish it up."

On the one hand, it's like, dude, if you're, like, doing 90%, you may as well finish it. I don't want to just be polishing somebody else's work. And, you know, oftentimes, it might feel like it's done 90% of the time, but it's actually, like, there's a lot of edge cases and nuance that have not been handled. And, you know, a spike is meant to be throwaway work to start with. So, I felt like those sorts of handoffs often, I don't know, they don't sit with me well.

STEPHANIE: Yeah. You could also come in and be like, this doesn't even look like an owl at all [laughs].

JOËL: I feel like maybe in my ideal world, a branch with partly written code is, I guess, an intermediate artifact that might be useful to show. But what I really want from a spike is answers to questions that will allow me, when I build the thing from scratch to make intelligent decisions.

So, probably what I want out of a spike is something that's closer to documentation, a list of questions that we were asking, and then the answers we came to by doing the spike work. And that might be maybe a list of trade-offs, or maybe we didn't really know the correct endpoints from this undocumented API, and we tried some stuff, and we, like, figured out what endpoints we needed, or what the shape of the JSON payload needed to be, things like that.

Maybe we tried a couple of different implementations, or we did some exploration around, like, what gem we'd like to use, and we have a recommendation for a gem. Those are all, I think, very concrete outcomes from a spike that I can then use when I'm building it from scratch. And I'm not just, like, branching off your branch or having it open in another browser and copy-pasting snippets while trying to, like, add some testing and maybe modularizing it a little bit.

I think that leads to probably a better outcome for the person who's doing the spike because they have a tighter scope and also a better outcome for me, who's then trying to build that feature correctly from the ground up. I think that would be my sort of ideal workflow.

STEPHANIE: While you were saying that, I thought about how a lot of those points sounded like requirements for a feature. And that, I think, is also a good outcome when a spike then leads to more concrete requirements because those are all decisions that were thought through, right? And even better is if that also documents things that were tried and the trade-offs that came with them or, like, the reasons why they were less viable or not ideal for that added context because that is also work that happened [laughs] and should be captured so someone can know that that might not be time they need to spend on that.

I am really interested in one piece that we haven't quite touched on is the complexity of the app and what it means for spiking to be a challenge because of the complexity of the app.

JOËL: Yeah. And I think sort of inherent in there is that maybe the idea that if you have a really complex app, it sort of forces you to go to the 90% of the work done in order to successfully answer the questions you wanted to answer with your spike in a way that maybe a better-structured app would not. Do you think that's true?

STEPHANIE: Well, I actually think that if the app is complex, you're actually seeing that affect all parts of feature development, not just spiking, where everything takes longer [laughs] because you maybe feel less confident. You're nervous about breaking something. Edward called the real work ensuring that it interacts with the entire system correctly, and that's true of, I think, just software development in general. And so, I wonder if, you know, spiking happens to be one way that it manifests, but if there are signals that it's affecting, you know, all parts of your workflow.

JOËL: There definitely is a cost, right? Complex software imposes costs everywhere. In some way, I think maybe spiking is attempting to get around some of those, in that there are some decisions that we can just say, you know what? We'll build the feature, and we'll just kind of figure it out as we go along, and we'll, like, build the thing.

Spiking attempts to say, look, let's not build the whole thing. Let's fake out a bunch of parts because, really, we have a big question that we want to answer about a thing that is three steps down, you know. And maybe the question is, look, we're trying to build the super admin role, and we know it's got all these, like, edge cases we need to deal with. Maybe we need a list of the edge cases, and maybe that's how we, like, try to drive them out.

But maybe this is a, hey, do we want to go with more of a, like, a role hierarchy inheritance-based approach, or do we want to go with some sort of escalating defaults? Or whatever the couple of different strategies you might want to do. And the spike might be trying to answer the question, how can we, as cheaply as possible while doing the minimum amount of work, sort of explore which of these implementations works best? And in a complex system, is it possible to get to the answer to those questions without building out 90% of the feature itself? I think, going to what you said, you might have to do more work if it's a complex system.

But I would also encourage everyone to go absolute minimalist, like, keep your goal in mind: what is the question you're trying to answer? And then ruthlessly cut everything you need to get to your point where you can answer that question. Do you need to hard code? Do you need to metaprogram? Do you need to do just, like, the worst, dirtiest code that you've ever written? That's okay because, like, the implementation does not matter. The fact that you're not exercising the full system does not matter, as long as the part that you're trying to exercise and answer your questions does get used.

STEPHANIE: Yeah, I like that a lot. And I wonder if the impulse to want to spike something is coming out of nervousness about how complicated the ask is. And it's like, well, I don't want to tell you that it's going to take a long time because this app is extremely complex, and everything takes a long time. You know, it's like not wanting to face that hard question of either we need to just set our expectations that things take longer, or we need to make some kind of change to make that easier to work with. And that is a lot of thought and effort.

And so, it's kind of an answer to be like, well, like, let me spike this out and then see [laughs]. And so it may be a way to appease someone making a request for a feature. I don't know; I'm perhaps projecting a little bit here [chuckles]. But it could also be an important question to ask yourself if you find your team, like, needing to lean on spikes a lot because you just don't know.

JOËL: That's really interesting because I think that maybe connects to a recent episode we did on breaking features down into smaller chunks. Spikes can often manifest, or the need for a spike can often manifest when you've got a larger, less well-defined feature that you want to do. So, sometimes, breaking things into smaller pieces will help you have something that's a little bit more well-defined that you feel confident jumping into without doing a spike.

Or maybe the act of trying to split this sort of large, undefined task into smaller pieces will reveal questions that need to be answered and say, look, I don't know where the seam should be, where to split this task because I don't know the answer to this one question. If I could know the answer to this one question, I would know where to split this feature. That's your spike right there.

Do the minimum amount of code to answer that one question, and then you can split your feature and confidently work on the two smaller pieces. And I think that's a win for everyone.

STEPHANIE: Yeah. And you can listen back to our vertical slice episode [laughs] for some inspiration on that.

JOËL: On that note, shall we wrap up?

STEPHANIE: Let's wrap up. Show notes for this episode can be found at bikeshed.fm.

JOËL: This show has been produced and edited by Mandy Moore.

STEPHANIE: If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show.

JOËL: If you have any feedback for this or any of our other episodes, you can reach us @_bikeshed, or you can reach me @joelquen on Twitter.

STEPHANIE: Or reach both of us at hosts@bikeshed.fm via email.

JOËL: Thanks so much for listening to The Bike Shed, and we'll see you next week.

ALL: Byeeeeeeeee!!!!!!!

AD:

Did you know thoughtbot has a referral program? If you introduce us to someone looking for a design or development partner, we will compensate you if they decide to work with us.

More info on our website at: tbot.io/referral. Or you can email us at referrals@thoughtbot.com with any questions.

Support The Bike Shed