How to Do Long Term Workflows

Video version on youtube: https://www.youtube.com/watch?v=ErO9Ujccwds

My conversation with Jaden Baptista introducing him to Temporal.

Transcript

[00:00:00] swyx: Hey everyone is Swyx. I've been gone for the past couple of days and you may or may not be wondering what's happened. I basically got a cold and at this conference, it's actually nice to have a conference cold. I guess, because that means that things are going back to normal, even the, not so great stuff about being meeting back in person. 

[00:00:22] But I have a cold and I did not feel great. And I think gets COVID. But anyway, I decided to take a little bit of a break, but I also, I'm not sure if you know, but the weekday topics are all done on the same day. And I tried to do this pattern of batching things in weekly themes and this week's theme. 

[00:00:43] Basically it was poorly chosen. I thought that I had enough to do some, some stuff on basically the how technologies get adopted or get their traction. But I just didn't I ended up not liking any of the other episodes or any other podcasts that I shortlisted for that particular topic. So I just ran out and I just didn't feel inspired. 

[00:01:06] I felt trapped in the format and didn't really know what to do with it. So I ended up not doing anything. I also had other work to catch up on, on top of the conference stuff. So that was all in my head recently. And in general, and wondering whether or not I'm going in the right direction. So if you do that, And I really am grateful. 

[00:01:26] Checking out this podcast, because it's like a weird experiment with no particular theme or direction apart from cause it's stuff that I like. If you are, if you have a strong opinion and if you particularly think that there's some ideas that I should just pursue further and you're just mad at me for not doing. 

[00:01:42] Now's the time to tell me, because I'm also a little bit direction as to where this podcast could go. That's it? I do think that there's a strong thesis for exploring or having a central place for exploring ideas. I am personally interested in and recent dictating my podcast appearances on other people's podcasts to my own feed. 

[00:02:03] So people who are interested in what I do can follow up directly. So here's a conversation that I had with Jen Battista on Twitter spaces that was recorded. And it's about Tim Poro. So this is the first of maybe. Three podcasts that I did in the last month or so. So I'll be dripping them out over the weekends the next couple of weeks. 

[00:02:23] And we'll see, we'll see where this goes. I want to get back more into writing. I still have hopes of spinning out my YouTube more seriously, but I think the creative journey. The part-time creative journey of these, where I have a day job and that should take priority over the other side of stuff. 

[00:02:42] I do dictates the format of the things that I can take on. So, so far the podcast thing has been really great. I actually have a lot, a long backlog of things, which I selected for sure. They just don't fit in any, any particular theme. I think that theme is really nice when I can do it. And just, maybe you don't have enough backlog to do it just yet, so maybe I might go back to not having themes anyway. 

[00:03:06] So here's my conversation with Jane and review stuff. Well, thank you for  

[00:03:09] Jaden Baptista: joining me today, Sean. I really appreciate you taking the time are you doing today?  

[00:03:14] swyx: Very good. I'm very entertained by your Twitch title called temporal nutso temporary. Very interesting.  

[00:03:22] Jaden Baptista: Oh yeah. I was trying to come up with like a S a stupid clever name for the Twitch streams, despite them not really helping out with what the stream is usually ends up being about. 

[00:03:34] We tend to wander from topic to time.  

[00:03:37] swyx: Sure sure. Yeah. Thanks for having me in a happy to chat to portal. Awesome.  

[00:03:42] Jaden Baptista: Yeah. Well, let me ask you just the first quick question. You know, every, every big program that we all talk about w we really enjoy using was, was built to solve a problem, some sort of problem. 

[00:03:55] What problem was tempura both too soon, but what was the point of building it in the first.  

[00:03:59] swyx: So, to be clear, I did not build it. Who did exactly it was built to solve the problem of The abstract problem that the category of problem, this is called is workflows anything long running that needs to take anything more than a simple request response cycle, a request response will be just like, you know, you're paying a serverless function. 

[00:04:20] It gets back to you in, let's say 300 milliseconds, right? That's a typical cycle, but sometimes you need to do long running work. Typically I would think this is something like video projects. So, if you kick off a job, it takes like four hours to transfer code audio file. But actually it is both longer and shorter than that. 

[00:04:40] So, this is actually a topic of my recent blog posts. Because even if you, so Dropbox, so box is one of our users and they use us for file transfers and normally a file transfer just feels instantaneous until you try to transfer a million files. Right. They just change that. A thousand of them. 

[00:04:58] So you need a solution that scales pretty nicely from like a single transfer that should feel instantaneous to you know, something that's gonna take a while. Cause, cause it's just distributed across a lot of systems and you need it to be perfect or you lose data. And if your box you cannot lose data the other super long running task is why does anything have to ever end? 

[00:05:17] So what if you could just model the entire journey of your customer from. Like their first contact with you to the time that they unsubscribe. And that is a single entity that you just interact with. So it then becomes easy to say things like, okay, on their seventh visit, send them a coupon. Every month charged them based on their, you know, their, their usage or the billing. 

[00:05:38] And all of this is encapsulated in a single function. So  

[00:05:43] Jaden Baptista: yeah.  
 

[00:05:44] swyx: Yeah, because because we have an internal basically, so. Solves, it brings together a database, a scheduler some networking, some search capability and it, and it does. And then it uses, it offers you an SDK, so you can write it in idiomatic language. 

[00:06:01] The, there, this is not a new problem. Our founders have been working on this for something like 20 years. The tech leads for Amazon SQS. And then in simple workflow Azure doable functions and then when they arrived at Uber, they, they built the initial version of temporal where it now power is like, Something like 400 use cases at Uber mostly driver onboarding marketing which they call communications or something like that. 

[00:06:25] It's just like, whenever you do something that is so fundamentally asynchronous, you could use it for a lot of things. And it tends to grow that way. So, you know, it was open source that Uber and then it was adopted by other companies like Coi...