Pete is the senior security researcher at Brave Software and the co-chair of the W3C Privacy Interest Group.We discuss:The differences between academic research and product developmentHow websites track us with cookies, fingerprinting, and other techniquesThe surprising amount of data your browser gives up without a permission promptWhat features should go in the browser vs being native onlyThe confusion behind what incognito or private modes doThe role of EasyList and the underappreciated people behind itBuilding tools like PageGraph to identify and rewrite tracking codeReplacing resources at page load to preserve privacy without breaking websitesDeveloping web standards at the W3C while preserving privacyBrave's plan to fund websites with ads instead of paywalls while preserving user privacyGetting involved in privacy and web standardsRelated Links@pes10kPersonal SitePrivacy Interest Group (PING)W3C Privacy Community GroupResearch at BraveSpeedReader: Fast and Private Reader Mode for the WebDetecting Filter List Evasion With Event-Loop-TurnGranularity JavaScript SignaturesEasyListuBlock OriginPanopticlickProtecting Against HSTS Abuse (WebKit)What’s the Difference Between First-Party and Third-Party Cookies?Understanding Redirection-Based TrackingMediaDevices.enumerateDevices() (Specifications give server all devices and labels without asking for permission)CSS difficulty with adoption (CSS2 spec was written without an implementation causing problems)What Are CSS Vendor or Browser Prefixes?FPRandom: Randomizing core browser objects to breakadvanced device fingerprinting techniquesBrave Fingerprinting Protections v2: Farbling for Greater GoodPageGraphPuppeteerTarget: You Can’t Hide That Baby Bump From UsBrave Ads FAQLUMAscapeMusic by Crystal Cola: 12:30 AM / OrionTranscriptYou can help edit this transcript on GitHub.Jeremy: [00:00:00] In this episode of software sessions, I'm talking to Pete Snyder about the many ways websites track us. How ad blockers like uBlock Origin work. And the process of developing web standards with privacy in mind. We start by discussing his role as a senior privacy researcher at Brave software Pete: [00:00:18] Brave is kind of interesting or unique as a startup in that we have a proper research lab. I think our research team is seven or eight people right now. Those are people who do research in the form of published publications but also doing research that ties back into product in some way.My research responsibilities are to figure out new ways that you can improve browser privacy, address tracking on the web, and solve the kinds of problems that Brave is interested in solving. I have one foot in engineering world and one foot in publishing world.Jeremy: [00:00:48] Why is academic research important in this space?Pete: [00:00:52] My gut feeling is that what's useful about academic research is that it changes the incentives and it gives you a chance to do things that are more novel and particularly things that are less tied to a short term ROI cycle. That is particularly useful for things that have watchdog functions over industry, things that are more difficult to monetize but more useful to average web users.That's not to say there aren't people who try to build businesses around privacy or responsible computing but the incentives don't always work that way. What's really neat about doing a research focused computing career is you can do things that don't have to make somebody money in the short term. You can pick more oddball projects. The things that might not come to fruition right away.Jeremy: [00:01:36] And is there a key difference in how you approach a problem when you're doing it in an academic context versus as a product for a company?Pete: [00:01:46] Sure. So they go both ways. If I'm working for something at Brave the emphasis is on correctness and certainty. And knowing that when we ship it to 10 million people or whatever that it's not going to break and it's going to do what it says on the tin and that it's going to be a material improvement over the state of things before we ship that feature.And that's really different than if you're trying to come up with a research project where.. sometimes good, sometimes bad, but the emphasis is not necessarily on a hundred percent correctness but is on novelty and doing something or figuring out some way to solve a problem in a way that it hasn't been tackled before.And so you'll read research papers that say it works 95% of the time and that'll be sufficient or compelling for a research paper. But you wouldn't want to ship something that breaks 1 out of 20 websites if you're actually making a product. The goals are different, but also the success criteria are different.Jeremy: [00:02:39] So it sounds like you can tackle things where it wouldn't be good enough for a product yet. But it's something that if you were working on it within the context of a company, they might say: Oh, we're not going to do that because it just doesn't seem like it's going to work.Pete: [00:02:54] Yeah, exactly. So, maybe because certainty of success isn't there, or there isn't a one or two step obvious path to being a product. Maybe it conflicts with the current business goal or whatever else. But yeah, you have much more latitude in terms of products you can choose and kind of problems you want to tackle.If you're writing research papers and not that I'm some incredible researcher or anything, but if you try to do successful research it doesn't reward you to solve that final 5% of the problem.There's no benefit to getting no, not none, but there's a small benefit of going from 95% to 99% success or accuracy. On product you have to grind out as close to a hundred as you can get.Jeremy: [00:03:37] And do you have examples of things where you worked on it in a research context and it actually became a part of a product?Pete: [00:03:46] Sure. Yeah. So a couple of things. One is that.. so there's a research paper that we wrote at Brave called Speed Reader. Speed Reader is a different way of doing a reader mode in a browser. Right now, if you use any of the reader modes in popular browsers, you download the page, you render some subset of it.You throw some JavaScript at it and it extracts sections that it thinks are useful, then it presents a new page to you. That's not a hundred percent correct. Chrome's DOM distiller does something slightly different, but to approximation you render the page and then you extract stuff out of it. Brave speed reader does something different.It intercepts it at a network layer. It examines the text HTML. Does the analysis there and then feeds that back to the rendering engine. And so there's a bunch of nice benefits there. There's a privacy improvement in that you're executing less code talking with less third parties. There's an performance improvement as well in that you don't have to do the initial displaying and tear all that stuff down and build it back up. So that was the research paper that we published at. WWW 2018 2019. I don't remember, but either a year or two ago. And it's now in beta in Brave. That's maybe the oldest one. The most recent one was a project that I did last summer with a student from North Carolina, Quan Chen on figuring out ways that we can do blocking better.Right now, if you're using a privacy tool in a browser in most cases you're downloading a big list of things that should get blocked. They look kind of like regular expressions. It says: Yes, block this. No don't black that, and it's a useful thing.But it has the trade off of it's very easy to circumvent that. Somebody can just change the URL [and] move it to a different domain inline in the page, whatever else. And so the approach that we took from this paper is.. Let's not focus on the URL. Let's build signatures of the execution paths of these scripts and we can use that as the Oracle to identify this is known bad or known good. That machinery ended up being very complicated and it isn't something we want to ship to all of our users because of the performance hit. It's something that we use for generating filter lists that we should download to users regularly.Jeremy: [00:05:40] Existing projects you were saying are just looking at a list of URLs. And you said using something like regular expressions to figure out if the URL it's pulling is on that list. The part I wasn't clear about is the new way that you were describing worked.Pete: [00:05:57] Yeah. The alternative approach that we came up with is to instead.. Not care about where the code came from or even how the code is structured. So if it's obfuscated in some way, but instead to look at the DOM and JavaScript operations that the code executes and sequence those and use that as the identifying signature of the code.There's some cleverness in there that makes that particularly difficult to do in JavaScript versus other languages. But at a high level, it was saying let's identify things based on their behavior, not on their source.Jeremy: [00:06:28] And so would that be where the browser would have to load the script.. see how it would affect the DOM. And then based on that, you would determine whether or not this was something that was probably showing you an ad or trying to track you, that sort of thing?Pete: [00:06:43] Yes. The way the project works toe to tip is.. There's these long, long, long lists of things that people previously have identified as being tracking related or ad related those are things like EasyList and EasyPrivacy and the uBlock Origin lists and all this kind of stuff.And so you can throw those at the web and you get some kind of nice labeled data set of these things are tracking and ad related, these things are benign. So you can run those, execute those files or load those pages and get signatures of how that code operates in the page. And so now you have your ground truth signatures of this is what known bad code does. And this is what known good code does. Then you can run that stuff against a bunch of things that you don't know the labels of and you can rebuild those labels on top of this code that people have examined before. And so you can do a couple of things with that. You can either use that to build even more lists in an automated way. You can use it to do code rewriting since some parts are good and some parts are bad. You can use it for online blocking things like that.Jeremy: [00:07:38] You're basically looking at things that people have identified as being bad behavior or tracking behavior. And you can load things that you haven't seen before and use that to instead of having a human curate the list you could have your code load things that it hasn't seen before and figure out.. Oh, this looks like this thing that I've seen before that somebody said was bad. And so I'm going to make a new list based on that.Pete: [00:08:08] Yeah, exactly. For the Alexa 1000 or Alexa 10,000 like the most popular sites on the web, those have a lot of eyes on them. And so things that are tracking related get picked up pretty fast on those. But for the long, long, long tail of the web, that stuff is barely examined or at least, has a lot less eyes on it.And so this is a way that you can use the the code that people have looked at to identify code that fewer people have looked at.Jeremy: [00:08:32] On a broad sense, how deeply are people being tracked and do you think people are aware of just how deeply they're being tracked?Pete: [00:08:41] So in the first case unimaginably. The amount of web surveillance and offline surveillance that people undergo.. unimaginable. Large amount. And then the second case, very little. You'll find these tools like Brave or the new version of Safari or Adblock plus or any of these.. uBlock Origin.Good tools by people who are sincerely trying to reduce this stuff. And they'll put a little number in the URL bar and it'll say 10 trackers on this page or whatever. And you'll go to some new site and it'll have 95 or whatever. And that's just the known bad stuff. I think people have very little understanding of how atrocious the situation is.Jeremy: [00:09:24] And what are some of the ways that people are being tracked by their browser?Pete: [00:09:29] Oh. Well. In most cases, the tracking isn't being done by the browser. That's not necessarily the case. Chrome absolutely does observed things about what you're doing and sends it to the Google mothership. But in general the tracking isn't happening because of the browser itself.But rather by things the browser is loading because of things the web pages tell it to do so. So there's a whole long tail of extremely boring everybody understands kind of things that have been around for 20 years to like more weirdo stuff. By far still the most common method people are tracked is just I drop a cookie. I drop a cookie on one site. I fetch the same image on another site. And so the cookie gets resent and that's my way of learning the same person visited site A and site B. Web browsers like Safari and Brave will never send third party cookies with a very small number of exceptions. Firefox and Edge have a kind of complicated system for determining when they send third party cookies but do a good job of not sending them to the worst offenders. Then things get slightly more sophisticated.Instead of dropping cookies maybe what you'll do is throw storage into other places where people don't usually look for it. Right now there's at least four or five different APIs, six different APIs you can use to have persistent storage on the browser through JavaScript. Then there's a whole long tail of ways that things can get cached that also turn into persistent identifiers. That's maybe the second weirdest or second most understood. So then there's a whole bunch of places where the browser is implicitly keeping universal global state that you wouldn't necessarily think about as being a tracking vector, but anytime you have global state, you have the mechanism you need for tracking.And the most frustrating example is something called HSTS tracking or HSTS cookies. It's an abbreviation for a header that a website can send you that says, Always automatically upgrade this request to an encrypted version, to an HTTPS version, even if I requested over HTTP.Just in general what would happen is.. I make a request to some website I like, and it's going to be HTTPS. HSTS instructions are not respected over HTTP generally. But I make a request to a website I like it sends back this HSTS instruction that says good. Now that we have a secure conversation, I want you to never ever not communicate with me over a secure channel. So we got this one secure communication. We're going to use this as the kernel of trust to build the rest of our communication over.And so, that instructs me every time I visit the site again, automatically the browser will know add an S to the HTTPS. And the same thing is true for any sub requests or in general, until people started coming up with counter measures, the same thing is true of any sub requests as well.Jeremy: [00:11:59] If I understand correctly, you make a request to a URL and it tells your browser, in the future you try to go to this URL and you don't put in HTTPS to automatically go to HTTPS instead. And the part that I don't quite follow is how is that used to uniquely identify you?Pete: [00:12:19] Oh, okay. Step one is I make a request to your website it's a secure connection. Then on your website you have 26 different images. You know, an a b c d an e and so my browser will make new requests for each of those images. And those images in this configuration are each hosted on different sub domains on your website.A image off a.you.com that kind of thing. So now, if those are all requested over a secure channel, your server can decide I'm going to send the HSTS instruction or not for each request. So I'll get back to those images. And, I'll have, more or less 13 new HSTS instructions.And those will be different for me than they will be for you for anybody else. Just flipping a coin enough times that's like the setting, the identifier step. And then now a week later or whatever you want to identify me again. I clear my cookies and everything, so I think I'm not identifiable but I come back to your site and now you'll have me request the same 26 images, but over HTTP, not a secure channel. And now my browser will upgrade more or less 13 of those images. Your server will look to see which 13 images got upgraded. And now that'll be unique to me versus everybody else in the world.Jeremy: [00:13:28] I see. Wow. So it's a feature that had good intent, but the way that people are actually using it is they're building like a fingerprint right? They know for each URL which one they told you to upgrade to HTTPS and which ones not.And like you said, so even if you clear your cookies or whatever the URLs that should be upgraded based on HSTS they're still stored in your browser.Pete: [00:13:56] Yep. And so there's a long tail of these kinds of things where they were added to the web platform or to browsers or to the internet infrastructure for largely completely benign or really desirable reasons. But because of the way they've been implemented or because of the way clever people have misused them they become tracking vectors.HSTS is not at all the only one. It's just the one that is kind of the most galling because it's supposed to be helping people's security and ends up hurting their privacy or can hurt their privacy.Jeremy: [00:14:24] Right. So in the past you were talking about classic tracking that was making use of cookies. and that's something that gets stored in the user's browser. And my understanding is that for cookies in the past, you would go to a site and in order to track somebody while you're on that site you're making a request to another domain, right? To a tracking domain. And as you go from site to site, those other sites use a cookie from that same tracking domain. So that's what you consider a third party cookie. Is that correct?Pete: [00:15:00] Yeah. So I would go to your site. Your site is benign.com your site includes an image from tracker.com. I get the request back from tracker.com. It tells me to save a cookie from tracker.com. I go to some new website, it also requests tracker.com and now tracker.com can link those, those views.Jeremy: [00:15:16] And so now you were saying a lot of browsers like Brave and Safari and Firefox are starting to block third party cookies. What are some of the ways that sites are working around that?Pete: [00:15:30] So there's a long tail. Some people have been moving to these more esoteric tracking things. So HSTS tracking is one thing that people in the wild were doing particularly against Safari when Safari started blocking third party cookies. They've been moving to different types of identifiers in the browser.So maybe I don't store something in a third party cookie, but I set a cookie on the first party cookie jar. And then I just append that to my request. So things like Google analytics do that. That's called riding on the first party cookie jar because even though the code is executed by say Google analytics or any number of other tracking scripts. It's actually living in your cookie jar, the cookie jar is associated with your origin. So those are two and then it just kind of gets more oddball from there. So there's, browser fingerprinting, if you're familiar with this, which is ways of finding like things that are semi identifying or semi unique in your browser, then build up enough of them. Very quickly, I can identify a large number of people uniquely. It's like, guess who, and you just split the population in half enough times, so that's done extremely commonly on the web. It's very, very common. And then there's a new kind of thing called bounce tracking. Not, it's not that new but increasingly common kind of thing called bounce tracking where, Different browsers will only let you set third-party state if you visited them in the first party context. And so websites will play these games where they'll just forward you through a long number of first parties before you get to where you want to go.And now all these things can set third party cookies on in iFrames and things like that. I could go on and on. There's an endless number of ways these things are done, But getting rid of third party cookies is definitely an extremely helpful thing, but it's not the end all be all of web privacy.Jeremy: [00:17:05] One of the things you mentioned was browser fingerprinting what are some of the ways that, people's browsers get fingerprinted?Pete: [00:17:12] Sure. At a high level browser fingerprinting is looking for a bunch of things that are going to be different between people's browsers. They don't have to be unique to a single person, but there'll be minor configuration differences or subtleties that are different between my browser and your browser and somebody else's browser.For example, English is a very common language that's spoken on the web, and so if I know that's someone's language, maybe that identifies me. 1 out of 20 people speak English in the web or something like that. Yeah. Some number that's less than one or less than a hundred, I mean, a hundred percent.And so then now I've cut out a large portion of the web and now I look at and see, is this person running a Mac? That's going to also cut the search space down. And I look at this person, how many devices do they have plugged into their machine? That'll shrink it down further. What are the kinds of peculiarities of their graphics card when their different drawing operations does it draw a line in a slightly different way than on a different graphics card.That'll shrink it down further. Does this system have odd fonts installed. It'll shrink it down further, et cetera, et cetera, et cetera. And if you have enough of these kinds of things, you can pretty quickly put a lot of people in a bucket of size one.Jeremy: [00:18:20] And one of the things you mentioned is that you can identify what devices are plugged into your computer and maybe what your graphics card is. What are some things you think people would find surprising about how much your browser is actually sending to the server that you're visiting?Pete: [00:18:39] So it depends on the browser that you're using. But that's a good question. I expect people to be surprised that any webpage that you don't trust this is without a permission, websites can in some browsers enumerate all the devices you have installed, like the labels, the type, this kind of thing.They can learn about what kind of network connection you're on, whether you're plugged in. If you're in Chrome, the most popular browser you can learn about, the kinds of network errors the person is observing, which can be very identifying if you're moving between networks and they have different kinds of DNS configurations. If somebody has a two factor authentication device, like a hardware key, you can learn some things about the hardware key, like the strength of it.Even if you wouldn't expect a website can access that automatically. Those are just things the browser is intending for websites to be able to access. That's not like cleverness. That's just like, there is an API that will tell me specifically this. And then there's a long list of other things with a moderate amount of cleverness you can figure it out as well.Jeremy: [00:19:36] I think a lot of people are familiar with the fact that when they go to websites, it'll ask for permission to use a microphone or to use GPS, things like that. For some of these other things that you're referring to, that it's able to figure out such as your, your devices that are connected, things like that.Is that something where the person is giving permission or is that something that just just happens?Pete: [00:19:58] Yeah, so that, that just happens. So I represent Brave on the W3C and I co-chair, one of the privacy groups on the W3C, the horizontal review group that reviews specs for privacy. And that's something that we're working with the working group that authors that spec, the media and capture media streams capture group to improve that API.But that is just the way the API works in the standard right now. The website, says.. tell me about the devices the machine supports. It gets back a list of every device that the machine knows about that has like an AV component. And then the website now says, okay, I would like to access this one.And then you get the permission dialogue. But without any permission you can learn all the devices and all the labels and the categories and this sort of thing. I should say that the spec looks like it's getting much better. That working group has been really, it has been good to work with and it's been, been really receptive to those concerns, but that is that's the way it ships right now in Chrome, Edge, Brave makes some modifications to it. I don't know about the other ones, but that's the way the spec standard is written.Jeremy: [00:20:59] And that's interesting because you're saying the standard is currently being written but a lot of these different browsers, they already have an implementation of it.Pete: [00:21:09] Yeah. So this is another one of those good intentions that have turned out to have an unintended consequences kind of situations. So it used to be the case that you're running a web standard. A bunch of people would get together and work on the standard. And then when it was done, people were supposed to implement it.And then, this is a rough history. I'm sure I'm going to get some of the details wrong but to a rough approximation something like CSS2 happened where you ended up with a standard that was basically unimplementable and it had all these kinds of subtleties that hadn't actually been worked out because nobody had implemented it yet.There's other cases too and CSS2 might not have been the tipping point but it was definitely a famous example. Then there was a CSS21 standard that came out when people started implementing it, it had to get revised in certain ways to make it actually work in the world.Hand-waving simplification, but people thought this is not great. We need to like, actually build these things as we're talking about them, to make sure that they work in the world. And so then you got into this kind of like prefix situation where I don't know if you, if you do.If you're a web developer, you'll be familiar with like, until pretty recently, you had all these pre-fixed extensions like rounded corners, but WebKit rounded corners, and Microsoft rounded quarters and Mozilla Yeah. And you had similar things in, in the DOM.And there's still some hangover at where a bunch of specs in most browsers are implemented twice. Once WebKit prefix, things like that. And so understandably people thought this is not great. Now I have to write my code four different times. And so, right now, if you're trying to get a standard finished in the W3C, I'm less familiar with other standards organizations, but in the W3C, you need to have two working implementations, two independent implementations.They don't necessarily need to be shipping like unflagged like the way it's supposed to work in the best cases like they're running in the browser and there's some flag that you flip or it's only enabled for some set of websites, but that's not always the case. Things are getting shipped as they're being designed in the standards body absolutely is like a little bit begging the question right? And that if you find out there's a problem during review with the spec well, now a whole bunch of websites have already started depending on that certain functionality and have it baked in that is a real pickle and something that we fight with a lot during these reviews. But, yeah, that's, the less than ideal situation that things have come to at this point, I think it's getting better, but that's generally how things are done right now.Jeremy: [00:23:20] That's kind of interesting because it sounds like you have the W3C and you're planning on things that will go into the browser, but in order for them to become a standard, they need to already be in the browser. Which also means, like you said, that people are already using them. I wonder what is the negotiation or the back and forth in terms of, let's say Chrome is already using a certain feature and you say we'd like you to change this feature for this reason.They'll say well, we already have thousands of sites that are using this. how are we going to change this? What does that, that back and forth look like?Pete: [00:23:59] Yeah. So, the first thing to keep in mind is that the standards body is not a legal organization. The standards body can't make anybody do anything, they can say something's out of standard. They can remove it, but people listen to a standards body if they want to listen to a standards body and they don't, if they don't.So in that sense, a standards body works by trying to make it mutually beneficial for people to go along with it and providing resources that maybe an organization wouldn't have if they weren't in the standards body and some benefit in terms of interoperability, that sort of thing to strengthen the platform in general.So that being said, there's not an easy answer. Sometimes you can find clever games you can play with the way the existing APIs work that will reduce the amount of information exposed but without breaking the function signatures or the expected flow of the program. There's several different tiers of review in the W3C and some happen earlier than others.What we've been trying to do is push review earlier in the process to try to catch these things while they're still in prototype stage, instead of, web scale stage. But yeah, there's no way that the standards body can go to mr and mrs. Google and say, you must do a thing or, Mr. and Mrs. Apple or whatever else. Unfortunately that's just the case.Jeremy: [00:25:13] And you were saying, how you're the co-chair of the privacy interest group on the W3C. How much power would you say that that group has? Do you have examples of times where somebody has tried to push a feature through and you've rejected it on the basis of privacy and there's actually been changes made or the feature has been dropped all together?Pete: [00:25:39] I should say in terms of power we're a subset of the W3C and the W3C at the end of the day has, the organization has maybe some moral authority or some, you know, people respect it. And so there's some soft in quotation marks power there.And there's a lot of expertise that people respect in the W3C. And there's a lot of mutual interests between the browser vendors to have a web that is friendly for developers and friendly for users. So, there's no authority, but there is, web browsers are interested in sometimes what the W3C has because of these other reasons. So power is a funny word to use there through I take your point. To point out very specific changes that have been made. I'll talk about the WebRTC one the enumerate devices API, because that's the one we just mentioned before, so right now, partially because of interest among people in the working group, partially because of reviews that PING has done. PING is the privacy interest group. There's both immediate changes that look like they're going to go into the API.None of this is a hundred percent. This is all in the GitHub issues right now. But this is the direction it looks like things are going, best guess. That there'll be a number of ways of shrinking the amount of information that websites can access by default. So it's things like, without a permission, maybe you don't see this person has 18 different microphones.They just see there's at least one microphone. And then when the website asks for permission then they can learn about the number, the details, things like that. So that's a wonderful thing. I think the working group would agree that it's not the dream outcome, but it's dramatically better than what was there earlier.And then we're also working with the working group. They're doing great work in terms of figuring out like where can we go to that's even better? And so that looks like it'll be the website doesn't see anything by default. If the website wants to see devices, they call a thing, the browser prompts you with a list of devices.And then if you want to, that information gets passed. that's a example that comes to my mind. There's a long list of if it's of interest to your listeners too. I could also point you to the endless list of privacy issues that we raised and the back and forth that happens on them there.But sometimes they're very large things like that. Sometimes they're very small things like you're leaking fingerprinting bits here and let's figure out a way to sort through that.Jeremy: [00:27:44] One of the things I find interesting about your position is you work for brave, which is a browser vendor, and you have Microsoft with Edge. You've got Google with Chrome, Apple with Safari, and Mozilla with Firefox. And I would imagine that all of these different companies, they all have their own goals.They all have their own things that they want. I wonder from your perspective, what are the kinds of roles that each of these companies play and where do they butt heads and where are they on the same page?Pete: [00:28:19] I want to say things that I'm only very confident about. All these organizations and particularly the people that are sitting in these committees and these working groups that represent these organizations have an interest in the web.They see there's something unique about the web that that is appealing and desirable and positive. That's not on other platforms. They may not a hundred percent agree on what those positive things are, but there's something that appeals to us about the web that doesn't exist on other platforms.And so there's mutual interest there. I also think that all these people care about privacy, they care about making sure things are accessible to people with different needs on the web. They care about making sure APIs work well for people who speak different languages and come from different backgrounds and these sorts of things.At the end of the day, people who choose to spend their time in these long meetings working with each other.. We have very similar interests and we're all pushing the same way. Where they differ is the prioritization of those interests. Brave is absolutely like, we think there's something super duper wrong, like kind of fundamentally wrong.Yeah. Maybe that's too strong, but the web has really gone sideways and the privacy violations are endemic and really horrible. And like intolerable. I think other people would say, yes, privacy violations are bad, but also we want to make sure that we don't break the ecosystem that exists to fund the web as it exists today.And so that's like privacy is just one among many different interests, including making sure advertising dollars self fund websites and things like that. And then I think there's other people who exist in other parts of that, on that spectrum and have different interests.So I think we're all pushing the web in the same direction, are interested in making sure it flourishes, but what flourishing means probably differs between different people in different organizations.Jeremy: [00:29:58] Something that sometimes comes up and maybe it's a little more front of mind because Apple's worldwide developer conference happened recently is that people have a perception of Safari not implementing a lot of features that other browser vendors either implement or want to implement. I think a lot of times they say that they're doing it in in the name of privacy. And on the other hand, you have developers who are saying, Oh, we want all of these different features because we want to be able to build a progressive web applications. We want to be able to build a websites that are similar to apps.And I wonder from your perspective, how do you balance these two goals?Pete: [00:30:40] So I think that's a really interesting example you brought up for a couple of reasons. I bet we're thinking about the same tweet that went around and the same people blowing off steam, and I can totally understand their frustrations. But I should say two things first before going into the guts of your question.One is that, most of the things are not standards they are proposals. And so, as much as the web community, we like to treat them as standards because they're implemented in the popular browser. They are not standards, nobody's agreed to them, blah, blah, blah. They are proposals. Second thing is that, I think there was 16 or 17 or 18 different things on that list.I don't remember the full thing, but I remember looking through it and thinking Brave takes the additional step of removing these things from chromium before we ship Brave. I am completely sympathetic to the idea in the vast majority of those cases, maybe all those cases, I just don't remember the full list that those are really privacy risking features.And the permissions models around them are not well-defined. They haven't been well reviewed and the risk is really significant. Look, Apple's got more money than anybody knows what to do with Apple. Apple's not, not implementing cause they're lazy. They may be pursuing a different strategy.But I also know that the people in those committees have sincere strong, heartfelt interest in privacy. So I understand the frustration of the web community, but I find the privacy story there compelling.Jeremy: [00:31:58] And I think it's also maybe important to think about the fact that as soon as you put those into the browsers, it's going to be extremely difficult to remove them.Pete: [00:32:08] Yeah. I mean the web congeals around any features that gets there. And the moment you put something in it becomes extremely difficult to pull it out. Something that we deal with at Brave a lot, because we think that the way a lot of APIs work is inappropriate and intolerable and we have to be very clever in the kinds of ways we can modify behavior that websites already expect to exist in a certain way.Jeremy: [00:32:32] I think I know about the tweet you're referring to, and I don't remember all the specific features but, I wonder from your perspective, are these features that you think shouldn't exist or is it more that the way that people want to implement them now wouldn't be done in a privacy conscious way.Pete: [00:32:50] Hmm, that's a good question. So I also don't remember the full list, but I can pull off some examples. I think there's kind of three tiers. Some things just seemed like bad ideas that we should just not do, or at least not do without pretty fundamentally rethinking how they exist.Some of these things are things because they make more sense as operating system features or native app features than they do websites. And some of these things, are things that, yeah maybe those would actually be very useful on the web. If we could figure out how to do them responsibly.A lot of this stuff has its roots not in things that typical websites need to do, but like the union of a bunch of weird things that happened. One is like Firefox OS happened for awhile. And so a bunch of things got pushed into the web platform some of which got yanked out later, ChromeOS is another one, PWAs, things like this. And a lot of these things are really different from what we think about as websites. It's worth thinking about where are those lines and should they be firm or that sort of thing. A while ago, the example that sticks out of my head is there was a standard that got shipped in Gecko in Firefox and Chrome that allows websites to read the amount of ambient light in the room. The website could read it's very bright, it's very low, I don't remember the granularity, but any step in between. And of course the very first place this stuff got used was in tracking scripts to fingerprint people. Same with the battery API, there was an API that allowed websites to say, it's a full battery, low battery, that sort of thing.You can imagine why that'd be a nice feature in an app. But you can also imagine it gets sucked into the fingerprinting scripts immediately and starts harming and targeting people. And so yeah there's definitely a part of the web that says let's just permission prompt everything, or use a number of different kinds of proposals that concerned me to restrict this stuff or allow it on the web in a responsible way. The web as it is without adding more functionality has so many deep privacy issues. I feel very nervous about pushing for new functionality unless privacy is really treated as a first class citizen in those standards.Jeremy: [00:34:56] Yeah. And it sounds like where we're at now.. There's already so many different ways that you can be fingerprinted. And every time a new feature is added to the browser, it just gets more and more easy to track someone. Pete: [00:35:11] Yup. I think that's exactly right. And there's also cases where adding a new feature undoes a privacy protection somebody else has added in somewhere else. It's good to be very cautious before throwing new powerful features into the platform.Jeremy: [00:35:23] Another thing that you had mentioned when we first talked about doing this interview was you had said that Brave is based on chromium. And you said that you had a somewhat semi adversarial relationship with upstream chromium. I wonder if you could elaborate on that?Pete: [00:35:44] Sure that was kind of a silly in a goofball thing that way to put it. That's that misstates things too strongly. The chromium developers have been very receptive to questions that we have. And we've tried to upstream stuff we found to be a positive experience. But there are things where the vast majority of Chrome developers are Google employees. And of course Chromium is shipped in a lot of ways with Chrome in mind. I don't think it's a malicious thing, but it is the case and so there's a whole lot of stuff in the chromium code base that assumes Google. Which servers get talked to and account information stuff, and safe browsing things, and an incredibly long list of stuff that is just in the chromium code base but assumes Google including, and this is maybe this is what I had in mind when I said adversarial. Poor choice of words. There's a couple of features that Chrome ships that allow you to basically enable a feature only on certain origins and they call it field trials, this kind of thing.So if the chromium folks want to test out any feature they can say only these three or four or five or whatever partner websites can use it. Sometimes that feature gets shipped, they'll ship a feature ungated, not flagged. And then they'll use this feature to turn it off.So they'll ship some new experimental feature and then they'll say, but we're not gonna allow it on any sites. The field trials is zero or is empty. And so that's their way of making sure that sites don't get it. Well, if you're building a browser that wants to put firm lines between itself and Google data collection servers you don't get that information.And so now all of a sudden, the weirdo experimental feature is enabled globally in Chrome or in your version of Chromium. A long list of things like that. There are also other choices in the platform that makes certain things that we would like to do difficult. I could go on those examples if it's of interest. I don't think that's adversarial that was a silly choice of words. But it does mean that there's different interests being pursued in the code base that are not always Brave's. It's not always as privacy focused as Brave, would like.Jeremy: [00:37:40] I'm not sure if you would have an answer to this, but, when Brave was deciding, what rendering engine to use whether that's, Chromium's Blink or WebKit, or something else. Why, why make the decision to use chromium as a base?Pete: [00:37:57] So this predates me at the company so I can only think through some of these things. I don't want to say something I'm not sure about. The early Brave folks considered a bunch of different engines and Brave started as an Electron app. So basically when there is an extremely small number of developers at the company and it's extremely early days it was just everything was done on top of stock chromium. It allowed the company to iterate really quickly and try a bunch of new things and do some of the kinds of things that it knew it wanted to do that were easier to do at that level. Then trying to maintain a large patch set in this kind of stuff against Chromium.And there's probably some path dependency on that. We're no longer an electron app. We're a proper chromium project. That's part of it. I don't know the particulars of why Electron was selected and not a Gecko option or not a WebKit option. I couldn't say exactly what tipped the scale on one versus the other.Jeremy: [00:38:46] Something you mentioned was that private mode or incognito might be something interesting to talk about so could you elaborate on what you were thinking there?Pete: [00:38:55] Like the battle of what like private browsing mode is and the incognito mode is and what that is supposed to do is I think nobody has a single story for what it actually is supposed to be.In some browsers that basically means your storage doesn't persist after you close the browser. And that's all it means. The browser operates exactly the same way. Local storage operates the same way, et cetera, et cetera, except you have a separate cookie jar and a separate set of state that goes away when you close all your private browsing windows.That was for a long time the textbook definition or the whatever was agreed on. But you can see over time in standards bodies and in implementations.. I think there's been some recognition that users have a different understanding, or at least some users have a different expectation of what private means.And it can connotate something beyond just the state goes away. And so there's been a slow drip of new features. New privacy features into private browsing windows and major browsers. So Firefox by default if you enable a private browsing window you're in strict versus default mode for intelligent tracking protection, it does slightly different things.Chrome changes the operation of some APIs that allow you to query your quota on storage to prevent sites from detecting whether you're in private browsing mode, et cetera, et cetera, things like this. But I think it's interesting because it seems like a recognition that users want more privacy in a machine and are desperate for whatever buttons are in front of them.Even if what guarantees are being made by those buttons aren't totally clear.Jeremy: [00:40:26] Yeah, that's a good point because when I think of private mode or incognito mode, I think of your first example where it just means that it's going to clear whatever was stored on the computer like cookies or your history, things like that. And what you're saying is that now the opinions have shifted to where maybe private mode should be blocking trackers or maybe it should be... I think the example you gave was preventing sites from finding out certain things about your computer or your browser. That's a perspective that I didn't realize people thought but that makes a lot of sense.Pete: [00:41:05] And maybe this is a positive thing. It's become a little bit, eh, I'm not sure that's true. My impression I wouldn't go to war over it is, is that it's a little bit of testing ground for people to say, we know less people use private browsing mode than the typical mode.So we can be slightly more experimental in the kinds of features we test out in private browsing mode for privacy related features. If that's the case, then it means more and more stuff gets turned on by default over the medium term. I think it's probably a good thing for the webJeremy: [00:41:34] One of the things that you had touched on earlier was when you're trying to preserve privacy, when you have features that are blocking certain things that could be used to track you or block certain features in the browser, one of the side effects of that is that it can break websites.What are some, some common examples of where that can happen and how are, you know, you have brave, but browser vendors in general trying to to work around that?Pete: [00:42:04] Sure. So the most common or goofball way that can happen is say, you're using some ad blockers you're pulling in some filter list and it says you should delete everything that says, Ad in the URL or whatever, right. /ad/ something like that. Some website for whatever reason has something that's not an ad in a URL or something like that.Right now you're blocking something you don't intend. And there might be a script the page depends on for its execution. And given that the size of these filter lists, given that you could easily be considering hundreds of thousands, maybe even 200,000 rules if you're using a tool like Brave or uBlock Origin or something like that. The possibility for false positives is very high. So that's the simplest case that can happen. But then it gets more complicated. Brave by default blocks third party storage by default, there's a very extremely small number of exceptions that we make to unbreak websites.But by default we just block all third party storage. So if you're in an iframe, you don't get to store stuff, you don't get cookies. If you're a third party request, stuff like that. And, the vast majority of cases that works just fine.People don't usually care about the stuff that's going on in iframes on a page and when they do it doesn't usually need to touch storage, but you can imagine some places that'll break. Someone embeds a video and that video wants to store it's state or something like that.That requires some cleverness in dealing with it. And then just a third example. Like when COVID started becoming a popular concern is that people want to look at maps and see where COVID was spreading. And so these sites would usually use things like, either rendering these maps via SVG or be a canvas operation, and brave, by default did no, no longer, but at the time was blocking certain canvas operations and SVG operations because we knew they were being used by fingerprinters. And all three of those cases have privacy protections that ended up breaking things that at least in these cases are privacy harming.Probably even more so than my job doing privacy stuff at Brave is figuring out how to do that privacy stuff in a web compatible way or how to break less websites so people can use Brave without having to drop shields and drop those protections. And so each of those different things warrants a different response.So one has been to adopt a strategy that the uBlock Origin project takes. The uBlock Origin project is fantastic and all credit to those folks. That project, it is really fantastic work.Instead of just guessing, yes, I allow the resource or no, I block it. They'll also sometimes say replace it with some different thing that maintains the API signatures, but it actually nulls out the tracking behavior. And so that's been a really useful approach for unbreaking websites. If we can figure out what they expect, like the functions they expect to be in place but replace them with less painful stuff. And I can talk about our research project if it's of interest over the summer, actually with the student, Michael Smith, the student who's visiting from UCSD to leverage this, if that's of interest afterwards, Jeremy: [00:44:56] Are you replacing something in the JavaScript code that's running or are you replacing something that some browser API that is trying to get access to?Pete: [00:45:06] Sometimes, sometimes both. So in the simplest cases like Google Analytics provides some functions or like triggered some events on load. And if you block Google analytics, it means some things will never load. And so instead of blocking Google analytics, You just, you say, here's the request for Google analytics.Instead, I'm going to turn this thing that does nothing but trigger a load of it, but actually it doesn't touch network or anything like that. And so you're replacing the resource instead of requesting it. But you might also do things like, I see that some code that does something nasty or is inline so I don't get a chance to modify the request. I see it's inlined, so I want to somehow modify its behavior. And so I'm going to.. I mean, sometimes this stuff gets really gross, but I'm going to say overwrite some structure, the page expects to be there. I'm going to throw a stack trace. I'm going to look up and see if I'm in the inline code.If I am, I'm going to take path a and otherwise I'm going to take path B all these kinds of gross things. The web is a messy place and there's a whole bunch of tricks like that, that have to get pulled. So we pull a bunch of that stuff from the uBlock Origin project, we generate some of our own, for fingerprinting stuff.And this is something we've been able to pull from research that I'm really proud of us shipping, or I'm really glad about is in the same sort of way that uBlock Origin said it shouldn't just be yes or no. We should have some middle road that allows us to be more clever.We've taken the same approach of fingerprinting protection. So instead of just saying yes, it's allowed or no, the API goes away. We now do something we call farbling where we break the assumption that websites have that.. That the features are going to operate in a fixed way across browsers by adding a little bit of noise to the API response.So, if you're doing some canvas operations we'll with very low probability modify a pixel here or there or flip a bit like the lowest bit in the color channel for a pixel, that kind of thing. So instead of just blocking the API to protect people, we can instead have this more web compatible way where we still all the APIs do work, but we remove its identifiability by having it always do something different between sites, between sessions.Something that we're working on right now. And we actually are working with a student from North Carolina, who's prototyping this for us over the summer. This is another research intern named Jordan Stock who's doing great stuff. We're looking into a third option for local storage for remote storage.So instead of a frame, either yes. getting storage or no, not getting storage. We want this middle option where we can say the frame gets what looks like normal storage for the execution of the page. But by the time the top page for the top frame is closed, then that storage goes away. A lot of this stuff is just figuring out ways like the web compatibility game is, is figuring out a bunch of ways of breaking the binary choice and figuring out ways of sneaking more cleverness into the platform.Jeremy: [00:47:43] So when you're referring to a frame and the local storage going away could you kind of elaborate what you mean by that?Pete: [00:47:50] Oh, sure. So I'm on a website, like a typical website. you have your one frame, which is just this document object. And there's a bunch of like DOM structure that hangs off of that. But, one of those things off it might be an iframe, which is itself like its own contained document structure.And that can be infinitely recursed or infinite. It can happen infinitely deep. And so, this is usually referred to as the first party and the third party. Or the first like the local frame and the remote frame. There's some overloading of terms. Because, yeah, some browsers like remote frames are also remote processes, in the way that an operating system understands.But, typically yeah, the local frame is a frame that has the same, ETLD plus one, which means effective top level domain plus one, which is like the level of domain that you can register if you go to hover or whatever. And so all the frames that have the same ETLD plus one it's the top frame or local frames, anything else is a remote frame or a third party frame.And so browsers will use this as like a.. some browsers will use this as a heuristic for saying local frames the user trusts. And so I'm gonna allow it to store cookies and local storage and this kind of thing. Remote frames deserve less trust. And so I'm going to block storage or I'm going to partition storage or I'm going to do something possibly clever with storage, not all browsers do that, but it's a increasingly common.Jeremy: [00:49:05] I see, and I think you were explaining how you could have let's say an embedded iframe and it could use browser local storage, but maybe as soon as you click to another page, then that local storage goes away. Is that kind of what you were...Pete: [00:49:22] Yeah. So that's the approach Brave is taking. So there's another privacy group in the W3C called the privacy community group, which is, kind of like the sibling group to the group I co-chair. So I co-chair the review group that reads everybody else's specs and tries to improve the privacy of what other organizations or what other working groups are working on. Privacy CG is where browser vendors go to introduce new features. And so brave is involved in both. There's a lot of overlap between the two. Jeremy: [00:49:50] earlier you were talking about how people could be fingerprinted. They could be identified by seeing how things render, whether that's on a canvas or SVG and what you were saying, the way that you were dealing with it, which I found was interesting is it sounded like you were adding additional information. So your video card might render something a certain way, but then you would add additional things that would make it render differently than the video card normally would. And that's how you would remove that as an identifying factor. I wonder also, you were mentioning about how you had a research project at UCSD and I didn't quite catch what, what exactly that was.Pete: [00:50:35] Yeah. So in that order, the first one, the adding information parts, this, this approach came out of two research papers a while back. One is a paper called Privacator led by my current boss Ben Livshits who's a professor at Imperial in London. And the second paper was a paper called FP random fingerprint random. Both of those things introduced this technique or played with it. Brave is the first one who's productized it or included it in a popular shipping browser. But yeah the approach is to break the assumption that there's something unique about this browser that I can identify across sites.And so we randomize some of these features or we add an extremely subtle amount of noise that'll confuse fingerprinters, but look indistinguishable to users. We do it in a way that's random, but deterministic under each first party under each session. So you close the browser, you get a new fingerprint and if you go to a new site, you get a fingerprint.And so that prevents things from being linked. So it's been a nice way of taking academic research and figuring out a way to use it for a shipping, privacy protection.Jeremy: [00:51:36] Cool. So that was something that, in your role at brave or I guess brave as a company decided that this was something to look into from a research perspective. And then because the research went well, you're able to move that over to the product side.Pete: [00:51:50] Oh, well, I wish that was the case. I mean, so these are papers that preexisted at brave. It was a situation where we knew we had a problem. Most sites were breaking because of our fingerprinting protections. We didn't want to leave people less protected. And so research was one place we could start digging for a solution. So you asked about the project at UCSD this summer. There's a student who's visiting, Michael Smith, he's a fantastic student and a fantastic hacker. And so his project is, I mentioned before about the way uBO, uBlock Origin does these resources replacements. And so as you might imagine, these things are very difficult to generate. They take a lot of stepping into the debugger and manually figuring out how these large JavaScript blobs operate, particularly what, like what subset of the functionality you need to maintain to unbreak the pages. Extremely tedious and doesn't scale, it doesn't scale well. And so the approach that Michael and I are working on, Michael's doing the hard work. Is to see if we can automatically generate these things through a combination of browser instrumentation, a system we call page graph, which allows you to deterministically offline see the interaction of different elements of a page, AST analysis... AST is the abstract syntax tree or it's, it's a parsing step in, in executing JavaScript, or, parsing any language and then code rewriting to identify the parts that are privacy harming, rewrite, just those parts. And then we can programmatically generate these, these privacy preserving resource replacements in a way that can be automated instead of requiring the heroic amount of manual intervention that they currently do.Jeremy: [00:53:23] So if I understand correctly currently when you use something like uBlock Origin and you go to a website and let's say that website loads a script that has privacy implications, has some issues with tracking, but the behavior is still needed for that website to work. uBlock Origin will replace parts of the JavaScript source code so that the site still works. But it blocks whatever kind of tracking behavior that it was going to have. Is that correct?Pete: [00:53:53] Yeah, it's not that it fetches the resource and then does some rewriting on the fly. It just preloads like, this is the privacy preserving version of the Google analytics script, this kind of thing. Brave does the same thing. By the way, we had someone who worked with an intern last summer, Anton, who's now a full time employee at Brave and is phenomenal.But yeah, brave does the exact same thing out of the box. So we preload all the same resource replacements and are generating our own and do this in the same way.Jeremy: [00:54:23] And then in the research project that you're currently working on, the goal is for, the browser to be able to load these third party scripts and on the fly figure out if there's something, that should be blocked or changed in the script is, did I get that right?Pete: [00:54:41] That's mostly it, so it's slightly different than that. And the reason it's different is that, so JavaScript because it's so dynamic, it's difficult to statically analyze. You have to execute it and see what it actually does in a lot of cases to deal with all sorts of corner cases or all sorts of aspects of the language, because things could get aliased and functions get bound and there's dynamic code execution through evals and stuff like that. And so the difficulty there is you hand me some Javascript. I can't reason about it in a fundamental way about saying these are the seven places to where it's going to write a cookie or do a network request or touch your local storage or whatever.So that's one problem there. And the way we solve that is we have this heavily modified version of Brave that we call page graph, or that includes a feature we call page graph that allows us to among other things say, okay, these are the 18 parts of the JavaScript code that actually ended up touching local storage or doing a network request or whatever else.And so we use that for de-aliasing the values of JavaScript then offline we can, once we have those, we can programmatically rewrite the code by analyzing where those places are and replacing those lines of code or those chunks of the file with privacy preserving alternatives.And then at that point, we have our resource replacement automatically. So the process is offline and that we call the web and we will generate a whole bunch of these things beforehand that we can preload them in brave browser in the brave browser, or share them with the uBlock Origin project or anybody else.But the appealing thing is if we do all this work over the summer and this research project is successful, which I think will be, we have a way of automatically doing the stuff that before it would take an extreme, a pretty awesome amount of manual, labor to do.Jeremy: [00:56:28] And so it sounds like you have this special version of the brave browser and you could automate it to visit a bunch of websites. Pull all the scripts. And see what it does to the page. And then basically give you a list of, Hey, these are all the scripts that we think have issues.And we saw what it did, and this is the part we need to remove or change. And then you can ship that to users, either in uBlock Origin or in the brave browser itself.Pete: [00:56:57] Yeah. That's exactly right. And so most of that stuff already exists through fantastic tools that people like Google have made, puppeteer is a really fantastic system that Google has made that allows you to automate browsers and interact with sites and understand what browsers are doing.I mean, it's phenomenal, but it also doesn't answer all your questions. It's very difficult using puppeteer or using any system to understand this script modified this file. And that file then requested this image, and that image, whatever, you know, these complicated chains of interaction.That's extremely difficult to understand online in puppeteer or particularly after the fact just looking at the end result of the page. And so page graph is this system that allows us to with extremely high fidelity trace every single one of these operations in the page and then stitch them together in a graph in the sense of like edges and notes, not as a PDF.Jeremy: [00:57:49] Yeah, I think that's really interesting because I know one of your other, papers or presentations you have talked about Easy List, which is the list of trackers and ads that uBlock Origin and a lot of other systems use to decide what to block. And that sounds like a very time intensive process of you have all these different people that are visiting sites themselves and figuring out like, Oh, these are the things that should be blocked.Whereas with your research now, it would be more like we could have just the computer go and browse the internet for us. Figure out what needs to be blocked and save a lot of time in the future in terms of figuring out what we need to block.Pete: [00:58:33] Yeah, I think that's true. Although two complications there. And first I want to say that.. So EasyList is a fantastic project in there's a bunch of unrelated child projects. So there's EasyList. There's EasyPrivacy. There's a bunch of regional region specific EasyLists, this sort of thing. and, one the core maintainers of EasyList, who goes by the online handle Fanboy, is part of our team at Brave.He's fantastic. He's a full time brave employee and his job at brave is to maintain filter lists both for Brave, but also to benefit the larger community. And so things like Easy Lists are on one hand, phenomenal, like I think people just completely under appreciate how much there's four core maintainers of EasyList. And without these four people doing the things they do the web would be an infinitely more miserable place. And like the fact that like the web hangs on the evening, like the after hours jobs that these people have, until, at least until recently when they started being supported commercially is totally under appreciated and fantastic.Those lists are also deeply imperfect they're full of heuristics. Like the ad example, like /ad/ examples. There's lots of heuristics, there's lots of stuff that gets broken. And there's lots of, in quotation marks, dead weight or rules that were useful five years ago, but now it's very difficult to know if they're still useful given the size of the web.And so, it tends to just amass rules over time. None of that is the criticism of the maintainers who are fantastic or the community around them that contribute lists, but just the nature of the beast. Brave's approach. And some other researcher's approach has been to can we use these labels that these people have generated as high confidence things to start reasoning about the rest of the web.So it wouldn't be a replacement, EasyList. You still need some human in the, in the cycle somewhere to make some of these assessments, but can we like force multiply what that person is able to do through automation or machinery or, machine learning or, you know, different types of, of, of tooling.Jeremy: [01:00:24] Yeah, that makes a lot of sense. I thought it was really, surprising how few people were maintaining such a gigantic list. Like I think, you had said there were something like 300,000 entries, or I don't remember how many entries were on easy list, but itPete: [01:00:41] I think it's around 75 or I haven't looked recently. I know that Ryan's been doing some cleanup, but close to a hundred thousand in just EasyList. I think it's 70 something. And then there's EasyPrivacy and there's, you know, a long number of other lists too. So yeah, I couldn't tell you that the concatenated size, but large, very, very large.Jeremy: [01:00:58] So you've been centered on the privacy side and the tracking side. And I wonder in your work, if you had any visibility on the people who kind of want all these things to happen, like the advertisers that want to be able to do the tracking, has this sort of tracking actually been really effective for them. And on the flip side, I wonder how much of these ads are even being seen by real people? Could there be ad fraud going on in terms of computers are just looking at these ads? And we're not the ones looking at these ads.Pete: [01:01:37] Yeah. We're now stepping something out of my area and quadruple ironic quotation marks expertise. But I can, I can only share what I know, or my impression from working, doing what I do. Which is that yeah, absolutely. Fraud is completely endemic.To the degree that people have no idea how much it is, but numbers that I've seen up for online ad fraud are anywhere from 10 to 50%. These are not numbers. You shouldn't hold me down to, but, but simply to understand the magnitude of the problem, like enormous and in the number of like middle players, that are in these markets, make it extremely difficult for any one party to understand what's going on. There's a phrase that gets thrown around called the Lumascape. And after this call, I can try to find you an image, but it's, it's this kind of like 18 step deep, like flow chart of how advertising markets worked. And that was five or six or seven years ago when that image was made.But yeah, these they're extremely dense. And the vast majority of players in the markets you wouldn't recognize their name. Nobody would recognize their name unless you were an employee of that company. So, yeah, ad fraud is an enormous problem. And it doesn't seem like there's a way to, it's going to get better anytime soon.Like this system seems like it's definitely on its way out and kind of getting worse. One thing that's really neat about Brave is that a number of the people who work at Brave have like histories in ad tech markets playfully have said they're repenting for... their work at Brave is their apology for what they did earlier.One person who works BizDev.. Luke is like just a phenomenal dude and incredible at what he does, but he used to work in doing this kind of stuff, in terms of like helping to build tracking systems and understanding how they work and now, yeah, Luke, Luke is fantastic. Johnny Ryan is somebody who does policy work at brave, too.He used to work at PageFair. He talks a lot to enforcers, like people on the political side, who do like, CPAA and, GDPR kind of things to make sure that regulators are actually enforcing these things. And his sense is that just the amount of fraud and the amount of tracking is, is, is just unimaginable.And so, and so, yeah, the problem is, is well established. In terms of whether it's actually profitable I'm sure that's like very deeply debated. So I know that Google has some numbers that have said that if you remove like the behavioral component from tracking, you need to do just contextual tracking, so, or contextual ads.So ads that know like where they appear, but not who their who's looking at them. Their numbers suggest that like the profits dropped by like 50% something along those lines. I don't remember the exact numbers, but, something on that magnitude I know some people are extremely skeptical of those numbers. And of course has Google is not an unbiased actor, but those are the numbers that they've shared.And I know that these numbers on the other hand, that, that get pointed at you that says, The amount gained by, marketers in the, and people who are placing ads, is negligible negligible to negative, when you removed the, behavioral compo component, because, there's so much fraud in the market that they ended up, like behavioral tracking actually ends up having a negative return.So, so all that is to say is I, I, I deeply don't know. I know that the system relies on things that seem abhorrant to me, but, But there's a diversity of opinions. whether it's actually useful or useful for what it claims to do.Jeremy: [01:04:45] From site to site, the sort of effectiveness in terms of the ads, you see how relevant they are, it can vary really wildly right. And, and we're never really sure Why certain ads are being shown to us. Right. you know, the example that a lot of people will give is, on Amazon, right?Where you buy something and then all of the suggested items are like, for the thing you bought and people kind of joke like, Oh, you know, this targeting isn't very good. But on the other hand, you have platforms like Instagram, where I've heard that the advertisements on there they're actually very effective.They tend to show people things that they actually might be interested in buying and they actually go through and click. But it's interesting because like I was saying, I don't know why some things seem effective and why some things don't. It could be that they have tons of tracking information and they still do a bad job of what they show to you.Pete: [01:05:44] I had the same uncertainty about this stuff. I imagine that, I mean, I shouldn't hazard a guess. I honestly don't know, the usefulness of these things, I'm, I'm really dubious or I'm really uncertain about it. I doubt it, but I couldn't say confidently that it's definitely not the case.And I've heard the same kind of success stories and the same kind of, you know, catastrophe stories too. Two things here. One is that there's, This might be of interest. There's this kind of famous story of not success of tracking, but the harm of it. I can send you a link to the story if it's of interest, but there's a famous case of a family getting advertisements from target.These are paper advertisements from target. So the family starts getting advertisements sent to them for prenatal kind of stuff, cribs this kind of thing. And the father doesn't understand why this is happening and the parents don't understand what's happening.It turns out that the daughter is pregnant, and has been looking up information about how to take care for the expected child. And advertisers knew it before the rest of the family knew it. Anyway. So, I guess that's a story where maybe it was effective, but also morally reprehensible.And then the other thing I wanted to say is like, so this is maybe a chance to describe how brave does this differently than everybody else does. I think, one thing that I think is neat about brave is that brave does two things differently. One is that. There is no track, no information about you, your browsing ever leaves the device.And so this has two benefits. One is that your device is gonna have a lot more information about you than any third party is because it sees every website you visit. So, it can do a better job of understanding what might actually be useful to you. And second is that Brave lifts the incentive structure.So right now here on the web, the vast majority of ads are not going to be of interest to you. And all the ads come with this, like all these horrible side effects of hurting your performance. Violating your privacy, carrying the risk of malware, et cetera, et cetera, et cetera. And so nobody wants to look at it that's why ad blockers are popular.Brave's approach is different. We'll pay you to look at ads like brave incentivizes you to look at ads that gives you a reason to look at ads. It gives marketers a reason to prioritize your attention. It breaks the privacy and performance harm and security risk.And arguably can provide much better ads than, than some tracking based third party does. So I think there's something clever about what Brendan and Brian Bondy came up with. in terms of the way that brave goes about these things compared to how other marketers have.Jeremy: [01:08:05] We've been talking about how advertisers are using tracking to, to hopefully show you something that they think you'll be interested in, right. A lot of the research you're doing is to, to try and prevent a lot of that tracking.So if you do that and you show someone banner ads, how are you going to be able to ensure that those, those ads are relevant to the person, when you can't track them?Pete: [01:08:29] Two things, one is that brave will never, Brave, never puts an ad in the page. Like whenever you see an ad through Brave, it's very clearly not related to the page. It's in a notification to make sure that we are not putting ads against publishers who don't want them and for a whole bunch of other reasons, to prevent that kind of like, like brand confusion and all that other kind of nasty side effects about it. So brave doesn't track you in the sense that like your information never leaves your device. Brave if we wanted to, we couldn't learn anything about our users in any capacity like that.No bits hit our servers that describe your browsing behavior. But like that you're on the device that the advice is constantly learning and saying, Oh, it looks like you're looking at shoes. Looks like you're looking at cryptocurrency. Looks like you're looking at, you know, airline flights or whatever it may be.And so the device has a great deal of information that might be able to say, maybe you would like to see an ad about shoes, or maybe you would like to see it, an ad about, you know, vacations or whatever it may be. And so it's not tracking in the sense that like, nobody's looking at you.It's your own device, seeing what your device already sees. but it does have the kinds of information that seems like it might be able to, actually show you stuff that you might want to see it. The other thing is you mentioned, users understanding why they're getting the ads they're getting and to be able to control it.I mean, I think this is like a totally underappreciated concern in almost all of machine learning, where you have these extremely complex, deeply nested structures that are arrive at decisions that are completely opaque even to like machine learning experts, let alone to typical internet users.People's lives are being guided by these unauditable black boxes. Brave's commitment is we are committed to allowing people to understand and to see the model to edit the model, to partition the model, to add into, or remove certain interests.We're not at a threshold yet where, it makes sense to do that, but it is a commitment that Brave has made. It is absolutely in the plans and like, yeah, I mean, black boxes, like that terrify me and Brave is not going to become one of them.Jeremy: [01:10:29] And you had mentioned how Brave the browser is not going to add ads to a site that doesn't have them. Does that mean that for sites that will have ads, that they would have some relationship with brave where they say that we want to show, ads in brave and that's what has to happen in order for advertising to show?Pete: [01:10:50] Right now there's a couple different types of ads that Brave sends, the main one is there's notification ads. So, by default you see zero ads. You don't see anything, but if you say like, yes, I want to start getting paid to look at ads. You can say, show me, between one and five ads an hour and every, you know, one to five times an hour, you'll get a notification that looks like (fix transcript).But say notification, you get, if you received an email or whatever, and it'll say maybe you're interested in shoes or, you know, whatever it might say, that's the predominant way that you see ads in Brave products. You also sometimes see advertising and get compensated for ads. Like if you open up the new tab page and you haven't disabled it, you may see an ad there and that you similarly get compensated for that.I should say that for brave ads, the user gets 70% brave gets 30%. So it's like the inverse of the Apple app store. Right? Then there's the third tier of ad that brave considered, but does not ship and is working through the details on it. If we do ship it called publisher ads and that's when a website could affirmatively say yes, brave, please add ads in these locations on my website, Those, we don't do that now. if we ever did do it, it would be only with like the affirmative consent of the website.But there's a bunch of difficulties there that have kept us from shipping. It mostly like privacy concerns of, we don't want the ad, the site hosting the ad to be able to learn about the user based on the ad that Brave places in the site. Like that would be a way of just really enabling a lot of the same tracking that's happening right now.So we do not do publisher ads right now. We are thinking through ways that we might be able to do it in the future in a privacy preserving way. But right now the only ads that gets shown are notifications, new tab page.Jeremy: [01:12:28] I see. So, the new tab page would be something very similar to when you create a new tab in Firefox and they have like a list of suggested sites. Something like that.Pete: [01:12:39] Yeah. So right now, Like Chipotle advertised with Brave for a while. And so I think it was like one out of three times you opened up the new tab page. It would have a, you know, an image of a delicious burrito or whatever in the lower right hand corner, it would say Chipotle or whatever, like attractive images.It's not executing code. It's not doing animations. It looks attractive. That kind of thing. But it's just an image. And if you don't like it, you can turn it off. But if you like it, then brave will pay you to look at it.Jeremy: [01:13:01] Interesting. Yeah, it sort of reminds me a little bit of, back in the past, there were desktop applications that people could install and, I think they paid you. I don't remember if it was to click on the ads or, or just to see the ads. and this sort of sounds like a bit of, a modern kind of version of that.Pete: [01:13:20] I think that's true. Although, I mean, a bunch of things distinguish it. One is that, like the bonzi buddies of the world, like ended up becoming malware vectors, two is that they didn't have the kind of information that would be useful to actually like send you the kinds of ads you were interested in.They just pulled from a stock catalog. they were extremely obtrusive. I'm not aware of any ones that I paid you X like actual money or like a significant amount of money. I mean, but they might've existed. I couldn't say that they don't. One of the things that I want to say though, that I think is exciting to me about the Brave model is there is a sincere, honest question about like, how does content on the web get funded? And like advertising is not the only, but it's a significant part of how it content on the, on the web is funded currently.Brave's approach is different, right now, if you enable ads by default, that money goes to the websites you visit. I have, those are configured to show me five bites an hour. And at the end of every month, brave keeps track on the browser, not on the server, but the browser keeps track of these are the, you know, this is the distribution of your viewing time across the sites that you visit.And if you don't by default, brave, will just send your ad earnings to those sites. The sites that are involved like that are, are verified sites. They get revenue very similar to, or if not greater than, than the revenue that they would get for you looking at an ad that was an iframe on their page, but without the privacy harm, without all the nasty side effects.So I think this can be a really powerful way of funding the open web, but without all the horrible stuff that's comes, comes with it currently.Jeremy: [01:14:49] Yeah. I mean, I think that what you're seeing with a lot of news publications and even just people doing blogs and things like that is a lot of people are moving towards a subscription model, right. Where you pay me five bucks a month and, you can see my articles. And I think what's, what's tricky is that. You know, the web is so is so broad, right? You visit so many different sites a day. And so it's hard to imagine paying a monthly fee for every single site you visit. And yes, I'll be interested to see, see how that kind of model works out in the future.Pete: [01:15:26] Yeah. And I should say too, that it's all in exploratory stages, but, but an idea that brave is considering and may prototype at some point is can we have some sort of like, if someone opts into the brave system, then, then brave can be the way that you just automatically pass through those paywalls.Using the cryptocurrency, you could pay it in a, in brave is a way of saying, I don't want to have a subscription to a million different sites. If I'm in brave, then I just automatically do these, these invisible microtransactions to fund the sites that I'm viewing. I think there's something compelling about that.Jeremy: [01:16:02] Yeah, for sure. everybody loves complaining about paywalls.Pete: [01:16:07] Yeah, no joke.Jeremy: [01:16:09] Cool. Well, I think that's a good place to start wrapping up. is there, is there anything else you think I should have asked or you wanted to mention.Pete: [01:16:16] Nothing else comes to mind. I think this has been really enjoyable. I, well, actually, well, I can say two things. One is that, if you're any of your listeners are interested in privacy and web standards, like it it's a forum that could absolutely use more voices and more people who, Yeah, a greater diversity of opinion than people who work at browser vendors or ad tech companies.And so if any of your listeners are interested in those sorts of things, I would encourage them to get involved. They can send me a message or they can just go to the issues themselves, but that would be fantastic. They have more people involved there. And the second one is, I imagine that a large portion of your listeners are people who write software for, for a living or, or who are considering careers in writing software for a living.And a little bit soapboxy but you, yeah, that's a powerful thing and a privileged position for many people. And I would, it's worth thinking really, really well through like the morality of the kinds of causes you're spending your nine to five, like supporting.Jeremy: [01:17:06] And where can people, if they want to see what you're working on currently, where can they check out what you're doing?Pete: [01:17:12] Ah, so I have a website called peteresnyder.com where I have my publications and my research interests. a lot of the publications I work on at braid get published at brave/research. I write pretty regularly for the brave blog about new privacy features that are coming out in brave.And I will. Be writing as an additional set of articles on the brave blog about, standards work in the direction that privacy interests in what standards also I'm on Twitter at PES10k.Jeremy: [01:17:41] Cool. I think you gave everyone a lot to think about in terms of privacy and in terms of what's going on in their browsers. So thank you so much for talking to me today.Pete: [01:17:50] Thank you very much, Jeremy. This has been super fun. I appreciate it.Jeremy: [01:17:53] Thank you for listening to my chat with Pete. You can get show notes and a transcript for this episode at softwaresessions.com. The music in this episode was by crystal Cola. If you enjoyed the show, make sure to tell someone else about it. all right, I'll see you next time.

Vlad is a Pluralsight course creator and the author of Unit Testing: Principles, Practices, and Patterns.We discuss:ImmutabilityError handlingAvoiding nullPreventing invalid stateUpdating existing applicationsThis episode originally aired on Software Engineering Radio.Related Links@vkhorikovEnterprise CraftsmanshipIs Entity the same as Value Object?Combining ASP.NET Core validation attributes with Value ObjectsError handling: Exception or Result?Applying Functional Principles in C#F# for Fun and ProfitTranscriptYou can help edit this transcript on GitHub.Jeremy: [00:00:05] Hey, this is Jeremy Jung for Software Engineering Radio. Today. I'm talking with Vladimir Khorikov. Vladimir is the author of the book Unit Testing: Principles, Practices and Patterns he's a Microsoft MVP, and he's the author of many Pluralsight courses including Applying Functional Principles in C#, and today we're going to be talking about functional programming in enterprise applications. Vladimir welcome to Software Engineering Radio.Vladimir: [00:00:28] Thank you for having me.Jeremy: [00:00:29] The first thing I want to talk about is sort of what functional programming means to you, because it means different things to different people. To you, what are the core principles of functional programming?Vladimir: [00:00:41] If I were to describe functional programming in just a couple of words, I would say that functional programming is programming without hidden inputs and outputs. And that's basically it. And what I mean by hidden inputs and outputs is there are several examples of those. So the most, prevalent example of a hidden output is immutability.So let's say that you have a method that takes in some integer and then increments that integer by one. And what it can do is it can return that incremented integer back as the return value, but. It can also mutate some global state with that integer. And by the way, by hidden, I mean that this information is not present in the method's signature.So it's not present in the method arguments, and it is not present in their methods, return value. So in this example, when you, when you have this increment method if it, returns. A value, then it communicates what it does, pretty clearly. So it is honest about what it does. But if it instead mutates, global state, then this would be an example of a hidden output because that output is not present in the map at signature.And to understand what this method does, you have to drill down to that method and to see what. What's actually going on, because it can this information is not present, in the, in the signature itself. So that would be an example of a hidden output. The hidden input is a similar to that. So instead of taking that integer as a parameter, as an argument. This method can also refer to some global state. So, for example, some static property or field, or it can refer to some external service to request that in integer and then incremented and then put in to some global state. So that would be an example of a hidden output.Also reaching out to external systems such as the database or APIs would be that and also, the simple DateTime.Now would also be an example of hidden input because that input always changes. It basically refers to the systems low level, API, your, for example, windows API or Linux API to, to get, this input value so that's another example of, of a hidden input. Another example of a hidden output would be something like exceptions. So exceptions are hidden output because, when you throw an exception, and that exception is caught somewhere upper, the call stack. This exemption also is not present in the method's signature.And, you basically introduce another, Hidden pathway, for your, program flow that is not present in the method signature. So these are common examples of hidden inputs and outputs and functional programming is about avoiding those hidden inputs and outputs. If your mathematical function, your pure function would be something that, accepts a value, as an argument and returns a value and doesn't do anything other than that.Jeremy: [00:04:06] Okay, so let's sort of break down a few of those. So. The first thing is hidden outputs in terms of if you pass something into a function and the thing that you pass in becomes changed, then that in effect is a, hidden output because, there is no way of telling just from the method signature whether that behavior is possible.Vladimir: [00:04:30] Correct. Yes.Jeremy: [00:04:31] And so you're saying that the alternative to that is to make sure that the function is pure so that when you pass something in, if you are going to make a change, it would not be to what you passed in, but it would be something that you're returning back.Vladimir: [00:04:50] Yeah. So instead of mutating the state of some existing object, what you need to do instead, in functional programming is you need to create a new object with the required properties. So instead of mutating the object that is passed in, you need to create a new object with new properties and return it back.And with example, with a number increment that that's basically it, because when you increment the number by one and return it back, you're not, Changing the input parameter because it's a constant, you cannot change it. What you do instead is you create another number and return it back.Jeremy: [00:05:26] And when we think about objects that we pass in, or we think about collections. a lot of times the objects and collections we work with are mutable, right? Like, we can have a list type in C sharp, for example, and we may want to add something to the list or remove something to the list. If we are instead creating a new list every time we want to change something, what are the performance implications of that?Vladimir: [00:05:55] Well, yeah, definitely. So there are trade offs to functional programming. And one of the most common tradeoffs to, any immutability is this trade off of, Always creating new objects instead of mutating new ones. And, that's actually the reason why object oriented programming, has become so, so popular in the past.Because if, you know, functional programming actually was introduced before object oriented programming. but why it didn't take off is because, computers back then were not as powerful as now. And so it was very costly to do functional programming with all those memory allocations, with all those new object creation.So it was very costly to do so. And what we had had instead is we started to operate at the lower right level, of our programs. And we started to think of, , in terms of, memory allocations. So, but now we're kind of getting back to the roots and starting to, to do more of what we did back then.There are definitely trade offs here. And if your performance requirements for your application are strict, then there are some restrictions. So there are some limitations and probably you will not be able to implement some functional programming approaches.But in most business line applications, that's not something you need to worry about. So if you write some framework, for example, an ASP.NET Application not application, but the server itself, Kestrel, then you do need to worry about that. But in most enterprise level applications you don't, so performance is not the biggest concern.The biggest concern is usually the complexity and the uncontrollable growth of complexity. And, what functional programming allows you to do is it allows you to reduce that complexity at the expense of maybe not as performant code as you could have otherwise.Jeremy: [00:07:56] So would you say that in the average application that the developer should default to making things immutable, is that a reasonable default for most developers?Vladimir: [00:08:09] I would say so, yes. If it is possible, then you definitely need to default to creating immutable classes, immutable objects by default, it is not always possible and one of the limitations why is in object oriented languages it's pretty hard to create new objects based on the existing objects. So, for example, if you take F#, there is a really nice language feature where you, where you can take data structure or an object, and, create a new object based on that, existing object, but with the mutation, of, with the addition of new properties to that object. So you can say, for example, old object with, some property equals a new value and some other proper property equals other new value.And what it will do it will not mutate the existing object, but it will create a new object with those two properties changed and then, but all the old properties they will remain the same. And this is a really nice feature that helps you to default to immutability. Unfortunately, in object oriented languages like C#, we don't have such features and so it's not always feasible to do so.What I recommend you do instead is if you have some value, Or a simple class, you can wrap it into a value object, which is immutable. but for, for all other classes such as entities, like for example, a user or a company, that is usually something that you need to mutate, that is an example of an entity.It has its own, inherent identity, such classes usually it's usually not feasible to make them immutable, but what you can do is you can keep them mutable, but put as much logic as possible to those immutable value objects. And this way you can keep the separation between complexity and immutability.So your objects will be either mutable or complex, but not both. You keep the separation between the complexity and, mutability because it is, when you combine the two, you start to have, these problems with the, the ever increasing complexity with unmanageable complexity.Jeremy: [00:10:28] And you were referring to the concept of entities and value objects. Is that right?Vladimir: [00:10:34] Value object, yes.Jeremy: [00:10:36] And so what, what is the distinction between those two? You were saying that a value object should be immutable and an entity could be mutable. Like how do you decide which is which is, which?Vladimir: [00:10:48] Yeah. these two concepts are, from the domain driven design, but they are actually applicable to an application in which even if you don't follow, domain driven design principles, it, it is handy to refer to your objects in this way anyway. The main distinction between them is that an entity is something that is trackable, by your application has an internal, inherent identity.An example I often give is, let's say you have a dollar bill, so you have money class, and this money represents a dollar bill. So this dollar bill would be a value object in most systems because in most cases, you don't really care if you have the exact same dollar bill as you had before.So let's say if you give someone a $1 bill and they give you back a also $1 bill. You don't care if it was the same piece of paper as before because for you, they are interchangeable. And that is one of the most important properties of value objects. They are interchangeable, but that also depends on the domain model so on your context, for example, if you create a system that tracks those dollar bills, dollar notes, then in this case, the, those bills and those pieces of paper, they become entities because you do care about each individual a piece of paper. so you, for example, you can have the number on that dollar bill as the ID of your entity, and then you track, where it goes throughout its lifetime.So it becomes an entity because it starts to have its own identity, its own identity, and you cannot exchange $1 dollar bill for another one because you do care about the history.Jeremy: [00:12:42] Does this mean that pretty much anything that's an entity would also exist in some kind of permanent store, like a database?Vladimir: [00:12:50] Yes, exactly. That's because you as I said you care about the history of that entity and what it usually manifests in as you need to persist that entity in the database. And then look after the changes in this entity.Jeremy: [00:13:05] And you were kind of referring to the fact that these entities could act as wrappers to value objects. So I want to kind of give an example of, let's say I have a a ride sharing company, and I'm keeping track of my, my fleet of vehicles. So I have cars that are driving around the city and they're all reporting back to me, their position.And each of my cars has an ID which parts of my, data would be an entity and which part would be a value object?Vladimir: [00:13:38] Yeah. That's a good example of where you can apply this entity value, object separation. So the car itself would be an entity because you need to track it. That's, the primary indication that it is an entity. It has an ID, and the position itself would be a value object because you can replace it with another object, with the same content and two positions of the same type and content, they are interchangeable for you. So yeah, that's a good example.Jeremy: [00:14:08] And so the individual updates, they would have the ID of the vehicle and the actual position. That could be a value object. And as I'm receiving multiple updates from each of my vehicles, it's reusing the same ID. And, in my database, I might want to keep a history of, you know, all the locations that my car went.So with those historical positions, would those be their own entity, or what would those be?Vladimir: [00:14:41] So the positions themselves would not be their own entity in this case, what would be an entity is the historical record. So in this case, you wouldn't have the position itself as an entity with an ID, but you would still have some, let's say, vehicle history record or something like that, as another entity that would, also contain the position as a value object.So, you will have this kind of nesting here where you still have the same value object, but the entity would be a different entity.Jeremy: [00:15:15] I see. And so the reason that we have the value object is that it sort of tells our system that this concept of a location is identical, whether it's in the context of being an update, I'm getting from a vehicle versus, a historical position. They're really both the same concept, so I can use the same value object for both.Vladimir: [00:15:40] Yes, exactly. it is much easier to reuse this position value object between these two concepts. And also, another reason why you wouldn't want to introduce a value object for these two positions is because it is much easier to keep consistency, in this way. So let's say you have other properties in your vehicle other than the position itself.So yeah. Let's say vehicle has, a license plate. And let's, let's say it has two numbers of its license plate, and it also has, two properties that display the position of the vehicle. So X coordinate. And. Y coordinate let's say. Why you would want to introduce value objects is to reduce complexity. Because when you have, those four properties as just four properties, then the number of permutations between them is higher. Then it would if you group, those properties into separate value objects.So if you group the two properties that belong to the license plate into a value object, and you also group, the two coordinates into the position value object, then you will have only two properties inside that vehicle. And the number of permutations between them is much lower. It's just two, whereas the number of fermentation between the four objects is going to be well, mu much larger, much larger number (laughs).So, that's a good way to think of complexity of your system when you reduce, the number of properties that you need to keep in mind your software automatically becomes much simpler because it becomes much easier to keep the consistency because, for example, when you accept a position, what you need to do is you need to just, check the correctness of that position on its own without It's connection to other properties of the vehicle and the same for the license plate, you don't need to validate that license plate against the, the position coordinates. So yeah, the validation becomes simpler and just maintenance overall becomes much simpler too.Jeremy: [00:17:46] And so it sounds like rather than having vehicle update type, for example, and making that object be responsible for it, the validations of the license plate and the validations of the position information. Instead, you break out those concepts into their own objects so that those objects, when you create them, they validate whether or not it's valid.And so as long as you can successfully create one. You pass it into the constructor for the vehicle update entity and you, you know that they're correct because, they were already validated before you passed them in.Vladimir: [00:18:26] Yes, exactly, and that is one of the benefits of functional programming is that when you keep your objects immutable, your value objects immutable, you only need to validate those objects once. When you create them after that, you are, you can be sure that those objects remain in the consistent, in the correct state, so you don't need to validate them afterwards.Jeremy: [00:18:49] Do you find that in development code wise, is it easier to reason about what's happening when you're creating these objects and then creating the entity and then passing these objects into the entity versus having a larger constructor for the entity?Vladimir: [00:19:08] Well, it depends on the use case. So, yeah, usually what I like to do is I like to, Do it hierarchically. So you first create, lower level objects such as, as you said, value objects. Then if you have some other value object that consists of those lower level of value objects, you create that value object and then you pass that you, you kind of create that structure where you go bottom up from the lower level objects to the higher level objects, and you create them sequentially one by one, and on the top at the top of that pyramid, you have the entity itself, which you can create just by passing those already validated value objects into that entities constructor.And you don't need to do much else. So the entity itself becomes a much simpler to maintain.Jeremy: [00:20:01] And these examples we've been giving the value objects, they've been able to validate themselves. Like for example, the position, there's only a fixed set of numbers that are valid for the position, and so we could validate that without talking to an external store or a database or anything like that.How about when you have a case where. To see if something is valid. You need to talk to an external API or talk to a database. Like for example, if there was some kind of, driver registration or like a license that's associated with the car and you needed to talk to, some kind of. State API or, city information to find out if that car update is valid, where, where would that exist in your application?Vladimir: [00:20:50] Yeah. So there is kind of debate into where you should put this logic. I strongly recommend not putting it inside your domain model because your domain model shouldn't talk to any external systems. So you should make that domain model as functionally pure as possible. Let me step back for a second and, describe what I mean by that.So what you want to do in your application, especially if you try to, adhere to functional programming, you don't actually want to make all your code immutable because that's usually impossible. when you create an application, you do want that application to mutate some state because otherwise that application would be useless.For example, if you create a user or if you update the vehicle information, you do want to, you do want that information to be eventually updated. So the vehicle record and the database, it should be updated. What do you need to strive for is the so-called functional architecture. And this architecture is, where you have some sort of a sandwich between, where you first gather the information required for the business operation.Then you delegate, that, information. So you pass that information. And delegate the work to the domain model. And then when the domain model, completes its work, you persist the results of that work to the database. And so what you can do here is you can make your domain model as purely functional as possible, and this way you kind of push the mutable operations to the edges of your system, to the edges of your business operation.Because as I said, you cannot, skip those mutable operations altogether, but you can kind of, work around them. and with this approach, with this functional architecture, what you achieve is you achieve the separation between, the two important concerns. So the domain model is responsible for your, domain logic. And, I like to call it, immutable and the rest of your system becomes mutable shell. So it is a dumb shell that is responsible for communicating with external systems. primarily. And so, the two responsibilities here are the domain modeling and orchestration and you don't want to mix them together because, when mixed together, they overcomplicate your code.You know, you don't, you don't want to do that because, the code becomes much harder to maintain. So, yeah, so this is the functional architecture, this kind of sandwich where the beginning at the top of your business operation, it talks to external systems.Then it passes control to your domain model, which is as purely functional as possible. And then at the bottom, all the decisions made by the domain model and communicates it to external systems, including your database. In some cases, the flow of your domain logic itself depends on what kind of response you get from external systems. So in your, in your example, this response would be whether or not this vehicle. With this, number already exists in the database. And if it does, you cannot register a new vehicle with the same number.What you can do is you can delegate some decisions to the controller, for which you have to reach out to external systems such as the database because the alternative to that one would be to delegate this responsibility for communication with the database to the domain model itself. And this way you will keep the controller simple, but you will make the domain model impure. And that is, in my opinion, a worse option. even though you kind of keep the controller simple and you kind of keep all the domain logic inside the domain model, it's still not the best approach because you want to separate that domain model from this responsibility of communicating with external systems because, when, these two responsibilities are combined together, that's, when you start to have an over complicated code base.Jeremy: [00:25:09] If I understand correctly, it sounds like anything that has to do with external systems, even if it's a part of validation, those calls should be made at the outside layer. Like for example, in your application controller or in an object that's receiving messages from an external system.And, that is where you would make the call to check in your database to see if, for example, if this was an ID that already existed. Or if, you know, we needed to talk to some citie's registration system to see if this vehicle is licensed. We would do all of that outside of the, the value objects. We would do that more in our controller or in some area outside of all of the internal domain objects.Vladimir: [00:25:59] Yes, that's correct. So, your, domain model you cannot delegate this responsibility of maintaining consistency that spans across the whole database. So you only need to delegate to our domain model. the consistency requirements that span, the objects themselves. So ideally it should stay within a single entity or, in domain driven design terms an aggregate an aggregate is a group of related entities.If you're consistency requirements spans more than one aggregate or one entity, then yes, it's usually, it should be attributed to the controller itself. So, and, they check for uniqueness, let's say user, email, uniqueness or vehicle number uniqueness. That's an example of such a validation. Yes.Jeremy: [00:26:49] Let's say you have a value object that is a, vehicle and before we checked with the database, we didn't know if it was a valid new vehicle or not because we didn't know if it was already existing or if it's truly new. And at the controller we make a request to the database and we find out that this vehicle doesn't already exist yet, so it's a valid request to create a new vehicle.Would we then need to create a new type that says like, this is a valid, vehicle creation request, because if we hadn't done that check yet, then we, we wouldn't actually know if it was valid yet. I don't know if that makes sense?Vladimir: [00:27:32] Yeah, it does. so what you're talking about is you want to have, let's say an idempotent request that would either create a new vehicle or update an existing one if it already exists. Correct.Jeremy: [00:27:46] Sure. Yeah.Vladimir: [00:27:47] Yeah. I wouldn't create a new request, for that. So, if it is a requirement, if it is a business requirement to do that in one go, which it sounds like it is here, then I would just attribute this logic to the controller.So first check, if this vehicle with this ID exists, and then, go from there, either update its position. using this value object or create a new vehicle using the same value object. So, in this case, you will work with the same position with your object anyway, because you will use it for creation and for update of the vehicle position. And so it's just a matter of, what to do next when you see if the vehicle exists or not.Jeremy: [00:28:36] And so after we've completed this validation and we've created our value object, then when we persist it to our database, that would also be in the controller. Is that correct?Vladimir: [00:28:47] Yes. So the creation of the vehicle. That part would be the domain logic part. So if we are talking about the sandwich architecture, again, the functional architecture, then at the first, the top part of the sandwich would be reaching out to the database to see the vehicle exists.The middle part for the main model part would be the creation of new vehicle or, the update of the, the update of the existing vehicle. And then the bottom part would be a saving that vehicle to the database. Yes.Jeremy: [00:29:22] And I want to kind of step back to, at the start of our conversation, you were talking about hidden inputs and hidden outputs. And one of the things that you were talking about as being a hidden output is, a exception. For example, like if there's an error that occurs. You may not know, that an exception is something that can return, Walk us through like why that's an issue or if that is an issue.Vladimir: [00:29:53] The issue here is not the exception per se. But how you use those exception . So if you use those exceptions for validation, then it does become an issue. what I'd like to see with regards to exceptions is that exceptions are for exceptional situations only. And what I mean by that is situations that you did not expect to happen in your application.Validation is by definition, an expected, situation. Because you do expect your clients or your users to enter incorrect values or send you incorrect data and so on. And so when you validate that data, you do expect it to be sometimes at least to be incorrect. And so, what you can do is you can, for example, validate that data and then if it is incorrect, you can throw an exception saying that this data is incorrect. But, it has the drawback of, of this hidden output that I mentioned earlier where you create another another path in the program flow that is not, that is not present in the method signature.So let's say if you have a method a void method, that accepts some data and then throws an exception, you, you cannot be sure what it does, so you cannot be sure how you react to these exceptions, because it could be that these exceptions are caught at the method that now call this validate method, but it could also be several layers upward the call stack, and so it becomes much harder to debug this application and much harder to understand what it does.Whereas with something like a result type, what you could do is you can explicitly return the result of the validation from that method and this, this way, you will make the signature honest. So an honest signature is something I like to also refer to a purely. Purely functional method would have an honest signature because it will tell you explicitly what it does, what it, what inputs it has and what outputs it produces. The drawback with the result type is that you often need to write more code, because without them, you can basically just, make several validations, call several validate methods, and then if something is wrong, you will catch an exemption in the catch part of your application. But with the result, you have to, process each output separately. So you have, the first result, which you need to process and the second one and so on. And also another issue here is that, it becomes, at least in object oriented programming languages like C#. It becomes simpler to, omit that result because there is nothing preventing you to just forget about it and ignore it. And yeah, that, that could be an issue. And that is one of the trade offs of this functional programming approach. but again, I think, the benefits of this approach, overweight the costs because you're becoming much more explicit. In the values, those methods for return.And it is much more maintainable in the long run. And by the way, this issue with, the ignorable outputs, it is only present in OOP languages, like C#. For example. If you take F#, you cannot just ignore an output, which is none, which is non void or unit in F# terms, you have to pipe it into special function, ignore.So you can always see that you are ignoring something because you're seeing that, the return value is piped into that, ignore method.Jeremy: [00:33:44] For those who aren't familiar with result types, can you kind of give a brief explanation of what they are.Vladimir: [00:33:51] Yeah, sure. So a result is an explicit representation of some operation. So let's say, when you perform some operation, let's say you're trying to, save. Something in an external system. let's say that you want to update your profile on Facebook and you're using a Facebook API for that.And, so you're, you're doing some API calls to the Facebook API and this operation, obviously may fail. And one of the ways to to deal with that failure is, as I said before, is to use exceptions, but that's not the best way to deal with that because I, it is not often obvious where exactly you process those failures.A better way would be to catch those potential exemptions from that API at the lowest level possible, and then wrap them into an explicit structure such as a result class and the result class is a simple structure that tells you whether or not this operation succeeded. So it has. such fields as is it a failure or is it a success? And if the operation was supposed to return a value, then you can also make this result, class generic. And if it is successful, it can also contain a value, that will contain the result of the operation.Jeremy: [00:35:12] The benefit of using the result type is that your function signatures, they become very explicit in telling you that this call that you're going to make, it could succeed or it could fail. And this result type is going to tell you, whether it succeeds or fails and that way, you know, to write code to, to account for both cases.Vladimir: [00:35:33] Exactly. Yes.Jeremy: [00:35:34] One of the things about exceptions is that when an exception occurs, there's a lot of information that's kind of embedded with it generally a full stack trace, for example, whereas with a result type, you may not have any additional information on why something failed. how do you deal with that or, or are there cases where you would say it does make sense to use an exception instead?Vladimir: [00:35:59] Yeah. I'm not saying that you shouldn't use exceptions at all because there are use cases for them, and one of those use cases is, well, actually the only use case is where you have an, a situation that you cannot, you don't know how to deal with. And for example. if that Facebook API returns an error that you didn't expect.So let's say that when you wrote the software, you expected some set of return values, return errors, some set of errors from the Facebook. Let's say that the user doesn't exist, but if it returns some other error, some obscure error, you don't necessarily know how to deal with that error.And in this case, because you don't know how to deal with that, it is preferable to throw an exception. This would be an example of an unexpected situation for which exceptions are preferable. because exceptions represent unexpected situations in code, you shouldn't catch them.You should only catch them at the topmost level of your call stack. And the only way, you should react to them is to just log, what exactly happened. And then, basically crash the application. Or if it's some background process, you need to just restart all over again.Because otherwise if you are if you're trying to continue working after this exception took place what you can run into is you can run into an inconsistent state where your program entered some, state where it. It's kind of still working, but, it may become inconsistent and even save some data into the database where it will become much harder to deal with because you want to avoid that inconsistency as much as possible and as soon as that inconsistency takes place, you want to stop everything, basically crash your application. And that would be the only use case for exceptions. And so in this case, you do still have the call stack, which you can then log somewhere and deal somewhere somehow later.But. If it is an expected situation, let's say that the Facebook API returned that this user doesn't exist, then you do know how to deal with that and you basically don't need the exception, stack because you can just, process this, error from the Facebook. Turn it into a result object, and then, return back to the caller and that caller then can in turn show some friendly error message to the user.Jeremy: [00:38:42] So, so basically when you're working with external API APIs, like the Facebook API, you may make a HTTP request, and maybe it times out and the HTTP library that's built into C#, I believe it would throw an exception. and what you're saying is that you would know ahead of time that I expect that there may be times where my request times out or it fails, I'm going to catch this exception and then I'm going to return a result type that kind of explains what the failure was, rather than, just throwing that exception and catching it somewhere else.Vladimir: [00:39:19] Yes, but that depends on whether or not you know how to deal with that, even if you expect some time out. Yeah, let's say that the Facebook API, calls to that API may time, time out from time to time. So you need to see whether or not you can deal with those errors. Because if you cannot, then even if you expect those situations to happen, you basically cannot do anything about them.And so you need to throw an exception anyway and that could be because let's say, the Facebook API is essential for this operation, and you cannot proceed without a response from Facebook. But if it is not essential, let's say if a user updates its profile and you want to update its Facebook profile as well simultaneously, but if you cannot do that, then still fine you can proceed further.So in this case, you can see you that this facebook call was a failure, but you know, that it is not essential for this business operation and you can just ignore that result and move on. And another example, let's say you write an ORM such as entity framework, and, in this ORM. The lack of connection to the database would be an exceptional situation because you cannot do anything about that.And you don't know, how the user of your library will react to that exception. And so in this case, because you as a library writer, you do not know how to deal with that exception. You also need to throw an exception and then the library or a user such as yourself or myself, we can decide whether or not this operation was essential for us and whether or not we can proceed, with that exemption or not.So let's say that when you're trying to save something to the database, it is preferable to do that. Say, for example, when you try to log something into the database. So it is preferable to that the log is successful, but if it's not, then not a big deal. And so in this case, you also need to, to catch and that exception that the library throws and then transform it into a result class and then process it, upper the call stack.But if it is essential for your application, let's say you're saving not a log entry, but the user itself, then even if you know that this ORM can throw an exception, you cannot do anything about that, and so you shouldn't process that exception. You should basically allow it to pop up to the upper layers where it will be logged and the application will crash.Jeremy: [00:41:58] Another thing that you sometimes talk about in the context of functional programming is this idea of how object oriented languages, they usually have a null, concept where. Instead of returning the object, you expect you return a null. And that could be because you couldn't find the element it could be any number of reasons. What are the drawbacks of returning a null?Vladimir: [00:42:25] Yeah, it's a common $1 billion mistake that all object oriented programming languages have in them. And that is the problem we have now is that, they make all your code dishonest because what, . What they do with your code. Let's say that you return a user from some method, and that user is a class.So in C# classes are nullable. all classes are nullable, so they, you can return not an object of that class, but now that would be a valid program from the compiler perspective. the problem with that is that you cannot differentiate between nullable user and the non-nullable users and where you can, see a method that returns that user.What it actually does, it returns a special, class, which you can call user or null because it may be either a user or null. And so when you want a non-nullable user, you, there is no way for you to do that. because all, as I said, all classes are nullable by default. and yeah, that's the problem because that introduces another hidden output that you cannot see just by looking at the method signature.Jeremy: [00:43:36] One of the things that you've, done as a way to kind of push back against that as this concept of a maybe or an optional type, could you explain what those are and how they're used?Vladimir: [00:43:47] Yep. I think what I did, in this course, yeah, I'm trying to remember, it was several years ago. Yeah. So what you can do instead is there is a good tool, it's called fody null guard that you can use to basically inject null checks in all your methods and properties. And what it will do is it will check all input arguments for nulls for you and also it will check output, return values for nulls as well. And it's a very good tool. I tried to use it, in as many of my projects as possible, but it's not always possible, let's say that. And what it does is it. Helps you to approximate your code to this world where nulls do not exist.So, if you try to return a null, where your method returns a user, your method will automatically throw an exception because of this automatic checks for nulls. And to avoid that, you will need to use a maybe or an option as in F#. So, and Maybe is a special struct that you can use to explicitly tell the clients of your code which parts of your, inputs or outputs can be null.And because it is a maybe it itself can not turn into a null because it is struct and structs in C sharp are not nullable. And it becomes sort of a nice trick, to avoid, these null problems. Because if you want to make your return value, your user nullable, you have to wrap it with a maybe of user and you cannot do that otherwise because if you try to return null without that Maybe your method will throw an exception.But if you, if you do use the Maybe, then your null will be automatically transformed into an instance of that Maybe type and your code will, will proceed. The validation here is not as strict as in functional programming languages because, this issue with the null it will not be caught at the compile time.But still it's close enough because, even though the compiler will see this code as valid, I mean the code where you return a user, but, return null instead, but it will still fail at runtime. So you will have sort of, close to functional guarantees. here as well.Jeremy: [00:46:17] Sounds like it's similar to the result type in the sense that with the result type we were saying we would wrap an object in a result, and what that would do to the method signature is it would say that this function you're going to call it could succeed or it could fail.And similarly, this Maybe type, it sounds like it's wrapping your object. It's wrapping your response and telling you that your response could have something in it or it could have nothing in it, and it's making it explicit as a part of the return type.Vladimir: [00:46:48] Yes, exactly. So, the, the maybe type it gives you the same benefits of the result type. So it makes the, method explicit. But in addition to that, if you use it, these Fodi null guard library, it also gives you some guarantees, some runtime guarantees that you will not actually have a null. Where you return a non Maybe user.So if you return just user, then you have this guarantee that it will actually be a non-null instance. Because if you try to return null, then the application will throw an exception.Jeremy: [00:47:21] Something that C# added recently in C# 8 is it added non nullable reference types. So it has a compiled type check, to see if you could possibly be using something that's null, is that a good substitute for this Maybe type or kind of what are your thoughts on that feature?Vladimir: [00:47:42] It's a nice move in to the right direction, but I don't think that it is a good enough substitute because those checks they will only give you compile warnings. So they are not compile errors, but that's not a big issue because you can turn those warnings into errors by setting up a couple of things in Visual Studio.The main issue here is that, it doesn't catch all those situations where you may have nulls. And so you still may have some issues with nulls even though, C# 8 will tell you that everything is fine. So that's basically my concern with that, that it's not as strict as they might be type.Jeremy: [00:48:27] It kind of gives you some protection, but there are some cases it doesn't catch. So it may give you this false sense of security.Vladimir: [00:48:36] Yes, exactly.Jeremy: [00:48:37] Another thing that you bring up in some of your courses is that. When data is coming in from an outside source, like let's say, you have a API and somebody sends you data via JSON, or you get data via a message queue, you tend to create a separate DTO, a data transfer object rather than use the entities or the value types that you've created internally.Why do you make the decision to do that?Vladimir: [00:49:11] Yeah, that's a very important thing to do in my opinion, because, you need to maintain the separation between data contracts and your domain model. And this is important because if you are using the same domain classes, and as, as these input data structures, then you may fall into several problems.So let's say that you you have a controller that is responsible for user creation and, one way to represent that data that clients send you when they try to register a new user. one way to do that would be to use the same user domain objects. Object as you have in the domain model.So let's say that it has a username or password and maybe some other properties that map one-to-one to the properties that the client sent you. The issue here, the first issue here is the security hole, potential security hole. Because, you may introduce some properties in the future to your domain class that you don't want the client to set. So let's say that you introduce a flag saying that this user is an admin. let's say it's a boolean isAdmin flag. And if you introduce it to your domain model, then, it, it becomes a potential security problem because now your clients can send this flag as well, and it will be deserialized into that domain model.And if you save it. as as-is into the database, then, you will basically create another admin in your system without knowing that. And so that's one another problem here is that. When you use your domain classes like that, you are setting those, domain classes in stone, so to speak, because you often need to maintain backwards compatibility with the clients.And what it means is you cannot refactor those classes as often as you may need. So let's say that, for example, you, your user has a name property and you want to split it into first name and last name. But because you want to maintain backwards compatibility, you can not just do that because the old clients of your application will break because they will not know that split they will not know about it and they will continue to send you just one name. And in this case, it becomes problematic because now you have to maintain sort of the old name property. But you also need to add the first name and the last name and then somehow correlated between the two maybe transform name into first and last name user using some rules.And you don't want to do that. Instead, what you need to do is you need to have a separate layer of data contracts. DTO data transfer objects where you have as many versions of those data contracts as you as you want. So if you decide to split the name into first and last names. You don't need to modify the old DTO.You can create a new endpoint that accepts a new. DTO, version two, let's say, that we'll have the first and the last name, and then you will do the conversion between these two end points. So the first endpoint that still has the first version of the DTO. You can do the conversion between your domain model and the old data structure there.And so in this way, you are free to modify to refactor your domain model without looking back to how how it makes incompatible or backward compatible changes for your clients. and so you, you sort of. Decouple that data contract from your internal domain model. You want the internal domain model to move as fast as you want, so you want to be able to refactor it, but you want to keep the data contracts backward compatible.Jeremy: [00:53:06] It's almost like the difference between when you have a class, you have a public interface, and then you have the private implementation. And when you. Use data transfer objects, you expose an interface that you want to keep the same, but you want to be able to modify how that's handled internally in your system.And so having these DTOs, it makes sure that you can make as many changes as you want internally in your system without affecting what your API looks like to the outside.Vladimir: [00:53:41] Yes, exactly. It's a good analogy. Yes.Jeremy: [00:53:43] The one thing I can think about as far as DTO is and converting them to internal domain objects. One of the things about that is it sounds like you could potentially have a lot of conversion code. How should you sort of plan for that and where should that exist in your application?Vladimir: [00:54:02] So, my view has evolved since that course. I think what I did is I created, extension methods on top of, result, and it's still a good way to do that. Let's say that, when you create a user, you need to validate, his first last name, let's say an email, and let's say a couple of other properties.And what you can end up with is a lot of code that does validation. So you are creating a value object first, and then you need to make sure that the user with the same email doesn't exist in the database, and then you need to create another value object and validate it . And so it creates a lot of, if else statements that clutter your code base.What you can do instead is you can follow the so-called railway oriented approach, which was introduced by Scott Wlaschin. and so what I did is I basically adopted this approach from F# to C#. and you can introduce extension methods that will drastically simplify all those if statements, It will help you to reduce the number of line codes by a factor of three, and without losing any readability. And for simple validations, it's still a good approach, but, there is a nice, way of dealing with validation in asp.net and that is validation attributes.And what I did in that course, I said that validation attributes is nice, but they kind of don't play well with value objects. And so if you want to really adhere to domain driven design principles or functional principles, then you need to switch from those annotations to this railway oriented approach.But you actually can combine the two. So you can combine the approach with annotations and still have this validation logic in Value Objects. So one of the biggest disadvantages of having those validations in annotations is that you are duplicating that validation. So you want to keep those validation rules inside your domain model because it is part of your domain model it is essential part of it. when you put let's say regular expression validation attributes on top of your DTOs you are duplicating those rules between the two parts of your system. So now you have a value object with the same rules and, and also, that same rule that exists in the data annotations.So to combine the two, you can actually create your own custom annotations that would delegate, those checks to the value objects, but still would work exactly the same way as the regular annotation attributes, meaning that you can declaratively put them on top of your DTO properties and they will work, very well.So you will still reduce the, number of validation code lies lines drastically, but you will still keep this nice declarative approach that you had with annotations. And I have a blog post on my website, we can link it in show notes where I showed this approach in detail.Jeremy: [00:57:25] Just to make sure I understand correctly, so what you're describing is in ASP.NET. When you have a model for a DTO, you can put annotations on it. You can have your property and above the property, you could say something like, the max length is 50 so this person's name can't be more than 50 and what ASP.NET is able to do is if you were to create a form and you used that property in the form, if somebody typed in a name and they put in 80 characters, ASP.NET, using that annotation would be able to automatically, sort of create an error and you would be able to put that next to the field. And I think what you're saying that you can do is that you can keep those sort of validation rules inside the domain objects that you create, or I think you called them the value object, and you're able to still write an annotation that just refers to the validation that exists in your value object rather than using the builtin, data annotations.Vladimir: [00:58:37] Yes. Yes, exactly. And that's a nice way to combine the two because it sort of combines the best of the two worlds. You still have your validation rules in one place.Jeremy: [00:58:48] What's your approach to, when you have a code base, that has exceptions and it passes back nulls the calls to the database are sort of mixed in with the objects.Like how do you start that process of bringing in more functional concepts or just bringing in more concepts that are easier to follow and to understand?Vladimir: [00:59:09] Yeah, that's a great question. And yeah, it's a tough one it depends a lot on the specifics of that project on specifics of the team and the management. It's one thing if this project doesn't, evolve much and it's just some project in the maintenance mode where you don't need to introduce a lot of new features in this case, I actually don't recommend that you do much because it will be, it will most likely will not pay off in the long run.But if it is a project that is actively developed, then it's a different story, and in this case, you need to come up with some refactoring approach, some refactoring strategy, there are a couple of approaches here. In Domain Driven Design, for example, Eric Evans wrote a great piece where he talked about, so this approach that involves bubble contexts. And so a bubble context is something that you create inside a legacy code base that adheres to all the good principles. So you have a nice separation between the domain logic and the orchestration and your domain logic is ideally, purely functional and, because you, you cannot refactor the whole application at once.And I actually don't recommend that you rewrite your application either because it's, it's not a good idea in most cases. You still want to start somewhere. And where you can start is by creating these bubble contexts. Let's say that you have some new feature or you need to modify an existing feature, and this feature is somewhat not too connected to the other system.And so you can start to isolate this functionality into the bubble context and surround that bubble context with an anti-corruption layer and that anti-corruption layer, it's basically a repository that converts your good and clean domain model into the database with this messy legacy structure and converts back into your nice and clean domain model.And what you can do is you can start expanding that bubble context. You can gain territory, more and more with new features with new, refactorings. And eventually what you want to do is, come to this point where your bubble context becomes the main part of your application. And, it's the legacy part that is surrounded by the anti-corruption layer.This pattern is also called a strangler pattern where you strangle, these legacy part, and cut off slices of functionality from that part and transform it and refactor them, into your bubble context.You need to first define the building blocks of your domain model. And those building blocks are usually value objects. So the , easiest to create classes in your application, let's say as simple as an email value object or as simple as a customer name value object. And so, when you do that, you can put, domain logic that relates to those email and customer names to those value objects. start using these value objects from the rest of your system, and then, start, from there so you can build a hierarchy of objects. So let's say that you have another object that consist of, those smaller building blocks, smaller value objects. So you do that. And then, as I said previously, you can proceed to your entities and refactor that entity. So instead of separate properties on the entity, you can start to have, properties, defined as value objects. And so, you are attributing more and more logic from that entity to those value objects. And the entity itself becomes simpler. And then, from that level, you step even further and push the domain logic down from controllers to those entities. Because what you usually have in such legacy systems is the anemic domain model where your domain logic is separated from the domain data. So data is separate from the logic and that we can talk about it a bit too.But the main drawback in this system is that it's hard to maintain encapsulation. It's hard to maintain consistency, inside. the domain data, because it's separated from the logic that works upon that data. and the logic itself is usually like in something like services. And so you can push that logic from services down to entities.And so what you have it's sort of a cascade of a logic that you push further and further down. And the more down you can push it, the better because the easier it will be to work with. And the problem with the anemic domain model is that, well there is actually a nice, dichotomy between, anemic domain models and functional programming because, anemic domain model it's about separating data from functions or from operations that work upon that data. But functional programming is kind of the same. So it's also about separating data from operations that work upon that data. The big difference between the two is that in functional programming though, the data is immutable and it is a big deal because it's impossible to corrupt immutable data. So it's basically impossible to come up with something that you cannot change. but anemic domain models, although they exhibit similar properties to the functional approach. The biggest, difference is that it is mutable that data inside domain models is mutable and you can never know who mutates that data and how they do that. And so it becomes impossible to enforce restrictions on everyone whom mutated that data with, with such an environment.Jeremy: [01:05:18] Given all the things we've talked about, if people want to kind of see an example of a lot of these things in action, are there any code bases that they can take a look at that are open source or any good examples that you can point them to?Vladimir: [01:05:35] So, if we are talking about C-sharp, then I would recommend my Pluralsight course. it's called, applying functional principles in C or something like that. I actually have a trial code for Pluralsight side, so if you want just reach out to me. We can put my email address.So they will give you a, I think it's 30 days, unlimited access to Pluralsight, so you can watch all my courses and more during that time. Also if we're talking to F#, I would highly recommend Scott Wlaschin's books on this topic. So he. Has a great site it's called F# for fun and profit, and it has a section with books in it where one of the books is basically the collection of them articles from the site itself.But the other book is that it is about Domain Driven Design combined with the functional approach, and it's really great book. It, it explains how to do the Domain Driven Design in a functional programming language like F#.Jeremy: [01:06:38] And, where should people go if they want to see more about what you're working on and follow you?Vladimir: [01:06:45] The best place is to go to my website. It's called enterprise craftsmanship.com. And yeah, you will find all the links there.Jeremy: [01:06:55] Cool. Well, Vladimir thank you so much for coming on the show.Vladimir: [01:06:59] Thank you for having me. Jeremy: [01:07:00] I hope you enjoyed the conversation with Vlad. You can get the show notes and a transcript for this episode at softwaresessions.com. Alright see ya.

Brian is a Senior Developer Advocate at GitHub and was previously a Developer Advocate at Netlify.We discuss:Unintentional gatekeeping Formal onboarding for your projectsThe value of discord communities for newcomersStreaming issue triage and programming on twitchHow Open Sauced helps developers get involved with open sourceRelated Links@bdougieYOOpen SaucedExpressJS Triager GuideGraphiQLBabel's contributing guide (Gives suggestions on familiarizing yourself with codebase)Webpack's funding pageSE Unlocked episode with Dan AbramovExplore GitHubChangelog NightlyCode TriageFigmaDiscordTwitch StreamsbdougieYOjlengstorfNoopkatMusic by Crystal Cola: 12:30 AM / OrionYou can help edit this transcript on GitHub.Jeremy: [00:00:00] This is Jeremy Jung and today I'm talking to Brian Douglas, he's a senior developer advocate at GitHub, the host of JAMstack radio and the creator of open sauced, an application to help new contributors to open source. Brian welcome to Software Sessions.Brian: [00:00:14] Hey Jeremy, thanks for having me on.Jeremy: [00:00:16] The first thing I want to get into is. What's the biggest barrier for people getting into open source?Brian: [00:00:23] Yeah, that's a good question. I think the barrier for open source is something I found or discovered right off the bat. I've been developing for over seven years now, seven years ish. And getting into open source can be daunting, especially if you don't know where to get started.So I think the biggest barrier is actually onboarding and it's just knowing is a CONTRIBUTING.md, the proper place to go to or is there some other secret channel somewhere or a Slack group or something else where you could actually get a relationship with the project? I think a lot of us leverage a lot of these tools that are open source, and go years of leveraging them without even knowing who's contributing to them, who's powering it.What community is involved in the project. So just knowing where to start is usually the hardest part. I think that we do a good job as a developer community. There are guides on how to contribute, like open a pull request, manage your commits and stuff like that.But there's no guide of how to say hello when it comes to giving your first open source contribution.Jeremy: [00:01:27] Not knowing where to start, even if there is a CONTRIBUTING.md and there's issues out there. People are like, I don't know which one to pick up. I don't know who to talk to first. It's just awkward, I guess.Brian: [00:01:38] Yeah. Each project is different. So there's no centralized CONTRIBUTING.md file everybody is sourcing from so where one project can be, could say, okay, CONTRIBUTING.md, git clone, git, check out a new branch, git push origin. And then that's it. And some of them don't even have contributing MDs.Some of them are just READMEs. Then you go to the README and there could be missing information. Some projects don't have READMEs. Some projects have READMEs and websites and documentation and Slack groups. So not knowing the balance of how to actually get involved in the project.And I think what it really comes down to is if I started a new job the first thing I'm gonna get is a step by step okay. Here's your laptop. Here's how to do this thing. Here's how to clone the repo. Here's who to talk to. A lot of projects don't have that. Like they don't have like area owners, plugin owners, who's on the review team. Who's on the triage team. How big is the contributing group? You can go into the GitHub repo and discover it all. But it'd be nice if someone just gave you a piece of paper or one file to get all that information. I think we've sort of grown out of the CONTRIBUTING.md and we need something else.Jeremy: [00:02:49] When you are looking at an open source project, there's all these different issues whether it's bugs or feature requests. And it can be hard to know which things are suited for your skill level. And what do you think is the solution for that for somebody trying to pick out that issue that would work for them.Brian: [00:03:08] Yeah. And I mean, the easy answer is they have labels like good first issue and like documentation. If some people don't know this, if you go to any GitHub repo, so like github.com/nodeJS/node. Or node/node, I think is what it is /contribute. So if you add /contribute on the URL you can see all the issues that are available for up for grabs and you can leverage them and jump into.I don't actually recommend doing all that first and going to labels. I think the very first step is actually talking to a person. So the quickest place that you can find communication synchronous of like, Hey, I'm looking to contribute. I've been using this thing for six months on this project. I just want to give back. I had this idea for a feature like open issue, like ask questions on the issue, or even like now we have a feature at GitHub called discussions. In addition to that go into the discussion, but also limit the amount of back and forth you have to do asynchronous. And just go directly to the source, which is the person who is on-call already to chat with you.A lot of projects have discords now. So find the discord link and then jump in there and say hello, because your experience is going to be completely different when you're actually talking to somebody and asking questions synchronously in discord.The chat scrolls so it doesn't matter if you say a random question or you ask a question that's been asked a hundred times. Someone will give you a link, but it's better to do that than to be the person on the issue asking the same question for the fourth time. Or asking the wrong question at the wrong time.I think that's a little daunting as well. If you don't know how the project, the underlying secret sauce of the project is actually laid out for you.Jeremy: [00:04:45] When I think about open source, I think about communication being asynchronous, right? Going through issues, emails, mailing lists, things like that. And you're saying if you're first starting out, actually the best thing would be to, to find more synchronous communication, find that discord room or gittir, or, whatever it is, where you can actually have a conversation with someone.Brian: [00:05:10] Yeah. And to be fair I'm catering this more towards a beginner opensource contributor. If you're experienced, do the regular thing, reference the issue, open up the PR, and no need to look for a synchronous communication if you know how to solve the problem.GitHub itself is like 50 million developers worldwide. There's not 50 million developers doing open source. Let's just be clear on that. So there's a big difference between the users on GitHub who are just shipping code, like normal building websites for their companies or mobile apps or whatever it is to the people actually contributing the code that's powering all that stuff and powering GitHub as well.I'm using this term called unintentional gatekeeping. I've been thinking about this a lot and I want to write a blog post on this because it's around the flow of information. So if I happen to be in the right Slack channel or the right discord, I have more information than the person who's not there.Because there's more information flowing through there than there are publicly on issues, because issues are treated as like a statement of work. You're declaring that this is the way it's working or declaring that it's broken and next steps to reproduce it or whatever it is.And same thing with PRs. Like you're declaring this is the work I've done. This is the next steps. You review it. It's very robotic. But when you have that relationship that's built in a Slack channel and like, this is similar if you go to meetups or if you happen to know somebody from college or high school or whatever it is very similar. That relationship is like a relationship that helps give you that extra edge.And I think when we talk about things like tech, there's definitely a lot of conversation about diversity, especially today. so when we have diversity of backgrounds, diversity of culture, where people coming from, you tend to find a lot of the, especially newer, smaller startups have a mono cultured diversity.And people are well aware but there are VCs who are telling us it doesn't matter when you first start out just build a product, get all your friends in the same room. Build only the culture fits and stuff like that. And then let's move on and then we'll figure it out when the company is like much bigger and it has like much bigger issues.So I say all this because it's opportunity for people to become quote unquote of the culture of the open source project by just being in a room like this being in the room and listening. And if you find out that maintainer or the contributors are not your cup of tea you can just move away and move to another project or fork the project and create a new project that's similar but has a different culture.Like, open source a lot of these things are MIT licenses, no limitation for you to try things out and maybe copy code and create your own project and see if there's growth in that approach. I don't recommend that. But if that's your approach, definitely try it out.Jeremy: [00:07:56] Another thing that I often hear when people talk about wanting to get into open source is they have trouble finding someone to mentor them or help them through the process. I wonder what are your thoughts on how we can improve that experience?Brian: [00:08:13] Yeah, this is more on the maintainer. Individuals who are managing the projects. I mentioned the onboarding experience. There's obviously opportunity for them to have better onboarding. Have some clear steps of what your expectations are for people to contribute to the project. Not just how to clone it and open a PR but more of like, how do you report an issue?Is there a template for reporting issues that can guide the person into actually asking the right question as opposed to free for all and then your issues turn into stack overflow which is not the best place to ask questions is GitHub issues. Like you could do that, but stack overflow it's an entire platform built for that reason. So how do you kick people from issues to stack overflow instead?We didn't talk about what sort of code, right. But, I do a lot of JavaScript and there's this one library called express JS. it just builds quick servers for your websites and web apps and ExpressJS actually did ship something really recently, I think back in April, they merged in this new quote unquote feature or a guide, which is called the triager guide. Are you familiar with this at all?Jeremy: [00:09:16] I'm not.Brian: [00:09:17] Yeah. So, basically what they're doing is they're instead of saying, Hey, We have a lot of issues go ahead and pick one up and like merge it. Or not even merge it just open the PR and we'll go back and forth for like weeks or months. And then we eventually merge it.Instead, they're saying if you want to intro into express, you don't have to know anything about express. You don't have to use express. We have this role called the triage role and it's literally a team in the org that you can join if you just raise your hand and your job is to triage issues.So if someone provides an issue. If they don't provide reproduction steps, you kick it back and say, Hey, can you provide reproduction steps? So if you don't know how to do it, then the maintainer probably won't know how to do it. Or maybe they do, but that's a lot of time for them.So joining like a triage role or having an opportunity to do that, to label issues, to mark things as ready for review or ready to contribute or whatever or good first issue or whatever it is like that's-- express has a lot of issues and there's a lot of time spent trying to figure out is this valid or is this not?They're actually taking help from the open source community, giving them a badge, which is the triage role in the project. So it shows up on their profile. That was great for prospective employers. Like, Hey, you're in the org, we're using express, you have access to the maintainers. Maybe we can get our features on there.That's eye opening and it's eye opening that I have not seen that at all until very recently. So me personally, for my project, I just launched a triage role. Cause I want people to be able to have an introduction into my project, which is a react app without needing to know react, like all you have to do is know how to answer questions or how to find information.If you don't, there's other people on the team that can help guide you. And we have a discord as well that can guide you to actually getting things shipped.Jeremy: [00:11:02] I've noticed when you watch people's livestreams for coding and who work on open source projects-- a lot of the time they spend is actually on issue triage. Is on looking through all these GitHub issues, figuring out which ones are valid and which ones are not. And so I think that's an interesting idea of getting people started there so that they get to see the process of open source without necessarily needing to jump straight into the technical details. That's an interesting path to get more involved.Brian: [00:11:36] Yeah, I like it. I'm curious of, what Twitch streamers you watch too as well cause I've been trying to collect the list of myself. But I like watching people do open source, actually. I think right now, Jason Lengstorf is doing some open source right now on his Twitch stream, I'll catch the VOD later.But, yeah, I think that's actually a good thing you brought up to you as well. Cause I've been doing some Twitch stream myself and trying to figure out what is the purpose for live coding on Twitch? Is it to give webinar type tutorials like screencasts, is it to interview like what we're doing on a podcast or do that as a Twitch stream and where I found my niche or what I like to do on Twitch streams is actually do exactly that triage issues.I'm actually gonna be live streaming later today. And I've been doing some sketch, some UI building. I'm not a designer. At all. But I took a course last fall and learned how to use sketch to build some UI templates to not have to rely on somebody else to actually get me across the goal line for our shipping projects.So I'm going to spend 90 minutes building out some UI, and actually trying figma for the first time too as well. Cause figma, it's sort of like the GitHub for designers. I'm not sure if that's the summary of their product. I don't work there, but, yeah. So basically I'll be doing figma.I'll be building some UIs and some wireframes. Just sort of figure out the next steps, cause I've got a backlog of features I want to add to my project, but I don't know how to tell people that this is how we're going to work on it. Cause I have a whole I think 17 contributors at this point I was going to say team, but they're contributors. We do have teams in squads or whatever. But yeah, so it's easier for me to get everything in my head and the vision for the project out onto like a UI. And let individuals know. But my question, I first asked, like, I'm curious, who are you watching on Twitch?Jeremy: [00:13:19] Yeah, so I just, dabble a little bit. One of the people that I find interesting is, Suz Hinton and sometimes it's like issue triage type stuff, but also sometimes she works on more hardware type projects, in sort of the intersection between working with JavaScript, but actually working with, physical hardware and actually wiring stuff up. I don't watch a lot of streams but things that are interesting for me is being able to see someone's thought process. Cause often when you watch a streamer they're talking through their process and what they're thinking and whether it's doing triage or whether it's working on a bug or a feature. You get to see how somebody works in a way that you wouldn't from a screencast. With a screen cast or a lecture they're very well prepared. They've been practicing, whatever they're trying to teach, whereas in a stream it's more this is something that they haven't done before, and you're just going through that process with them.Brian: [00:14:25] yeah, Suz Hinton, I guess. Is it nope cat or no op cat?Jeremy: [00:14:30] Yeah, noopkat, I think that's right. Yeah.Brian: [00:14:32] Yeah. Yeah. Not hearing the word said out loud. I say noop and nope. noop. I interchange it's likeJeremy: [00:14:38] They're probably all right.Brian: [00:14:41] Yeah. So I've actually watched Suz and I like her style and I like the projects that she works on too as well. I've yet to catch her live in a while. So I don't know if she stopped streaming. But yeah, similar to yourself I like seeing the thought process and the people walking through how to build things. Because at the moment a lot of us are working from home. Especially in the state of California. And we're not sitting next to our coworkers anymore and asking those questions.I think twitch has in the last three months it's sort of exploded with live coders, and live coders as in the general people who live stream and code at the same time. Because I think a lot of people will just figure it out, like, Hey, I need to have community and I'm not getting it through my team Slack channel. So it's been an interesting transition as far as like a whole other culture that's growing on Twitch at the moment.Jeremy: [00:15:35] For sure. And then for yourself as the person who is doing the streaming, what do you get out of it and what are you looking for?Brian: [00:15:43] Yeah, I mean, it just comes down to community. I started the stream, mainly because I wanted to have a place to start throwing my ideas out there for the project I'm working on right now, which is open sauce and I started streaming two years ago.I'd heard of people doing live coding on Twitch, but it wasn't very popular at all. Only a handful of people were doing it and I even talked to some people at Twitch about it. Some people who were familiar with this space and were knowledgeable and so I started doing it, but I didn't really have the proper equipment. I had my Mac and I was just streaming from my Mac. And nowadays you gotta have proper lighting and step up the game a bit and a green screen as well which I'm sort of sitting in front of it at the moment.I was doing that but I sort of fell off because my daughter was born just a couple months after that. So I took time off from work but also took time off from coding, just to enjoy some paternal leave. But anyway fast forward to very recently which is a couple of months ago I started streaming again focused on trying to build an open source project and just have a place to write code consistently cause my day job is developer advocate and I don't have any long standing projects that I work on a regular basis. A lot of stuff I work on it just sort of ships complete and then we don't touch it unless there's something broken. So once I get done with the actual shipping something, I just move on to the next thing. So there's nothing that I can feel proud of that I continue to work on. Or where I do like the latest and greatest things like Vue JS or whatever. I shipped Vue projects, but I move on and they just work until I have to do maintenance on it. So I wanted to have a consistent place where I could just talk about a story of a project that I was working on, which again, I keep mentioning it, which is open sauced. And since then I've actually built a community of quite a few developers interested in the same problem I'm trying to solve which is open source onboarding.Jeremy: [00:17:32] Let's get a little bit into open sauce. We were talking about a lot of different troubles that people have when they're trying to get into open source. How is open sauce trying to address those?Brian: [00:17:42] Yeah, you tee'd it up for me earlier with the whole trouble getting into open source, it's onboarding. So we're building a platform to provide structured onboarding for open source projects. So like me connecting with maintainers and projects to add a simple YAML file to the project.So that way anybody who navigates to the project on open sauce, can have a good, like step by step process of who to talk to, how to get involved in the project, where to go for the synchronous communication. The other thing is that tracking projects is something you can do on GitHub, but it's not really built for that. At least today, like hopefully GitHub gets on board. And puts open sauce or adopts a lot of the features that we're sort of building. But the goal is not to track projects you're already a part of or even track projects that that you're working on. It's more tracking projects that you want to quote unquote stalk. So like a GitHub star is a thing sort of like you hit a star, like, or whatever on a project. And it goes into like a list. And usually most people just forget about that list because you just add a star, like that's about it. It's like sort of Instagram. You just add a like, and you move on and that's what GitHub stars have become.So it's hard to track things that you're interested in based on stars. You could watch projects, but then when you start watching projects it becomes a basically signal to noise ratio. So then with a very popular project, you don't know what you're looking at. So then you fall off immediately, cause this is too much information or not enough information, one or the other.Yeah, so basically there's not a lot of tools. So I think a couple of years ago, actually, when we met a couple of years ago, I had just shipped a little side project for myself, which is essentially bookmarking issues. Issues that I wanted to work on GitHub. So my day job is at GitHub. I've actually seen internally this feature be built a couple of times, but we sort of backed away from it cause it didn't really solve the right problem. And so as soon as I joined GitHub, I was like, Oh, maybe I won't work on this thing anymore. So I took a bit of a break and then noticed that we weren't shipping it.So I just picked it up recently, but essentially you could find the project you're interested in. Find the issues that you're interested in and mark them to save. And then manage the note taking. So if you want to take notes on like this is the maintainer to talk to, this is an example that I can leverage to solve problems or help triage things, even like-- So I was trying to contribute to the graphiql opensource project graphiql. It's like a little playground to test out GraphQL queries and trying to contribute to that was actually pretty hard. There was a lot of context I'd missed. And at the time the project itself was transitioning from Facebook's org to the GraphQL foundation.But also pretty much everybody who was becoming maintainers on the project were actually transitioning in and owning the project. So there's a rough transition into that, moving from org to org, but also the maintainer is becoming acclimated to the project as well.They were all familiar with it, but now they own it. So they are all trying to figure out the best practices and how to clean it up. So at that time that's when I was trying to contribute. So I was looking at the issues and I'm like, man, I think I could solve this one, but I'm not finding the bug's actually invalid because it seems to be fixed.So they had a ton of old issues that were just sitting there that were invalid from that because the transition cause they had fixed a bunch of stuff. They were still getting acclimated. No one went through and closed out a bunch of old issues and to close out a bunch of issues automatically with no reason or question sends the wrong signal to the users. So they just sat. And I tried, I tried working on an issue that was invalid. And I discovered that when I commented on the issue with my thought process, the maintainer or one of the maintainers came back to me and was like, Hey, this is actually invalid. All that backstory, I just told you, he told me right there on the issue. And he's also like, Hey, we also have the discord, that you should come and chat with if you want to work on anything else. And I was like, Oh, okay. That's weird. Like in the repo, there was nothing about a discord. They since added it. But then I was able to get all that context, the conversation and the questions, like, what is happening with this project? Like where can I help out all in discord. So that's like, that's sort of the summary. That story is a summary of what I'm trying to accomplish. No one, like myself needs to go into a project and be confused with skills. Like knowing that they can actually do something. To fix problems, but they don't know where to start and they don't know how to approach it because the way I do code at GitHub is different than the way GraphiQL is doing it in their repo.So that's the high level goals with some other features that we're trying to work on, but, we're always taking ideas. If you go to opensauce.pizza, that's the actual website that's live github.com/opensauced that also exists for anybody who wants to contribute or just ask questions.Open ideas, open the cool ideas, or bad ideas. It doesn't matter. Open up a discussion. We'd love to hear what problems they're facing in open source.Jeremy: [00:22:39] Do you envision this being something where the list of projects is curated or is it more somebody can pick any project on GitHub?Brian: [00:22:48] There are projects that do curation for open source projects. So GitHub has the explore feature. You can sign up for a newsletter, you get a bunch of projects every night or every week. I forget what the cadence is. And then change log has a nightly, the most popular projects. Here's a list of them, check them out. And then there's like also code triage was another project too, as well. Where you can also be curated a list of like Ruby projects or JavaScript projects as well. We do want to have curation as a feature. Like this is more there's a repo that you're using or a library that you're using. Add the library or the repo just the URL to open sauce to your dashboard. And this is all login through GitHub. It's using your own data. The backend is all the open source repo. So when you log in, you click the create the repo button, it starts tracking all your notes and all the issues in the repo itself, all open source. So once you've done that, then you have a nice tracking issue to then say, okay, I've looked at this issue, look, this issue, doesn't work, invalid or whatever. I closed this. We also track your contributions as well. So if you do any sort of PRs they'll show up in the list, but also in addition to that, it also tracks your issue contributions. So if you comment on an issue, it shows that in the list as well. So that in the eyes of open sauce, nontechnical contributions are contributions. That's another thing that I stand on, which is just because you don't have a green square for that day. Doesn't mean you didn't do anything.The platform itself, the answer to your original question-- No curation today, curation in the future, maybe it is on the roadmap. It's not actually realized in a plan. But the focus really is around if I already know the project I want to get involved in, can I just take it to open sauce and get all the information I need, digested.So I can just click the steps one, two, three, even to hammer down on that onboarding experience, like there's a project called babel they do transpiling for JavaScript for different versions. Like one of the best things you can do if you want to contribute to Babel is use Babel.I did mention triaging is another thing you can do, but if you already know how to do it and you're ready to start-- use Babel use Babel plugins, build a Babel plugin, like try going that far and seeing actually how it actually works under the hood and how you interact with the actual babel core library.So that's a recommendation and that's like a recommendation I'm actually trying to work with that team. Hopefully. I talked to him months ago, but I haven't really picked up the conversation because I wanted to focus on actually getting a dashboard working. But I would like to see as an onboarding experience, if it's like a Webpack or if it's Babel or something else, as part of my onboard experience, build a simple tool or clone or a hello world to actually get my brain wrapped around it.So that way you can confidently go in there and answer questions around, how is this broken for this user? And how can I fix it in the context of what I know?Jeremy: [00:25:39] So it sounds like coming up with-- I'm not sure what you would call it, almost like an exercise of before you contribute to this project here's a well-defined thing that you can build so you have an idea of how to tackle a real problem.Brian: [00:25:56] Yeah. Yeah. And I think it's easier for some projects than others, But I think that's on the maintainer to say, Hey, here's the contributor guide. But in addition to the contributor guide, here's the actual action items to do to get yourself up to speed. So whether it's build something on your own or just clone one of the example repos and walk through that, those are all possibilities. But it's up to the maintainer, not everybody has to have the same sort of step or guides or not everybody's working on projects on the web. But as long as you have the steps, that's all that matters. So if someone actually knows what the step is to actually get started, that's helpful.And like, we're talking about at the moment we're currently in like a existential crisis or at least America is. And there's a lot of people who have been underserved by their leaders and their community leaders and even the higher level of government. And like you go into cities. And there's a different, this like take like LA County. LA has one of the largest police forces in the United States. LA has one of the worst public school systems in the United States. I know we're talking about a political issue so I won't go too deep in that, but really what it comes down to is like actually information sharing.So if somebody who is in LA County, and working towards life skills or just like growing their career or whatnot. If they have to go to the public school system there they're going to miss out on a lot. Like there's going to be a lot of information they just don't know. And if you happen to be just one County over, which is Orange County, then you're in such a better experience. And it's such a much better step up. And I think that it comes down to like, if I want to contribute to open source and I wanted to level up my skill in my career am I getting the right information by contributing to this project? Or even using this project? I think that should be a decision that we should make as far as contributing to projects.If there are not people going in there and contributing and it's that free form, like free flowing information. And there happens to be few people who are managing the project whether it's good or bad that should be eye opening. Cause then you have one or two points of failure, like one person gets sick or has a kid, or takes time off. Then it's down to the one person left over to actually contribute. And there's nobody else in this entire developer community that has knowledge to actually contribute back. This is maybe not popular to talk about but Facebook has a lot of open source projects that we are leveraging entire products our features our companies on. Yeah, but the only people who work on that are technically Facebook employees. So is that really open source? And I know things like React, they do have contributors outside, but the individuals making all decisions are internal Facebook employees.And I know they have the best interest in the open source community. I'm picking on them because they just happen to be the example I have on top of my head, but it didn't seem like information is really flowing in back and forth. And maybe I could be corrected too. I'm happy to be corrected on that. And if there's information on the react community that allows people to onboard a lot easier than I'm all for hearing it.I'll probably do my research after this podcast cause I pulled it out of thin air and picked on them without having any sort of backing statement. But anyway, regardless, there are projects that do not have information flowing that we're supporting or we're leveraging in our projects. So whether it's react or not, we should take a hard look on, is there a proper onboarding for anybody to basically jump in there and get things done?Jeremy: [00:29:37] Yeah, I think that's an interesting point in terms of when you have companies, whether it's Facebook or any other company you have people who are being paid to work on these open source projects and ultimately the company that's paying, they want to get something that's of value to their own company. And, whether it's a benefit to the rest of the open source community is it may or may not be front of mind. So I think that that's an interesting sort of, I don't know if you'd call it a problem, but a discussion to have. How much control do companies have over the software we use and is it too much, and on the flip side, it's like, if it's not companies doing it, then it's volunteers doing it. Maybe that's an issue too, right? Like that we're relying on so much software that's being worked on by people for free.Brian: [00:30:36] That's the thing that I like discussing too as well, which is not just a onboarding decentralization of open source, like future. This might be counterintuitive to everything I said before, but when you talk about working for free, there is money being funneled into open source.And again, I apologize for picking on projects that people love and leverage in their, their projects day to day, but look at a project like Webpack and I only pick on them cause I know them and I use them. And, I know the maintainers as well, but you see the project is making half a million dollars a year, just in open collective.So that's the one location that I've looked at it and I can sort of cite today cause I've just looked at it recently. But that actually pays contributors to contract, to help solve and squash bugs. So like when you look at that, that's awesome. Actually, hats off to them and I think we should see more of that. I don't think that's a bad thing just to be clear. But what about projects like rollup or parcel or all these other bundlers and packagers and stuff like that? Those are all valuable projects, but they're not getting the same sort of funding. Are we voting by the dollars that we donate as well? And, that's another question that was asked and like, I'm not here to say that's wrong or bad. I'm happy to fund other people doing open source. Cause I think it's more about not about true open source. Like the Richard Stallman, like open source, everything type of deal. Basically what I'm getting at is that we should put our dollars where our mouth is, but also we should put our money in the things that are actually providing value and providing information and providing access to all developers as well. The best thing about this is that you have all these bootcamp grads, all these college students, coming out the gate, leveled up, and ready to ready to ship day one, which is great.There's no month long process of like, Oh, you're only stuck to doing bugs or reading articles. You can actually ship code day one because you have the GitHub account while you're in college, you have access to all open source technology. So if you want to build a quick website or a Minecraft server, whatever it is how to interact, like with stack overflow in forums and answer questions to get your job done.And, that information sharing has exploded the ability for us to grow our developer community. To be able to hire developers and train them quick. And like all bootcamp grads are only two years behind from anybody else because they just need two years of experience to actually get up to speed because the web, the mobile, everything like code changes quickly. Well, not all code not all code's the same, but I can speak for the web. The web moves quickly. So you're only two years behind the last person.Jeremy: [00:33:19] Yeah, I think that's really great that more people are getting exposed to the idea of what open source is and having the skills to be able to contribute. And what I also think is interesting is Dan Abramov, who's on the react core team and he's also the creator of Redux.He was talking on a podcast and he was saying he has all these projects that he no longer maintains that he used to work on and he feels a little guilty about it. But he was also saying that if somebody comes in and takes over those projects, some of them, when he was working on them, he was working on them at Facebook.So he was getting paid to work on them. And when you have somebody come in, who's coming in on a volunteer basis. I'm not sure the word he used. It's it's almost like they've been tricked, I guess is what he was saying is. I was working on this thing, getting paid for by my employer and somebody else is coming in and taking that on for free. And so there's this interesting imbalance in terms of the people who are getting paid to work on it and the people who aren't.Brian: [00:34:25] I mean, it's a challenge cause there's a lot of people actually able to-- GraphiQL founder or sorry, maintainer I was talking to, he's getting paid full time to work on GraphiQL. So there isn't a balance. Like he definitely is a knowledge holder, but I think that's a testament of like I spent at least two minutes, dogging Facebook, but also it's a testament to Facebook that they actually value putting open source maintainers there full time, to support the community and also even open sourcing it in general. Like there was a time when I first got into programming where you didn't open source stuff, just because like, it didn't make any sense. I talked to people at Pinterest and they open source somethings like they had a very similar front end framework, which they called Denzel. And like, maybe it was open source. I don't remember, but I'd never even heard of it until I talked to someone at Pinterest. Facebook put the time into actually promoting it, putting a conference on and actually getting people to care about it and saying like, Hey, this is actually the way to do it. They get value because then it's easier to get hired at Facebook. Despite the fact that they don't actually use React in their interviews, but it is a leg up, like you're knowledgeable that Facebook is hiring and hiring React developers or JavaScript developers, or we're even doing JavaScript at Facebook.So I would say that the value that the person who's sliding up against, Dan and working with them and getting feedback from him, he's actually getting mentorship directly from Dan. And I would say that's not a monetary value. That experience that relationship that you get is invaluable to be quite honest.And I said, I was talking about the whole LA County and the information sharing, like the more, the information is shared, the more value it's going to be like, the information I've gotten for free from just doing open source, it'd be involved in the community and going to meetups, is invaluable. I would not be here today without it. But if I relied on someone to tell me that or me reading my own blog posts or me figuring that out myself, I would not be here today. So I would say like, yes, it would be nice to have a six figure salary to work on open source every day and triage a bunch of issues. That'd be amazing. But also the fact that Dan's accessible, and makes himself accessible, I think, is what it makes the biggest difference. Dan he is a figurehead for the React community. But the fact that I can go to React. Open up a PR and get Dan or Brian Vaughn or somebody else from the team, to actually review my stuff and give me feedback and tell me what's up and make me feel comfortable. That's a big deal.Jeremy: [00:36:57] For sure. the way we learn the quickest is when it's from somebody who knows more than us, or has come before us, is able to teach us. And like you said, I think that can be really invaluable for sure.Brian: [00:37:07] Yep.Jeremy: [00:37:08] Another thing I want to talk about is you've been a developer advocate for GitHub and previously for Netlify. And I know in the past you had mentioned you had been a little hesitant to take on the developer advocate role because you were really interested in coding and engineering work.Have you ever thought about going back into a more engineering focused role or what keeps you in the advocacy role? Brian: [00:37:37] So what keeps me there is my paycheck. So I'm paid as a senior developer. That was the whole deal for me to go to, GitHub. And, that's helpful. Also. I love community. I love interacting with the community and having opportunities to be out there. I missed being able to just put on my headphones and just write some code all day, go to lunch, come back, write some code all day and then have maybe a meeting once a week, do a standup once a day. I do miss that. And I do miss that solidarity time, but also, I mean, I am a pretty outgoing person and happy to have those conversations. So like there isn't a balance, of doing that. And I think a lot of devrel folks, they come and go, not like they don't quit devrel, but they do go work on a project for awhile just to get back in the right head space, to be able to actually talk about devrel.Like one of my biggest fears from doing developer advocacy full time is that not working on a project full time on a regular basis your skills start to not keep up because I had mentioned you're only two years behind in the last thing that came out. So if you're not constantly trying new things out and seeing what's out there, then it might be harder to get an engineering job later on.I've mostly give up on the dream of climbing the engineering ladder. And I've only made that decision recently, because I think I get a better feeling around writing code when it's my own code, but also open source. So another reason why I even have open sauce is because it was a project for me to have long standing code, like learn how to write, test, learn how to use hooks in React when everybody was transitioning, like I had a project the ability to leverage and there's no pressure to ship there's no PM pushing me to like, Hey, we should have had this last week. Like I get to basically instead of sit and write code, I watch a lot of tech videos. I do a lot of screencasts. I do a lot of Twitch videos as well, so I have more freedom and less pressure to ship things. Mainly because I don't have a project that needs to be shipped constantly. So I tend to build, and I like the pattern as developer advocacy and I recommend this for anybody, like build a project that you can actually use to leverage your skills and keep that going.So whatever it is, if it's your sourdough bread making app or whatever it is to tell you when to feed your starter, Which I mean, I mentioned that because I actually want to build that, but, anyway, like build something like that so that way you can leverage and talk about on a regular basis and I think most devrel folks, they have that app for them. And I think open sauce is mine. So yeah. I guess the original question was like, yeah, I do have feelings around doing full time engineering, but I'm actually pretty content with my role today and my access to information and leveling up my skillsets.I am not spinning up Kubernetes clusters or even know how to do that. I've done it before, but like, it's going to take me a bit to refigure that out. Like just give me the one that's working on your repo and I'll go from there. And that's my approach to code, it's write it quick, get it done. And maybe write a test.Jeremy: [00:40:41] Yeah, that's cool. You may not day to day be diving in really deep on coding every day, but you keep that for yourself for your own personal project. So that you keep your skills up and plus you get to work on your own terms at your own pace. So you don't lose that joy or the the fun of just building things.Brian: [00:41:04] Yeah, and I mean, to be clear too. I do have projects that I do maintain. It's just that these are projects that I only maintain like twice a year for updates and I'm just basically having the dependabot, update them for me. And then every now and then we'll add a new feature or answer a question or something like that, but all closed source stuff to make my devrel a lot easier.Jeremy: [00:41:26] Cool. I know we're running up on time. And I just wanted to ask you one more question. Five years ago you were a new developer. you moved from Florida to the Bay area. You attended a lot of meetups and community events, and now you're on the other side, you're the one giving talks, giving presentations and talking to new developers. How do you feel like things have changed?Brian: [00:41:52] Yeah, I mean, it's changed a lot. And I think asking the question now at the time that we're in currently, I envision it's gonna change even more in the next year. But I would say when I first got the programming, jQuery was definitely a legitimate place to put all your JavaScript. CoffeeScript is probably the next level above it. And they were pretty legit things to use. I know a lot of JavaScript developers from 10 years ago probably are cringing for me saying that but you didn't have to know a whole lot. I think we had a lot of stuff that we just took for granted and we've seen a lot of security vulnerabilities because of that.So I think now-- I feel like the developer space is just leveled up in being educated in things like security, progressive web apps. So with that being said there's a lot to learn. So you can't be counted on to know everything. And that's the other thing about being a developer advocate. It's like no one knows everything.There's no pressure for me to get back in the engineering full time so I can know everything. Cause no one does, no one's perfect at backend orchestrating of servers and spinning them up in containers. And even on the front end doing CDNs, like no one's really expert on that.And I think people are really focused on things like the JAMstack where you can just pick and choose and leverage tools and free accounts that get your stuff mostly done. I think that's been a big change as well. And I think I've rode the wave in that change where I now have an entire project where I have no database.My database is literally github.com and could I have done that as easily five years ago? Probably? I roughly did it four years ago but four years ago as a junior developer. So like that goes to show we're transitioning the way that if you wanted to build something on top of a third party API or whatever, like there's a lot of tools for you to use free and I think there's a lot of VCs and a lot of founders and a lot of open source projects that are really looking at the space and looking at this sort of mock regurgitation of developer tools and how anybody has access to anything. And it's been super fascinating to see that.Jeremy: [00:43:53] Cool. Well, I know you got to get off to a meeting, so I just wanna say thanks for chatting with me today, Brian.Brian: [00:43:58] Cool. Thanks, Jeremy. Looking forward to seeing what comes out.Jeremy: [00:44:02] I hope you enjoyed the chat with Brian. If you're interested in getting an open source, you can check out the show notes at softwaresessions.com. I've got links to Brian's project, open sauced and a link to where he does his Twitch streams. The music in this episode is by Crystal Cola. Alright, I'll see you next time.

Lauren is a Software Engineer on the React Organization's Web Core team at Facebook and was previously an Engineering Manager at Netflix.We discuss:Being empowered to say "no" as a senior engineerStraddling the line between engineer and managerThe programmer's midlife crisisResisting the urge to use clever abstractionsIf you enjoyed this discussion with Lauren, be sure to check out her episode on the Changelog.Music by Crystal Cola: 12:30 AM / OrionRelated Links@sugarpirate_Personal SiteThe Engineer / Manager PendulumShould I Become a Tech Lead?Does it Spark Joy?Changelog Episode - Engineer to Manager and Back AgainDan Abramov's Redux twitter postTranscriptYou can help edit this transcript on GitHub.Jeremy: [00:00:00] Hey, this is Jeremy. Usually when software developers are talking career progression, it moves in the direction from being a software engineer to becoming an engineering manager. And today I'm talking to Lauren Tan who moved in the opposite direction. She was an engineering manager at Netflix, and she recently made the decision to become a software engineer at Facebook. We discuss why she made that decision and the differences between being a software engineer, a technical lead and an engineering manager. We also discuss what it means to be a senior software engineer and the ways that you can increase your impact and your influence in a software engineering role. I really enjoyed the conversation with Lauren and I hope you do as well.Hey Lauren, thanks for joining me today.Lauren: [00:00:42] Hi Jeremy. It's such a pleasure to be here. Thanks for having me.Jeremy: [00:00:45] If we look back at 2015, you're moving from Australia to Boston, you're starting your first senior developer role at the dockyard consultancy. How did you get into this position where you decided that I'm going to leave this country I live in and I'm going to start this senior developer role in Boston?Lauren: [00:01:03] A long time ago. I never really planned to leave Australia, let alone come to America. And I kind of traced his back to essentially how I got my career started in technology where really what started as a hobby creating silly applications then.In fact, one of my earliest introductions to programming was through excel and making elaborate spreadsheets and writing Visual Basic or VBA. It was something that I never really planned to do. The long story short is after college, I started a startup called The Price Geek with one of my classmates.And at the time I was getting really interested in essentially exploring more of this hobby that I had of programming and potentially exploring the idea of turning that into a career. So the year or two that I worked on that startup was really fun. We learned a lot about product development, about the business side of things, how to manage your money and how to get funding and financing.That was all really interesting. And near the end of the startup when we were basically throwing in the towel I realized that I enjoyed it so much and despite the fact that my degree was in finance and not computer science, I enjoyed it so much that I thought to myself, wow, it would be amazing if I could keep programming as a career.So I was very fortunate to get a first job in Australia as a software engineer. And I had started writing a bunch of blog posts and started sharing them on Twitter and on medium. And slowly but surely, I got people reading it. And there was a point where one of the creators of that JavaScript framework that I was writing about got in touch with me to say: Hey, would you be interested in coming to speak at one of our conferences? And of course, I was totally taken aback because first of all, I had never even been to a tech conference at that point, let alone speak at one. So I had totally no idea what I was doing. But I was convinced by them to apply. So I did and I'm very grateful that they did that.And doing all of this essentially, I started to get the attention of some of the people working in America. The CEO of that consultancy DockYard reached out to me and asked if I would be interested in working there. And at the time they were pretty well known in the field of building Ember applications, Ruby on Rails applications. And so I thought it would be pretty interesting to go and work there and learn from some of the people that I really looked up to in that community. And that was the start of my career in Boston. And really it was a difficult decision to move. I think moving anywhere it's difficult but the move from Melbourne to Boston was exceptionally hard because it's a totally different country. It's so far away. My family and my friends would be not even in the same time zone anymore in opposite ends of the world really. So that was particularly difficult. And of course the Boston weather, it's terrible. And part of the reason why I was like, I need to maybe live somewhere else is because of the terrible Boston winter that I experienced in 2015.Jeremy: [00:04:33] That makes a lot of sense how you ended up in California.Lauren: [00:04:36] Right. I was like-- I need to go somewhere warm.Jeremy: [00:04:40] One of the other guests I'm going to have on, Swyx-- he often talks about learning in public, which you were doing with your blog posts which got you noticed. So I think that's good advice for software developers in general that putting yourself out there and sharing knowledge can really make these opportunities come to you.Lauren: [00:05:01] I think it can, but I also want to say that I think that developers that learned their craft during the time that I started I think we were very fortunate in the sense that the web was a bit of a simpler place back then. People would build applications just literally using HTML, CSS, and vanilla JavaScript back then. You might just consider using jQuery or Backbone, or MooTools even.A single page application really wasn't the norm. I think today is a very different world because software development-- I don't know if it's gotten more complex, but I think at least in the world of front end development it's gotten much more difficult to just get started.Not saying that you can't build an app with just HTML, CSS, and vanilla JavaScript. But if you want to get a job doing it, then there is a bit of a higher bar I think. So I will say learning in public can be very helpful. But I also don't want to lie and disguise the fact that the environment has changed.Times have changed and things are getting slightly more complicated and complex to build and that just means that there's a bit of a higher hill to climb.Jeremy: [00:06:18] If you are going to make a site, you have so many options you have React, Vue, Ember, Svelte. There's all these different frameworks and do I use Javascript? Do I use TypeScript? It's definitely a lot more-- I don't know how you'd describe it. Intimidating, I guess.Lauren: [00:06:39] It shows the evolution of how front end development has improved in a way. It's like, it's a mindset shift I think in the industry where previously, like 10 years ago it was still okay to just build what people might call enriched documents. Really documents sprinkled with some interactivity. But these days you're often building interactive applications that warrant a framework like react or angular or svelte or Vue. So I think maybe the problems that we're trying to solve have also changed that warrant more complex solutions.Because I don't think the answer is to say like: Oh, we just need to get rid of all the complexity. The complexity exists for a reason. I think if I had advice for someone who was coming up in the industry, I would say, don't get intimidated by all these different technologies. And honestly, it probably doesn't really matter in the grand scheme of things which one you pick as long as you pick one and then you don't shut yourself off to learning from the others as well.Because frameworks will come and go, but the knowledge that you acquire from using these frameworks will hopefully stay with you for a long time. And so those are much more transferable than knowing every single detail about the React API or something like that.Jeremy: [00:07:56] Yeah, I think that's good advice. And I also wonder, when you started-- you had experience building applications in things like Rails. There are a number of frameworks where you can build a front end using primarily server side code, not necessarily build a single page application. People starting out, is that still something they should look at or do you think they should jump straight to single page applications?Lauren: [00:08:24] I feel like it depends on your goal and hopefully if you're learning to program, hopefully you also have a project or some kind of motivation for learning those technologies. You should hopefully use the right tool for the job. And if you're building something that really doesn't require a lot of interactivity, then maybe a single page application is overkill, even though it might be beneficial for you to learn.So I think it depends on your goal. If your goal is purely just for educational purposes, then by all means, choose the fanciest technology stack and learn away. But if you're actually trying to get a project going off the ground, I feel like it's probably not that useful to bike shed on like, do we use Svelte or do we write our own thing, or do we just use server side rendered templates?I think those are all fun as technologists to debate and think about. But they're just in my opinion obstacles for actually trying to do what you're trying to set out to do. So that's a bit of a roundabout way of saying that I think it depends on your goals. Is it to learn or is it to build something that you know you can get out the door really quickly. And depending on what goal you have, I think my suggestion would be slightly different.I think fortunately, if your goal is mainly just to learn, then any one of those single page application frameworks are great to pick up. My only suggestion would be, again, like not to tie yourself too closely to just one framework, even though one may seem like the incumbent. The one that every company is hiring for and that's fine.Maybe you start there, but don't let that limit you from learning everything else. Because again, like there are a lot of concepts from the different frameworks that often make their way into other frameworks as well.Jeremy: [00:10:17] That kind of reminds me of how when you first started, you were very focused on Ember. And now you're deeply involved in React you don't have to feel like you're tied to just the one you start with.Lauren: [00:10:29] Absolutely. And I think in the tech community there are a lot of these people who say: Oh, you know, don't bother learning a framework. Just learn the fundamentals. In spirit, I agree with that principle. I think that you should learn the fundamentals. But I also agree that actually learning a framework first is not a bad thing. In fact, it helps you.Sometimes you don't need to peel away all the layers of abstraction straight away because that can be very overwhelming. And I think single page applications, there are a lot of tutorials online that you can follow and you can get something working. And that is your basis for starting to then peek under the hood to say: Oh, how actually does that work?Why did I use a component here instead of make a component that does this other thing. I think of it more like the onion of knowledge really. I don't know what a good analogy is but like an onion in the sense that there are layers that you peel away and you slowly understand what the frameworks and the languages are doing.And in fact, I see even today, like my career and the stuff that I'm doing is continually peeling the layers. Maybe today I may not be working on writing an application anymore. I might be working on the infrastructure that powers the tools that allow this application to be made.But I wouldn't have been able to have gotten here if I had not been building applications before. So you go deeper and deeper. But you can't go deeper without a strong foundation. So my advice is start with what's comfortable. Start with something that's easy to learn and use that as a foundation for going deeper into the technologies and the areas of programming that you're interested in.Because maybe you'll find that front end development is not for you and maybe you'll realize that actually I prefer back end development. And that's perfectly fine. There's no one path in this industry which is pretty cool. So I would say keep it broad, learn as much as you can, and then follow what interests you and what excites you.Jeremy: [00:12:35] A lot of people when they're learning, it's hard to stay motivated unless you're building something that you can see. I think in that respect if using a framework like React or Phoenix or Rails-- if it's going to get you to the point of being able to see something working that will keep you motivated, keep you moving, then it makes a lot of sense to start there.Lauren: [00:12:57] Yeah. I totally agree. There are a lot of great concepts in these frameworks that will apply in other areas as well. Again, whether you use this framework or that framework or no framework, there are still a lot of programming patterns that you can learn. Which is why if I were to start learning how to code again, I would still start from the same place. I would still pick a framework and go with that and then figure out how it works.Jeremy: [00:13:22] I want to take us back to your time at DockYard. I believe your title was senior developer. What do you think made you a senior developer or did you feel like one at the time?Lauren: [00:13:34] I think that's a great question. I think my general viewpoint on this is that I don't think we have agreed upon standards for what we deem senior, and I don't want to be the gatekeeper of what determines someone as a senior engineer. But I certainly didn't feel like a senior developer, at least in my definition of what I thought a senior developer should be at the time.And at the time, I think I had a fairly naive impression of what a senior developer was and my thought was all about the senior developer is essentially the person who is the best programmer. Who knows every single API by heart. Is a genius at all the internals of every library that they use. And they're just technical, technical, technical chops.But interestingly, the more I worked there and the more I interacted with others, people who had the same title. The more I realized that my viewpoint of what makes somebody a senior developer or engineer was totally off. And today I feel like the technical chops are just a small part of the skillset and the tool set of a senior engineer.And if that's the only thing that you're bringing to the table, then-- it's not necessarily a bad thing, but I think you're doing yourself a disservice by not flexing those other muscles. Which is a huge lesson I learned when I took on the role of a manager. But yeah, I definitely didn't feel like I was a senior developer back then.Maybe today I feel more like a senior developer but I think everyone has this different definition. But at least in my definition, I think I feel pretty confident in saying that. Yes, I am actually a senior developer.Jeremy: [00:15:22] So what would you say were the key differences then? Because you were saying that it's beyond just the technical aspect, but what are those pieces that make you feel comfortable saying that you're senior now?Lauren: [00:15:36] First of all, it was the mindset shift for myself that I can't pinpoint a specific point in time where it happened. But I certainly recognize it today where I essentially no longer feel the need to rush into writing code. Whereas in the past the moment you get a project, you're like-- all right, I need to write this proof of concept. You just focus on writing code and that's all like for you, your impact is all about the raw output of your keyboard essentially. And that was the wrong mindset to have because what I learned over the years in working on different projects and in different companies is that oftentimes the most impactful things were not actually the result of code.It could be a conversation that you have with your customer or your client to find out the assumption that you had made was incorrect. And if there's something you can ask in a question and you can get an answer to in 30 minutes or you could spend days and weeks building something and then you bring it back and showing it to them and then they tell you why this is not what I wanted.I learned that lesson very painfully because I was one of those people who would just rush into writing code. My viewpoint was if I don't have to talk to anyone then I'm succeeding. But that was totally incorrect and it was a tough lesson to go through, but I think a lesson that I sorely needed. It's definitely affected the way I operate today.I think today I don't shy away from talking to people. In fact, I will go out of my way sometimes to have conversations with people even when it's going to disrupt the time that I enjoy of writing code because I know how impactful conversations like that can be especially when you're trying to do things that are maybe not very certain or get more context or even prioritize things.I think another aspect of being a senior developer is knowing when to say yes to things and when to say no to things. I don't think there's a decision tree for when to say no or yes. I think it's very much based on intuition and your understanding of the context and the problems you're trying to solve. And also organizational challenges that may happen, but prioritization is something I feel like we don't often talk about. Because again, if your mindset is all about my impact is based on the pure output of my coding, then you're not going to be in a position where you can say, hold on-- before I go and just jump straight into writing some code, let me actually speak with my manager and challenge the idea of like, wait, hold on. Is this actually the best way to do it? Do we even need to write any code to solve this problem? Maybe it's an organizational problem. If I were to distill it down I think it's the realization that my output is not just code anymore.And I think that for me was the point where I could say to myself: I am a senior engineer. Even though maybe I'll join a company and not immediately be an expert in all of the proprietary tools that they have, which is expected. How can you, how can anyone be an expert when you haven't used those technologies. And there's certainly no expectation I think that any company you join you will be immediately the foremost expert on something that they do within the company. So the thing that you bring from position to position or project to project is really those core skills of your understanding of fundamental programming, things that are transferable, but also the organizational chops that are also equally if not more important than those foundational skills.Jeremy: [00:19:34] Earlier in your career like at DockYard did you feel like you had the authority to ask those questions like to challenge your manager, go directly to the customer. Was there anything that was stopping you from doing that then?Lauren: [00:19:34] I think there were a number of challenges for sure. The agency relationship with customers makes it a little bit difficult because as an agency or a software consulting company you are not always in a position to question or challenge the client because at the end of the day, the client's paying you to build something very specific and sure, maybe you can point out the flaws in their plan or deficiencies but ultimately the contract that you signed states that you have to deliver a certain product by the end of a certain timeline.That was probably the systemic challenge there, but I think I also didn't feel empowered to do that anyway, even if that wasn't the case. I think there were a number of challenges, but certainly I was, maybe not having the right examples either too. Like for example, maybe if some of the more senior people in the company were doing that and setting a good example, then I think others would have followed as well. But I don't really feel like we were necessarily in a position to do so.So I think that made it more complex. And I think once I started joining a companies like Netflix or Facebook where I currently work. I think that dynamic and the expectations also changed because now I'm in a position where my job is not just to blindly output code just because someone said so, but to be a problem solver.And so I think it's a very different relationship. And I think if you are in a software consulting role or a software agency role then a lot of what I'm saying may not necessarily apply because you're not always in a position to go and question your client or customer.Maybe you might find a customer that is awesome enough to let you do that and be receptive of the feedback as well. But that's not often the case, especially on projects where it's like super tight deadline, just deliver something in two to three months. So context is everything.Jeremy: [00:21:42] Yeah. That's an interesting point about how when you're working at an agency, you're ultimately telling the customer: I will do this thing for you. It's written down in a contract, whereas for a more traditional company, it's really dependent on the culture of that company and maybe that's something that when you're interviewing or learning more about that company, you would want to figure out how much agency or how much control will you have as an engineer being in that company.Lauren: [00:22:14] Yeah. I think that's a great point. And it's a question I will often ask when I've done interviews as an interviewee in the past is ask people, especially the engineers on the team that are interviewing me for examples of times where they were empowered to say no to certain things. And I think the way that the answer those questions will tell you a lot about the culture of the company. I often find as a meta point asking for examples in interview questions to your interviewers are actually always for me, very helpful in trying to reverse engineer what the culture is really like versus what it's advertised. Because sometimes, and it's not ideal, but there's often a disconnect between what is stated and what really happens. And I think there's no better way to learn that than to ask for examples.Jeremy: [00:23:08] What are some questions you would ask to reverse engineer that?Lauren: [00:23:14] Oh, I have so many. The things that come to mind are, like I just mentioned-- can you share an example of a time where you were empowered to say no, or, tell me about a time where you disagreed with a manager and, you were given the autonomy or freedom to go and explore that solution that you were proposing. The things along those lines where it gives the interviewer a chance to show that the culture is truly what they say they claim it is. If I think of more later I'll bring them up as well. Or I can share some thoughts in writing later as well.Jeremy: [00:23:52] When you ask those kinds of questions, reading people's body language or just the way they respond you can infer a lot of information, beyond what's being said.Lauren: [00:24:06] Yeah. Especially in the times where maybe that person is unable to give you an example and instead they'll talk about it more generally, which for me is a bit of a smell. It means that maybe they don't practice what they preach. And I think what you just said is very important.I think the way that the person answers that question tells you a lot, even if you don't come out right and say like: "No, I am not empowered to say no." I think it just tells you a lot, like just how the person answers what they say, what they don't say, is also important. But I mean, I also say that with the somewhat small caveat that there may be a chance that maybe that person just hasn't had that occur to them. Maybe not that they have not been empowered to say no and they just have never had to say no. It's not necessarily a bad mark, I would say. So a lot of judgment applies to how you interpret those answers. But again, they can be so subjective. So I don't know if there's a clear cut way to say like, "Oh, this company is definitively bad or good."And then I think to make things more challenging, depending on the size of the company you're going into a lot of the culture will really depend on the immediate team that you're on. And in fact probably the manager that you have is a bigger indicator of the culture than the general company-wide culture. So it really depends. But if you have the hiring manager in your interview panel and you're given time to ask questions, then I would definitely bring lots of really hard questions and really get a sense of whether this manager will be the right person to support you in your career.And, sort of going off on a tangent here, but I think my own experience being a manager has also taught me that there are lots of different kinds of managers and it's not like one is better than the other. I think there is some kind of matching that you have to do on your own as you understand what kind of support you need.For example. If you're still early in your career, maybe you do need a manager who is very technical, who can give you a lot of technical feedback, that can help you grow in your career, at least technically. But maybe once you get more senior in your career, maybe that kind of manager would no longer be as beneficial to your career as for someone who is earlier in their career.And instead, maybe you might look for someone who is more of a sponsor. Someone who goes out and finds really difficult problems and says, "Hey, can you solve this?" And maybe that's what you need in your career. So I think spending the time to introspect and think about if I had the perfect manager, what would they be like?And then go backwards from there and say what questions do I need to ask in order to determine if this manager would be that person? And obviously it's not perfect, you can never really know for sure until you start working with them. But it can at least give you more confidence if you're interviewing at lots of different places and you're trying to make a decision on where you ultimately land.Jeremy: [00:27:17] Yeah. That's an interesting point about finding a manager fit. After DockYard, you moved to Netflix as a senior software engineering lead. In that role, what were you looking for in a manager?Lauren: [00:27:34] So I joined Netflix, not as a lead per se. It wasn't an official title, but more so unofficial title after a period of time. I guess LinkedIn probably doesn't capture that very accurately. In terms of what I was looking for in my manager at that point when I had just joined the company, I did think I was at the point when I joined Netflix that I really wasn't necessarily looking so much for just raw technical chops.I wasn't looking for a manager who was a better coder than me. I think the thing that I was excited about was Netflix's culture of freedom and responsibility and context, not control and all those things that they write on their culture memo. And I can actually safely say that pretty much most of it, if not all of it is true and they do apply in practice.So I was very excited about going into a company where the culture was so different than anything I had ever experienced. And I wanted to learn what I had started to think about would define me as being at the level that I wanted to be. You know, like not someone who is only good at programming, but also someone who brings a lot of impact to the team that they work on, whether it's a contribution in the form of code or contribution in the form of an architecture document, or even a comment or some feedback that you've given someone. Because when I first joined, I didn't feel like I was in that position yet.But about a year and a half, I don't remember exactly I did start to feel like I was getting the hang of the culture and also the technology at Netflix where I was then very comfortable when my manager came to me and said: "Hey, would you like to be the lead for the team?" to say yes.I was like, yeah, absolutely. In fact, I already felt like I was operating like a lead. So this was more just a recognition that I was already operating at that capacity. So I think that my manager at the time was definitely very supportive and they looked out for opportunities for me.And they were never really prescriptive about certain things like they may have had different opinions from me from time to time but they weren't afraid to say, you know what go try what you think is right. And then let's compare notes and see what turns out to be better.And that was always very encouraging because it creates this almost psychological safety of going and trying different things that people don't necessarily immediately agree with. Like if you can prove that something is better with a prototype or a document or whatever it might be that you're given the autonomy and flexibility in the space to go and explore that and then come back and say, you know what? This was either a good idea, or a bad idea, or unconclusive. But I think that was, for me, something that I really enjoyed about that part of my career.Jeremy: [00:30:36] And it sounds like your manager gave you the opportunity to explore having more influence, having more control over the types of work you were doing, and how you were doing it and at that time in your career, that's really what you were looking for.Lauren: [00:30:55] Yeah. I think I don't recall when, but there has definitely been times where I've had what I would call the programmers midlife crisis. Where you start questioning what you're doing and the way that you've been doing things and the purpose and starting to look up from the keyboard and like, hold on a minute.I can get this project done, but is this really the right thing to be doing? And I think the more senior you get, the more that urge will come to you and you start thinking more about, Hm. The moments where you say to yourself like, hold on a minute, something feels off.And I think the turning point for a lot of people will be when you'll start turning those thoughts into action and instead of just saying, hold on a minute in your mind and then just continuing anyway, you start actually going forward and talking to people and say, hold on, here's something that doesn't sit quite well with me. Let's talk about it. And in fact, I think one of the things I started to recognize once I was operating in that lead capacity even though maybe I didn't have the title just yet, was that actually I was spending less time coding.And initially it felt kind of awkward. I was like, why am I in all these meetings? Why am I feeling like my output has dropped a lot? And it was true. If the only output that you're measuring is my code, then it definitely dropped quite a lot. But in terms of the impact I was having on the team and the projects that I was on I think definitely outweighed that. It wasn't a net loss because oftentimes when you have someone who's operating in a lead capacity, it means that in a way, they're giving away those problems that are maybe more difficult to solve. And allowing others to learn about them and, not hogging all the difficult pieces to themselves which sometimes the tech leads might do instead of giving opportunities to others to grow, which is actually a responsibility for a tech lead. So I think going back to your question of what did I need from my manager at the time, I think it was definitely being put more so than a lot of other things it was being put in an environment where I could really flex those nontechnical skills and understand, and almost in a way, create the environment you know? Like if a manager is like a gardener, then creating the right conditions in the environment so that I could not just thrive, but also evolve and grow and, broaden my branches. It's a weird analogy.Jeremy: [00:33:38] And, we've stepped around it but I think the the title of someone being a lead to a lot of people is a little fuzzy. Some people think a lead is the same thing as a manager. And it sounds like what you're saying is in your case, a lead was someone who is able to ask questions to figure out what should actually be built. They're able to decide who should work on these things after you've decided what needs to get built, and we haven't mentioned this, but potentially help the people who are building these things if they get stuck. Would you say those are the three primary things that a lead does?Lauren: [00:34:21] Um, at a high level, I think that's pretty accurate. To be a bit more granular, I would say it also depends on the kind of tech lead that you want to be, or, or maybe another way to put this might be the tech lead that your team needs. Because the truth is, at least from my perspective, just like managers have different archetypes. I would also say that tech leads had to have different archetypes and it really just depends on the kind of project that you're working on. I would say though, as a minimum for me, at least from the technical side of things, yes. Even though I wasn't writing a lot of code anymore as a lead, I was still reviewing a lot of code.In fact, I think I would probably say I reviewed more than I wrote code. I think that was also part of the dawning realization that-- Hey, you know what? You can contribute in forms that aren't just you writing the code and then slowly universe expanding of like, Oh, if I step back just a little bit, I start to see the forest of what impact as an engineer is. And it was the realization that I had been only just focusing on this technical tree and not growing all these other skills that are also really important. So I think tech leads are typically the people who are seen as the best engineer who gets pushed into the lead position.But I would say that tech leads are interesting in the sense that you're not a manager, you typically don't have reports, and you don't have any authority so to speak, over anyone. So all you really have typically, I feel is the influence that you've earned throughout your career in that company.And that kind of social capital, if you will, that people will start to listen to you because you've been around, you know your way around, and you've proven that you can handle large projects and things like that and grow other engineers. So I think for me being a tech lead in some ways can be actually more challenging in some ways than a manager because it's blurring the lines I guess. I think as a tech lead you're in this awkward gray zone between engineer and manager and you're not quite one. You're not quite a manager. You're obviously still an engineer, but you're in a position of greater influence and greater not really authority, but more respect typically is given to you.And so you're in this awkward position. Where it again, it comes down to what your team needs. And maybe like for example, if I was to join a new team and I was the tech lead for that team and if it was a team of one or two people, then obviously the expectations and the way I would do my job would be very different from me joining as a tech lead on a team of 12 engineers. It's a very different set of variables that you have to learn how to tweak. And again, it just depends on the makeup of the team as well. So like if I joined a team of 12 very junior engineers then also my approach would be very different versus if I joined a team of 12 extremely senior engineers. It all is very fuzzy. I don't think there's one, there's no one way to do your job as a tech lead, or as an engineer, as a manager. And maybe it sounds like a bit of a cop out answer, but I do think that a lot of questions can be distilled down to the age old answer it depends. Obviously just saying it depends and nothing else is a bit of a cop out. But I can say that there are different circumstances and some may require as a tech lead more involvement from you at the architecture level. Maybe some less or maybe some where you, instead of worrying too much about architecture, maybe the problems are more around organizational challenges or headcount or constraints I would imagine things like that the tech lead should be doing as well. Like that example I shared with you of joining a team and being one of two engineers. Maybe one of the first things of my job would be to point out to leadership that-- Hey, I've just joined this project and it's clearly very ambitious, but there's only two of us and the deadline-- that the timeline that we're gonna work on is way too unrealistic.So I actually need to campaign my manager to say, this is why we need another two engineers or one engineer on this project. And so that's why I think it's a bit tricky. Cause it really depends on the team that you're on.Jeremy: [00:39:05] That's a really good point in terms of the size and the experience and the actual project that you're tackling. I think that's why people have so much trouble understanding what it is a tech lead does. Because from what you're describing, it's a completely different job from person to person.Lauren: [00:39:24] Yep. Yeah. It's very context dependent because you're straddling the line between manager and engineering and individual contributor. And so you have to sometimes wear the manager hat even though you're not a manager, and sometimes you have to wear the engineer hat.But I think knowing when to switch hats is really important. And if the expectation that maybe someone else has set for you is that you are wearing the technical hat most of the time, then that's the expectations that you work towards. But I think for most companies, especially the bigger ones, I think there is an expectation that you also wear the project management hat, the organizational hat where you go and raise problems like that as well.Jeremy: [00:40:10] So we've been talking about tech leads and managers and how the role of a tech lead is so fuzzy. After your role as a lead at Netflix, you moved on to becoming a manager.What would you say are the key differences between being a manager and a tech lead? How did your job change, how did your role change?Lauren: [00:40:33] So I do want to make the distinction that even though I said earlier that as a tech lead, you're in-between a manager and an individual contributor. I do want to say there are a lot of manager specific things that tech leads don't get exposed to.It was definitely a big jump even going from tech lead to manager, let alone like as a non tech lead engineer to manager. And I think a lot of those challenges were in the forms of essentially problems which I never really thought about.Like, maybe I would have said-- we need more people on this project. But I wouldn't have then gone on to say, all right, I need to spend the next three months looking for the perfect hire to join the team. Because that was the job of the manager, to really think about the people and the conditions of the overall team that they're supporting.Whereas as a tech lead your sphere is slightly more constrained to maybe a project or two purely more in terms of the results of said project, whereas I think as a manager, the expectations become more at the organizational level and your success is really determined by the work that your team ultimately does or doesn't do.Maybe it sounds kind of subtle when I describe it that way, but I will say that it was definitely a very different job when I went on to become a manager. The first of which is I think a very false conclusion that I may have harbored a long time ago and I think a lot of people share the same sentiment as well. Which I want to go on record and say I kind of disagree with that. And that view is essentially that becoming a manager is a promotion. In some ways, maybe it is a promotion, like maybe financially you might get paid more and you might have more opportunities to have certain kinds of impact depending on the company that you're in.But I will say for the most part having that mindset that management is a promotion is not the right one to have because I think it disguises the fact that when you go from engineering to manager you're basically going from very senior engineer who was very good at their job to going to become a baby manager who knows nothing about their job.So it is very different. They're like two definitely distinct tracks and all the skills that you've used. Like, 99% of them, you've used to be successful as an engineer are probably not going to translate that well to being a manager. Like you're not going to be expected to write as much code or to even do code reviews instead your role is really more to ensure that you have the right people on your team, and also the right environment where those people can thrive and do their best work and achieve the goals that have been set out for your team and even shape those goals that your team should be working on. So it was definitely a very different career change.And I think even though I had expectations going in, that it'd be very different. It was a totally different experience doing it, if that makes sense. Like I was expecting one thing. I knew it would be different, but I didn't realize it'd be that different. And I remember as a manager, I was spending so many hours just looking through LinkedIn or reaching out to people on Twitter, and asking them like, "Hey, would you want to come work on my team?"Because as a manager, your biggest lever for impact is getting to pick who is in the team and it may sound like a very-- it may sound simple, right? Like you're just hiring, but I would say it's actually a very very high leverage activity. If you find that person who fills out a gap in your team where maybe there's a certain skill set or a certain technology, skill or organizational skill that your team doesn't have that you want to have, and you're able to fill that position and not just do that, but keep them there as well for-- well not keep them there but, to create an environment where that person is happy staying for awhile, then you've really done a great job because now you have a strong, solid dream team that has the capabilities of doing awesome work that you need, that you want to achieve the vision that you have for your team. And then you also have to balance that with the often difficult work of what is often called talent retention, but I don't really like that term because I don't think so much about retaining people because that sounds to me like they're constantly trying to escape and then you're just trying to hold them back.I think it's more about creating an environment where people are attracted and they want to stay not because they're handcuffed. But because they choose to stay, because you're a great manager and the team is good. That the work is impactful. If anyone listening is also going through that transition or you've just become a manager, I'll say that I think for me the biggest challenge I had to overcome in the initial couple of months of making this transition was really understanding that it was a completely different job and then changing all the things that I did for that new reality and not trying to go back to the things and the skills and activities that I had been relying on as a tech lead.Jeremy: [00:46:10] You gave a few examples of what a manager does that wouldn't go to a tech lead like hiring and if your team needs more resources, making that pitch to get more money, and also creating an environment where people want to stay there. Are there any other specific examples for the types of things that would only be a manager's role versus something that a tech lead would do?Lauren: [00:46:36] Yeah. So this is not going to be an exhaustive list for sure, but I can at least point out the things that are not immediately obvious, not unless someone explicitly says it. But I will say actually, depending on the company, I think one of the biggest jobs of a manager is actually the flip side of hiring, which is firing.And that's a really tough one. You never go into a team or hire someone or work with someone on your team expecting that you'll need to one day fire them. But as a manager, that is the thing that you think about on a constant basis, not because you just like firing people just to fire people or again, I don't know, maybe at certain companies where they do things like stack ranking and there's an expectation that, yeah, I don't know. I've been fortunate not having worked in companies like that. But if you are a manager, I would say that you're often the person who does a lot of unglamorous things like that. To me at least, it seems unglamorous to me. The hard work of recruiting and hiring and speaking to candidates and selling them on your team.And if you do write code it's most likely going to be the very boring parts of the code base. Like adding tests or writing a little script that does a certain thing. So you're not going to be working on those things that you thought were exciting or the things that may have even attracted you to software engineering in the first place.So very different job. And, there is going to be so many other things that you may not be aware that managers have to do. Like, I don't really like how people will phrase this, but a lot of managers will say like they provide air cover or they shield their team from shit.I think there's some truth to that in the spirit of what that means. But I think there are obviously different ways of approaching it. And personally for me, instead of thinking it like that, like I'm shielding my team from shit, I think it's more about maybe there is shit coming towards our way. But my job as a manager is to also not make that shit before it comes to my team, if that even makes sense. And so that often means talking to people in different positions, different parts of the company, people who are higher, like a VP or a director and convincing them that this path isn't the right one.And the truth is that a lot of individual contributors won't see that. Not because they are ignorant but because if your manager is doing a good job of that, you just don't see it. And sometimes managers can be a bit flippant, I think. And to say that, Oh yeah, I shield all the shit, so you don't have to, which I think is, again, like in spirit it, it captures the outcome of what, what that is. But I think it also doesn't quite accurately portray how the manager goes about doing it cause there are many different ways that yes, maybe you can just shield and keep your team unaware of everything, but that's not necessarily a great way to run your team as well because your reports will not necessarily trust you very much if you're always being very dishonest and like not telling them the truth, because of your desire to shield them from shit.Instead maybe the better approach is to let people know that. You know what, there are rocky things that are coming. They're things that the company that we work at doesn't do so well, and that's okay. We'll figure it out. But not to completely hide it, I think. I think that's the part which I am not a big fan of.It's a bit of a cop out to me if the manager just keeps things from their team because of that mindset or because of that belief that by doing so, they are helping your team, when in fact, I think it's actually making the team worse off.Jeremy: [00:50:25] That's an interesting perspective because ultimately, if you are shielding your team and the things that they're being shielded from are just shifting elsewhere in the organization, that's not really solving the root problem.Lauren: [00:50:39] Right. Yeah, and I think it can also be very powerful for managers to point out areas which need help. And then instead of feeling like the manager has to solve all of those problems I think we-- we talk a lot about the management parts of the job, but not the leadership parts of the job.Leadership is really more about influence and the way you conduct yourself and how others perceive your behavior versus management, which is more like-- I think a role that you play. So things like hiring and firing are obviously the role of a manager.But getting people excited about a vision and getting people to do certain things even though you're not explicitly bending their arm to do it is a part of the job that is not often talked about or even taught-- like how do you do that? It's not something that you can just read a book and do. It's something that over time and trial and error and maybe some intuition you build that up over time.A realization that I had over the past two and a half years when I was a manager was that leadership is not solely within the domain of the manager. It sounds silly to say this, I had to be a manager in order to realize this. But it wasn't as evident to me until I became a manager that, Hey, hold on. There's a lot of things that I'm doing as a manager that I didn't have to be a manager to do these. So when I started to think about the different parts of the job that I was doing, I started to realize that, hold on, there's the parts that I like, right? A lot of the leadership side of things I really enjoyed.And then the other parts which I maybe didn't enjoy as much. And I realized like, Oh, hold on. Actually I don't necessarily have to be a manager to practice these skills. That was actually the realization that I needed to maybe go back to be an engineer again.But I certainly don't regret the time I spent as a manager cause I was exposed to so many different kinds of problems that I'd never ever had to face as an engineer. Hiring, having to let people go. Dealing with the sometimes unreasonable demands of different organizations that we were working with and balancing that all.And another thing that maybe managers don't talk about is oftentimes people will come to you with problems that you can't solve. And these are maybe personal problems, emotional problems. And if you're a very empathetic person, then I think the job of a manager gets really difficult because people come to you with lots of problems that you can't solve. And if you're an engineer, you probably want to try and solve all the problems and it can be very frustrating.I guess I'll sum it up all by saying that being a manager is a totally different job from being an engineer, even a tech lead. It's totally different. It's not a promotion. I don't consider it a promotion. And I think if anybody chooses to do it, I think you learn a lot and hopefully you enjoy that transition as well.But personally for me, I didn't enjoy it. That doesn't make being a manager bad. It just means that it wasn't for me.Jeremy: [00:54:00] And now that you're at Facebook as a software engineer again, What's the thing that you enjoy most about being a software engineer as opposed to being a manager?Lauren: [00:54:11] I think when it came down to it. It was really a reflection of why was I in the tech industry in the first place. And I think the simple way to put it is that I mentioned earlier at the start of this conversation that programming started out as a hobby for me. And it was something that I would spend all my free time just working on.I would have these shower thoughts essentially of programming. And I realized I've been so fortunate that I was able to turn something that was purely a hobby into a full time career. And when I was reflecting at the end of 2019 about the next couple of years of my career, I did really start to think that there was a lot of programming in being an engineer that I really missed.And also me realizing that other part, which I mentioned earlier, which is that there's a lot of things I was doing as a manager that they were not things that only managers could do. But you may have to become a manager to learn the skills, which just sounds kind of weird.But I think it was that realization that Hey, one, I can go back to do what I love, which is programming. And two, I can also bring back all these lessons that I've learned as a manager and basically supercharge myself as an engineer and be so much more impactful. Not because I'm going to write all the code and solve all the problems, but because I know how to inspire. I know how to influence. I know how to communicate. I know how to get things done and get other people to help out with those problems. And I think that was for me, the realization that I could have my cake and eat it too, I guess. And I think that I'm very fortunate in that.I think at Facebook they think very heavily about career paths as an engineer, as a manager. And I think the company does a pretty good job at stating that one is not superior to the other. In fact, there's more or less an identical leveling track for engineers and managers and also very similar in terms of compensation.So there's not a penalty for you if you become an engineer. It's not like you're going back. It's seen more as you are just hopping over to the other parallel track. And one of the blog posts that I want to call out here that really helped me think about my career this way is a blog post by Charity Majors called the engineer/manager pendulum.And she does an amazing job of articulating this hidden career path of jumping between the engineering track and a management track every couple of years. And she does a way better job than I think I can to explain why it's an interesting career path to take, but it certainly inspired me to start thinking more critically about what I wanted out of my job.And then finally mustering the courage to go and interview again because I don't actually know anyone that I can think of who actually enjoys interviewing. I don't. I think it's one of those evils that we put up with. So there is some courage you have to muster up often just to interview and go look elsewhere. But I think her blog posts really spurred me to take action on it.Jeremy: [00:57:34] The interviewing problem is-- that sounds like maybe a job for a manager?Lauren: [00:57:43] Yes. I think, yes, it is part of the manager's job, but I think as engineers, we can also do a lot to at least point out the problems.Maybe we're not the ones to fix them, but we can at least say, Hey, this interview panel that I'm on-- I've looked at the other interviewers on the panel and you can call out things that aren't quite right, that don't sit right to you.Maybe the panel is very undiverse or maybe the interview goes on for like eight whole hours. There are things like that you can still do to influence that process and even influence the questions that get asked. I haven't been a part of an interview panel here yet, but if I understand it correctly, I think that engineers have a lot of influence over the kinds of questions that set a standard for the different interviews that we have.And so that's one way as well, to have a lot of impact and influence over the interview process. And make sure that the questions that we're asking are relevant, realistic, but also ensures that we keep that standard of engineering quality that we want, which is always a fine balance to strike.I could probably talk to you for another whole two hours just on the topic of interviewing, so I won't go into that right now.Jeremy: [00:59:01] Yeah. It's definitely something that everybody has an opinion and everybody agrees it needs to be better, but for some reason we as an industry just haven't gotten there yet.Lauren: [00:59:12] I think my short answer to this is that I don't think there is a perfect solution. Which is why we haven't as an industry adopted something that's better. It's a process that is very lossy and there's just no way to really tell in a short frame of time what a person will be like working on a job.And there are many ways to solve it. None of which I think is better than another. So that's all I'll say about that topic. Don't get me started.Jeremy: [00:59:43] Yeah, I think that anybody listening to this maybe the big takeaway would be regardless of what your role is, even if you are just a regular software engineer, look for what are the places where you can ask questions, whether that's what type of work you're doing, whether the technology you're using is right, or do you have the right people to do it?What are ways that you can really improve your team situation without necessarily having to change titles.Lauren: [01:00:16] Yeah. I think that's a great way to put it. The way I would try to summarize my learnings over the years is really that it comes down to ultimately for me, it all stems from this root. And I think the root of this is all of that. Realizing that your job is not to write code. Code is merely a side effect of your job.I think your job is really to solve problems and there are many ways to solve problems and I think realizing that is, to me, step zero in terms of growing more senior in your career. And, the other thing I'll say to that is also as you get more senior things will get more ambiguous. And you have to learn how to deal with that uncertainty and ambiguity and accepting that sometimes there isn't an answer.And that's okay. I think those are the two big lessons that I've learned.Jeremy: [01:01:10] That's interesting because I think as engineers, a lot of people feel like as they learn more things will get less ambiguous. But it sounds like as you progressed in your career, things are actually getting more ambiguous and that's how you know you're progressing.Lauren: [01:01:24] Exactly. Yeah. I think even in the code I write. Cause like I think you can see it sometimes. I've seen this in myself as well. When you're not a junior engineer anymore but you're not really senior and you kind of know enough to be dangerous and you start dreaming up these-- I'm very guilty of this in my past of writing these weird abstractions that you think will save you a lot of time. But when you look at them in a couple of months, you realize this was totally the wrong abstraction to have picked and it's actually slowing the team down.That is often because again, you're trying to feel your way around and explore and learn, and write better code in your mind. But I find myself these days, it's like trying to write the simplest possible code and delaying the point of abstraction as much as possible and writing a lot of comments about all right-- This could be better, but I'm not gonna make it more abstract right now because this is just a one off case and we don't actually know for sure if it's going to happen again. So that's, I think, the part of recognizing the ambiguity of things. And there are a lot of things that have subtly changed about my behavior.I used to be all about talking about best practices and talking about, Oh, this is an anti pattern, or, so and so said, we shouldn't do it this way. Or you try to read the tea leaves of someone's tweets into Oh yeah, Dan Abramov says, don't do this. So this is now law and we cannot break this law. But I think a painful but necessary part of growth I think is realizing that nothing is really an absolute, only the Sith deals in absolutes and being comfortable with that. Like, again, I was saying like there's often maybe no best answer, but picking the right tool for the job, the right solution, it takes a lot of patience and communication together with your team.Jeremy: [01:03:20] For sure. Yeah. Dan Abramov's example is actually really funny cause he is the creator of Redux, right? And he has this tweet where somebody is describing like how somebody put Redux into their application because Dan said to do it and he replies to the tweet and says this is the reason I'm going to hell.Lauren: [01:03:41] Yup. Yeah. Dan Abramov is a really smart person and someone I really enjoy working with. I think it's all part of our growth of realizing that the things that maybe we all believed were best practices a year ago are probably now anti-patterns, which is why I just shy away from saying this is the best practice and we must do it this way. And taking a more case by case basis to things. And again, this all ties back to being comfortable with ambiguity, right? Because if you don't have these laws, so to speak. Then you're introducing a lot of ambiguity in your code because now maybe people have a lot of uncertainty about, Oh, do I use this in this situation or that?And instead of you saying, Oh, you should always use this thing. You're now saying, right, let's evaluate it on a case by case basis. And that's okay. Maybe it's going to slow it down a little bit but in the long run, it actually makes us faster and more resilient to change. Especially if product requirements change and suddenly all the abstractions that you dreamed up are now totally irrelevant.It's a very interesting industry to be in. I think software it's, it's changing all the time. The way we build software also has to reflect that. And instead of trying to build these very rigid architectures and constructs-- which maybe in certain scenarios are warranted.Like if you're writing code that will never be updated for the next 30 years, then it probably makes sense to get it right from day one. But if it's something that's constantly being improved and evolved, then maybe you don't, you don't jump into pouring the concrete where the concrete doesn't belong just yet.Jeremy: [01:05:22] Yeah, I think that's a good note to end it on. Where can people follow you?Lauren: [01:05:27] The best place to follow me will be on Twitter. My handle is sugarpirate_. You can also follow me on LinkedIn, or add me on Facebook but Twitter is probably your best bet if you're trying to get ahold of me.Jeremy: [01:05:42] Lauren, thank you so much for chatting with me today.Lauren: [01:05:44] Yes. And thank you Jeremy. It was really fun talking to you. And see ya everyone!Jeremy: [01:05:47] That's it for my chat with Lauren, You can get show notes and a transcript for this episode at softwaresessions.com. And if you enjoyed the show, let someone else know about it. The music in this episode is by Crystal Cola. See you next time.

Swyx, a senior developer advocate at AWS and author of The Coding Career Playbook, discusses getting help without a big following, remixing and summarizing others' work, creating Friendcatchers, betting on technologies, and his new book.

Philip Kiely is the author of Writing for Software Developers and has written for companies like Smashing Magazine, CSS-Tricks, and Twilio.We discuss:Doing research beyond the first page of search resultsWriting e-mails people will readThe "get it done" and "learning" modes of readersMaking articles easily skimmableLong form articles vs Stack OverflowPromoting and launching a bookBorrowing the reputation of othersRelated Links:Writing for Software DevelopersPersonal Site@philip_kielyStripe documentation example (Complete code sample with switchable frontends, backends, and walkthrough as you scroll)Go by Example (Straightforward samples readers can copy/paste)Tailwind UI (Sustains Tailwind CSS open source project by providing prebuilt components) Music by Crystal Cola: 12:30 AM / OrionTranscriptYou can help edit this transcript on GitHub.Jeremy: [00:00:35] This episode, I'm talking to Phillip Kiely. He's the author of the book "Writing for Software Developers." We usually talk about code on this show, but knowing how to communicate with developers is just as important. We discuss topics like making articles easily skimmable and writing with a specific audience in mind as well as crafting emails that people will actually read.We also talk about his experience creating, promoting, and launching his book. And what he learned from his many interviews with people like Cassidy Williams and Patrick McKenzie. If you want to get better at writing or you've ever considered launching a product, I think you'll enjoy this conversation with Philip.Phillip, you recently graduated with a computer science bachelor's from Grinnell College and I'm sure you did a lot of research. When you have something new that you're trying to learn how do you get started?Philip: [00:00:48] As a developer, I spend a lot of time getting error messages while I'm coding. The first thing I do whenever I get an error message is I just plop it into Google, see what comes up, see what people are saying on Stack Overflow, GitHub issues, that sort of stuff.I also have a background in journalism. Having worked at the Grinnell College Scarlet and Black newspaper I have a lot of experience going and just talking to people doing interviews. And that's why that felt like a very natural thing to include in the book.It's also something I've been fortunate to do in articles. For example, I wrote an article in Smashing Magazine about remote part time teams using agile development methodologies. As part of that, I did a quick email interview with one of my professors at the time as well as a different professor who had written the book that we were using as the textbook in this class about software development.Bringing in those outside perspectives allowed me to write an article that was so much better than anything I could come up with from my own experience. So really when you're looking for things to use in your own writing look broadly and don't just look at the first page of Google results.Ask specific questions, whether it be of a search engine or of an experienced person in the field. And then cross check that information against each other to make sure that it all matches up and presents the information in a way that's orderly and meaningful to the reader.Jeremy: [00:02:19] The people you're reaching out to most likely don't know you and you're asking them for advice or for an interview. How do you get these people to say yes and agree to help you out?Philip: [00:02:31] It depends on what I'm asking for. In the case of the article that I was mentioning earlier, I was asking for maybe 5-10 minutes of their time to do a quick quoted answer to one or two very specific questions that I knew they had off the top of their head because I had just read a whole book about it and that is actually a pretty easy ask. Most people who are in the public do receive a lot of email but don't necessarily receive a ton of these sort of requests on a daily basis. And if you make yours compelling and specific they are pretty likely to respond to you.I'm nowhere near the public profile of the people I've been reaching out to. But over the past week with this book launch, I've gotten a ton of inbound emails so it's been really interesting to consider this question from the other side. Considering which of these emails I'm taking the time to write very detailed responses to. Which ones I'm shooting back a real quick one liner. And which ones I'm just pretending I never saw and sticking it in the trash.I think that making my emails to these people who I'm trying to get interviews from very short, very specific, very relevant to their interests. Showing that I'm not just reaching out to them because they're a big name or because I had an idle inclination but because I'm very invested in a project that I need help with.In terms of finding interviews for the book. It was the same process just magnified because instead of asking for a quick email exchange, I'm asking for 30-60 minutes of their time. I'm asking to ask them some really tough questions about their work and have them think through it and give answers that are going to get published and have a bunch of people read them.I'm also implicitly asking them to endorse the work that I'm doing and the product itself with their presence in it. I have the picture of everyone who I interviewed on the sales page, for example. And with that the pitch is a little different. It's about investing credibility. The people who I interviewed for this book were kind enough to invest some of their credibility in this project and if the book was total garbage they would have gotten a negative return on that investment and future projects that they're involved with.On the other hand, if I put in a lot of work editing and producing a great book then they're going to get a positive return on investment and to some small degree be even more credible for their involvement with this project.I never state that explicitly, but throughout the process both the first short cold email, the scheduling, the interview itself, and then keeping them up to date on the progress of the project after the interview... I just have to deliver to them professional correspondence that gives them confidence that this project is going to have a positive return on the credibility that they invest in it.Regarding the interview itself... I want to make sure that I'm asking them questions that people haven't asked them before or at least asking them these questions in new ways that allow them to say new things that they haven't said publicly in the past. Both because it's more interesting for them and it allows me to create a better product in doing so.I'm going to inform them that I've read their work and not just by saying, "Hey, I've read your books" or "I've read your articles", "I've listened to your podcast". I'm going to say something specific that I liked about a previous thing that they've done. Or I'm going to reference that I just interviewed someone for the book that they know from a previous interaction. By establishing that credibility and proof that I'm going to be very invested in the project I can make them more likely to respond.The final thing is that it really is a volume game as well. I sent almost a hundred pitches to the sort of people who you would expect to see on the list next to the 11 who did say yes. A number of people were just too busy to respond, have a policy against doing interviews or for any number of reasons didn't want to appear in the book, and that's totally fine that's their prerogative.What I would encourage people to say is, okay, if I need 10 interviews for the thing that I'm trying to do I'm going to send a hundred emails. If I'm writing an article and I only need one or two interviews I'm going to send 10 emails. If you get more interviews than you're looking for, that's great. I was only planning on putting eight in this book and I got 11. And they all add a ton to the product. It takes a lot of time because you are sending specific targeted emails at scale but it was also worth it.Jeremy: [00:07:18] So basically when you want to talk to someone... do your research, read their work, learn a little bit about their background and show them that in your email so it'll be more personal and it will make it clear that you're not just shooting emails out to everybody and hoping for a response.Philip: [00:07:32] Then the other approach that you can take is more of the approach that I used pitching you to come on this show. I think I wrote you about a three sentence email and the first sentence was: "Hey, I just graduated from college and made $20,000 selling a book." The second sentence was: "I'm doing a podcast book tour." The third sentence was: here's how you can contact me.When you're doing cold email like that it's almost like a non-negative form of clickbait. You're trying to intrigue the person who you're reaching out to say: Hey, I'm going to give you some information that might be interesting to you given what I know about your background. I'm going to include links so that you can verify that I'm not some scammer. Then here is the exact actionable steps that you can take with this information if you're interested.Jeremy: [00:08:22] It was very straightforward. Then if you click the link, you see the landing page and see all the people that you interviewed whether that's Patrick McKenzie of Stripe or Cassidy Williams from Netlify. And I think anybody who sees that landing page and sees all the people that you interviewed it'll probably make them think there's something to the book so there's two different strategies for getting someone to notice your email.Philip: [00:08:45] Right. You don't need that. I didn't have any of the social proof when I was getting the people who I was interviewing to interview with me. But if you have it you should definitely use it cause it makes it way easier.Jeremy: [00:08:58] And when I received your email, I previously saw the post on Hacker News about your book. It was really easy to make that connection because I saw the same title in your email and I didn't have to think too hard about it.Philip: [00:09:13] All credit for this title goes to my mother. I wanted to call this book the "Technical Content Development Handbook" and "Writing for Software Developers" is a much better title because it more accurately describes what the book is about, who the audience is, how they're going to use it. But also when you're sending cold email like this, every single bit of this very, very short email counts a lot. Having every aspect of your product optimized for people can understand this quickly is really helpful when you're sending cold email.Jeremy: [00:09:45] You had an interview with Cassidy Williams and she was saying developers don't like being sold to. I think developers like in a broader sense knowing the specifics of what they're getting.They want that title that says this is what this book is about. So that was a smart move.Philip: [00:10:07] Yeah. Her interview definitely influenced how I created the sales page for this book by highlighting the interviews that I did, providing a free sample chapter, answering a bunch of made up questions in my frequently asked questions section. All of that was designed to do exactly the same thing that I do when I'm writing articles for clients that then use them for content marketing. I'm just providing a bunch of information and trusting that the people in my target audience are smart and engaged and want to find valuable things on the internet and that all I have to do to make the sale is provide them with the information they need to make their own decisions.Jeremy: [00:10:49] Now that you're on the other end looking at inbound emails, what gets you to click versus just keep scrolling?Philip: [00:10:59] It's a little difficult because there are so many different things that can make me interested. I'm still new to this so every email I receive is in some way incredibly exciting. Check back in 10 years and see if that's still the case. But I think that having confident well written emails about just about anything will pique my interest as long as someone demonstrates that they've put a bit of thought and effort into the communication. I feel somewhat of an obligation to respond with an equal amount of thought and effort.Jeremy: [00:11:35] Instead of emails, how about when you are researching something. You're trying to learn how to do something and you see the list of tutorials on Google. Do you have a heuristic or something that you look at to decide this one doesn't look good I'm going to keep scrolling?Philip: [00:11:50] Absolutely. And it depends on why I'm doing research. If I am programming myself and I want to solve an issue like I was talking about at the beginning of this episode I'm just going to be clicking through things as fast as I can, looking for code samples and trying it and seeing what works.On the other hand, when I'm trying to learn a new skill or get some general background knowledge because I'm about to write about a topic, I do a much more methodical approach where first I click through the first few results, find something, read it, and then I read all of the things that article references. And then I read all of the things that those articles reference. It's more of a depth first search than a breadth first search.And by doing that, I rely on the efforts that other authors have made to curate good information in the text or a further reading or sources section at the end. And I make the conscious effort to do the same thing in my own writing including in most articles some combination of in text citations or a list of further reading at the end that both talks about the exact topics that I used in the article and maybe includes all of the sources that I've cited as well as interesting things in tangential topics that might not have made it into the article proper, but are going to be really useful to someone embarking in a similar journey trying to figure out a topic.Jeremy: [00:13:22] There's two very different modes of research. You're saying one is I am working on a problem right now and I want to know how to solve it. And for that you're basically scrolling through and looking for code samples that you can copy and paste.And the other would be trying to get some kind of deeper knowledge and the way you search through things is very different. I think it's interesting to think about it that way.Philip: [00:13:50] It is and it's very important to think about your audience's mode when you're writing. A lot of people talk about picking an audience, finding and defining exactly who you want to write for, and figuring out things like: How much programming experience does my potential reader have?Those questions are very, very important and can be difficult to address because as a writer you get to define your own audience. You write for someone with a specific background and that person comes and finds your work if you do a good job of distributing it. And people without that background might come across it but don't read it.However when you're thinking about your audience's mode that is something that you have even more control over because you can write the exact same topic with the exact same background two very different ways depending on whether or not you're trying to get someone to sit down, work through an entire sample application with you, and really develop a broad and deep understanding of a particular topic.Or you're just trying to get someone in and out and get them a code sample and get them a quick explanation of why they might be experiencing an error and what they can do in the future to resolve it and then get them back to their work.I think great publishers understand this difference and they're going to hire people to write technical content or they're going to write technical content themselves with an eye towards intentionally creating both types of content and trying to avoid hanging out in the middle where they have an article that's difficult to parse and doesn't provide sufficient depth.Jeremy: [00:15:28] When I'm looking for a specific code sample a lot of times I'll look for links that are to Stack Overflow because I know that if I go to Stack Overflow then it's going to be the short code snippet and I'm not going to have extra time trying to find the thing that I'm looking for.How do you decide whether to go to Stack Overflow vs click on a long blog post?Philip: [00:15:56] Stack Overflow is an incredibly important part of the technical content ecosystem online. When I think about my competitors I don't think about the people who I'm competing against to maybe write an article for a client or land on the front page of hacker news every day.I'm competing against the vast amount of free technical content already available on sites like Stack Overflow and what Stack Overflow is really great for is exactly this quick answer stuff that we've been talking about. But the places where I think blog posts can shine is when someone is not asking how, but instead asking why or what or who or any of these other questions.Because if all you need is a code sample it's probably because you already know how to use it. You already know exactly what you want to do. Bringing back my original point about asking specific questions: Sometimes you don't yet have a specific question and your goal in research is not to find an answer, it's to find a better question.And a lot of times going to longer form resources like blog posts or articles or even books is a great way of asking better questions which is a really, really critical first step in research that is easy to overlook because it might not feel like progress. It's kind of like when you're programming and you've been stuck on an error message for awhile and then you change something. And you got another error message but at least it's a new message and that's really exciting and that feels like progress. The same thing with asking a newer, better, more specific question can be achieved by consulting longer form resources first. Then you say, okay, great. I know that I just need to sort this database index by date ascending rather than descending and that's my issue. That's what's causing all these problems. I can look up the SQL query to do that on Stack Overflow.Jeremy: [00:17:51] Yeah, that's a great point because stack overflow is not always the best place to understand why you would do something a certain way. Somebody will answer with a code snippet and if you copy and paste that, it'll probably work. But sometimes what gets lost is, is that the thing that you really should have been doing?If you're not quite sure what you want it makes a lot of sense to have that be a long form blog post or a book or something like that.Philip: Absolutely.Jeremy: [00:18:23] Julia Evans creates zines, they're these illustrated guides on technical concepts. One thing she recommends is that people write for one person.How do you decide who that person is and what's the best way to let the reader decide if they are that person you're writing for?Philip: [00:18:45] That's a great question because once you figure that out, everything else is pretty easy. A lot of the time I write for myself in the past. I write the guide or the article or the book that I wish I had that I wish I had been able to read before starting out on a project.But sometimes I think about, okay, I have a friend who wants to do something. I have a coworker who struggled with this. Just figuring out someone from my life who I can think about as a example of the audience. And again, that's sometimes, but not always me. That can be very helpful for solving this question and based on my interviews it's what a lot of other people do as well.Jeremy: [00:19:29] When somebody comes to the article, do you expect them to scroll through and figure out if it's for them? Or do you think there should be a more explicit sign post saying: "Hey, for this tutorial you should probably know these things."Philip: [00:19:44] I don't really believe in categorizing tutorials "This is a beginner tutorial" because maybe it has something really useful. And someone might overlook it if they consider themselves above that or on the flip side "This is an advanced tutorial" could scare away someone who is ready for it but just isn't confident in their work yet.What I think of as some of the implicit markers of a tutorial first is the setting up section. When you talk the reader through here's what you're going to need to configure in your environment in order to run my sample code. If you spend a lot of time walking through the exact minutia of here's how you set up a virtual environment in Python, here's how you PIP install a package. That's going to be a really strong signal that this article is for a beginner level reader and an advanced reader can go ahead and skip that and assume that their thing is configured correctly. Move on to the thing that they're looking for and you haven't really bothered them by including that setup information as long as it isn't pages and pages of it.On the other hand, if I was writing about a very advanced topic I might not include very substantial guidelines on how to get set up both because it's going to save some space that I'm able to instead assign to describing this complicated thing I'm talking about, but also because if someone isn't able to figure out the setup steps that I provide, then they're probably not ready for the advanced article itself.I think that the setup section of any article provides a very important form of positive gatekeeping in terms of implicitly informing people whether or not an article is written with an audience in mind that they were part of.Other than that, I think that like a focus on very specific titles. Not clickbait, but explaining exactly what you're doing, how you're doing it, why you're doing it, both in the title, the summary, the first couple paragraphs is a great way of letting readers know whether or not they're in the audience. And that even ties back to my own book because the title "Writing for Software Developers" lets anyone who's not a software developer know that they're probably not going to get a lot of value out of this book.Jeremy: [00:21:59] Rather than being this gatekeeper you let the reader infer it through the title, introduction, the setup steps, and hopefully they can decide from there whether it makes sense to keep going.Philip: [00:22:16] Exactly. The other thing about technical content in general is that it's an opportunity to provide people with a pathway into the field. People talk about how computer science is one of the easiest things to learn without going to college. I'm fortunate to have gone to a great college with very strong CS program but even still I learned a lot of the things that I use in my actual job and in my work outside of the classroom by using technical content. So I think it would be counterproductive to explicitly tell anyone: " Hey, this isn't for you."Jeremy: [00:22:53] When you are writing an article how do you decide the scope for it? To give you an example: You mentioned you had listened to the episode with Courtland Allen. He talks about a lot of different things that are interesting.For example: How he made his site stand out with the design, how he scaled his site, the technical stack. If you were going to write an article based on something like that, how would you decide whether to put it all in one or split it up into pieces?Philip: [00:23:30] If I'm faced with a very large project like that, I would definitely split it up into pieces. I mean, there's so many interesting aspects of Indie Hackers. I'm sure I could write a entire article just about, the CSS within the upvote downvote buttons. You can get very, very deep and very, very specific and there's a lot of value in doing that.On the other hand. One thing that was consistently mentioned in the interviews is that if you write a series of articles, people aren't necessarily going to read the whole series. So what I would focus on is picking out a bunch of the most interesting aspects and trying to create connected but self-contained pieces of the puzzle.For example, this is something that I've been doing for the past few months with Smashing Magazine. Every so often I've been publishing a Django tutorial that all fit together into a broader understanding of how to use the framework. But I'll also very intentionally create it so that they are individually consumable and do not rely on any other bits of the series in order for you to get the code samples up and running in order for you to get a understanding of what the topic of the article is and what the main takeaways are.I think I would do something very similar. I would take the smallest piece that I think will make an article, because I'm always surprised by how much bigger every topic ends up being once I started writing about it. I would find the most interesting nuggets from this massive project. I would build individually packaged tutorials around each of them. And that's where I'd start.If you knew it was going to be a book or a video course or something that was delivered as a continuous whole, that's where I'd invest some time in talking about, okay, here is the overall structure. Here's how all of these things connect to each other, and here's how you can identify patterns in your own projects or your own work that match patterns in this example and will lead you to the specific pieces of content that I've developed that might address similar issues to the things that you're facing.Jeremy: [00:25:43] You would default to splitting things up into as small a piece as possible while still being valuable on its own. And then if you were going to try a larger project you would have to know upfront whether that's a book or a video course.Philip: [00:26:01] Exactly. And the thing with technical content is you have to trust your audience a lot. For example, the first half of the book is about writing a 2000 word technical tutorial for clients and I talked about this for three reasons.The first is that writing a 2000 word article for a client is a great, achievable first challenge for people who want to get into this.Then there was also a large amount of demand from publishers for this exact format and style of writing.But the main reason is because this is what I have experience in but that experience transfers very easily. Everything we've talked about in this interview about finding an audience, figuring out your scope, figuring out how to make your code samples complete and runnable. All of that applies. If you're writing a book, if you're writing a README, if you're writing documentation, a Wiki, the skills are really transferable.Similarly, when I'm writing technical content, I have to assume that the people who are reading it are smart, they're engaged, and they're going to take the specific context that I've presented them. They're going to take their own context and they're going to figure out how to bridge that gap.Jeremy: [00:27:15] You had an interview with Chris who's a community manager at Digital Ocean and he mentioned how readers will skim code and images and they gloss over long paragraphs.Does that match with how you navigate content or is that more specific to when you're searching for a certain type of thing?Philip: [00:27:35] That interview had a lot of surprising things in it. And that was one of the most surprising to me because I spend a lot of my time when I'm writing focusing on crafting not necessarily sentences that my literature professors would consider to be well-executed, but at least competent English phrasing. And I spent a lot of time thinking about the readability of my paragraphs. Starting at one idea, flowing into another one, and to hear that a lot of people were skipping them was kind of disheartening. But then I did think about how I read content myself. I'm fortunate to be a really fast reader. If I'm trying to read a book with another person, it can be a little unfortunate because I'll get all the way to the end and they'll want to talk about chapter three. The way that I skim articles is I just read the text but really, really fast.Chris had actual statistics based on using Hotjar on scotch.io to see how people were actually interacting with the site. And in aggregate, a large portion of the people were just scrolling down to the first code sample, copying it, and when I think about the mode of: I need to solve a particular issue in this application so that I can commit it and go eat lunch, then absolutely that's what I'm going to do. As a technical content writer, there are two things I can do to address that.One is to increase the skimmability of my content as he was talking about. To maintain the same focus on good writing just put more white space in there, put more paragraph breaks, more bulleted lists, numbered lists, quick summaries. Because if those are going to be really useful to someone, I should absolutely include them. And it's going to do nothing to detract from the experience of someone who wants to really read and sit with the content.And then the second thing is when I'm considering how to write this stuff, I do need to focus a lot more on putting in code samples that are complete, runnable, and understandable without the context of the article around them. Even if that makes those code snippets a little longer.And it makes describing them a little more difficult because I have to parse through eight lines of code including some import statements or configuration. For the majority of the people in get it done mode who are looking at the article if that's going to help them copy it into their application more easily then that's an investment worth making.Jeremy: [00:30:14] Let's say I'm going through a tutorial and I follow the code as it's described and if it's missing import statements or it's missing steps to install prerequisites.That's a lot of time that's sunk. You end up googling how to complete the tutorial that you're currently working on. I can understand why that's frustrating.Philip: [00:30:36] Exactly, and that's something that is worth investing in avoiding. One of the best examples I've ever seen of this is Mark McGranaghan's Go by Example where every single code example on that entire site you can take it, copy it, and paste it into an online go interpreter or your own go environment, hit run, and there you go. It works exactly the way it says on the site. The value of that is incredible.One of the places we can see a real investment being made in that is with Stripe's experimental documentation that they published a couple months ago where instead of just showing people: "Here's the single function you write to integrate the payments API." They created sample applications. Not just one sample application. But the same sample application with three different backends, four different frontends, two different frameworks, all this different stuff. And you can mix and match these and download these sample applications.And then they also did the two column design that Mark McGranaghan talked about where you have the code on the right, the words on the left and they're matched up, and you actually click through the steps in Stripe's documentation and it zooms around in the code example that you're looking for, highlighting the blocks of code that are relevant to the information that you're taking a look at.By doing something like this, you're creating more than just a piece of technical content. You're creating an immersive experience where it's a guided walk through a complete piece of sample code and that makes it really, really easy to figure out how to bring in the thing that you need into the application that you're working on.Or if you're not in get it done mode, you're in learning mode. Then it's really easy to take a broader look at the application and see how it's structured, see the decisions that went into it in the high level, and understand how you can structure things similarly in the future.Jeremy: [00:32:38] It's letting the consumer of the content, the person who's looking at the tutorial decide whether all they need is the code and they understand just by looking at the code or whether they need that extra sort of guidance that's going to show up on the left side.Philip: [00:32:53] Exactly. And making something like that is not a small investment. I don't even want to speculate how many hours of engineering time went into that. If you look at the fully loaded costs, they probably spent hundreds of thousands of dollars of engineering time on this demo. Which is worth it for them because if it gets people to integrate Stripe in the applications then Stripe's gonna make some money.Individuals such as myself can't necessarily make a similar scale investment in technical content. I write almost exclusively about JavaScript, HTML, CSS, Python, Django, because those are the languages that I know really well and that I feel confident teaching people.And if someone would have asked me to write a sample application in 10 different languages and frameworks I just straight up would not be able to do a good job of that. When you're an individual and thinking about how to make meaningful investments in your work. When you don't have those sort of resources, what you can do is even if you're not doing it in a bunch of different languages you can still focus in on one and you can make a really great sample application that's complete, runnable, configured out of the box. And that's going to provide a smaller niche audience of readers that same value. And then what's great is it's going to be free on the internet so someone else can come along and say, Hey, I really like this. They did a really interesting thing here. I'm going to do something similar in my framework and language of choice, and that's how great ideas and practices can propagate around the internet.Jeremy: [00:34:31] When I'm trying to learn something, whether it's searching online or even if it's trying to do something in a company. The way that I typically learn and I think a lot of other people learn is they see an example of something somebody has done before. And that could be a sample project or it could be the way a certain feature was built in a code base at work, and just by seeing how somebody has already done it, it's not a full copy and paste, but it gives you this guide for how you can build something similar. And I think that allows people to be able to build something new a lot quicker than, just looking at API documentation.Philip: [00:35:17] Exactly. There's no new code. We're all standing on each other's shoulders. And I think that's one of the things that's so great about technical content is even though you know a lot of it's paywalled or a lot of it you sell to publishers who release it for free with advertising or content marketing.Ultimately we're all helping each other out to build things and it's about creating a corpus of knowledge across the entire internet that everyone benefits from.Jeremy: [00:35:44] I'm interested to see how we can improve how we share that knowledge going forward. Is there something beyond the current tutorials we have like full featured applications or templates that people could use to learn from.Philip: [00:36:01] I think that an under appreciated source of this knowledge is open source repositories. You can learn a lot just by going and inspecting the actual source code of big open source projects. Now, open source projects have a ton of responsibility already. The developers and maintainers and volunteers are under a massive amount of pressure to keep everything running and up to date and to continue providing new features that people are asking for often with very little funding.So I can't just go ahead and say: "Hey, open source maintainers. You've got to add a bunch of documentation." because that's just not a realistic thing to ask for.But I think that one under explored area that can be pretty interesting is taking a piece of open source code. And writing either as a separate thing or as a contribution to the repository directly an explanation of how it works, and I can give a anecdote about this. Well over a year ago, I was working on this application with my friend and I wanted to render an iPhone in CSS so that I could stick an image inside of it. And I found a library, an open source library that provided these beautiful CSS renderings and the one issue is that they weren't resizable. So I created my own fork of the library and wrote this Python script that used SCSS and some variables and some other things to go through and automatically convert all of these non resizable CSS devices into resizable ones. I used them and just sort of forgot about it.A year later I decide "Hey, I want to write an article for CSS Tricks. When have I done something cool with CSS before? Oh, Hey, this time with this open source library." I pull out the code. I realize that a year ago I had no clue what I was doing and was a terrible programmer. And so I rewrite the whole thing. Which is a very common occurrence when I pull something out from the past to use in an article the first thing I do is decide it's terrible and rewrite it. But that's an important part of my own learning process and then sharing that learning with my audience. So anyway, I'm here. I have this great code. I write an article about it. And CSS Tricks is thrilled. They publish it.What's super great is first off, this is increasing the visibility of an open source product, both the original one and then the modified version that I created.Secondly, anyone who comes to this modified version of the product and wants to know more about it can go look at this article and read more about how it was created and thus really understand how to use it.The third thing is that when people read technical content, as we've been talking about this whole episode, the first thing they do is they take it and they copy it and they paste it into their own applications. Whether that's personal projects, things for work, whatever. So when you're publishing technical content, the responsible thing to do is to make all of the source code open source under some very unrestrictive license. All of my clients do this. Either they'll release it themselves as open source. There'll be a disclaimer in the footer. Or what happens a lot is I create an open source thing and then I write an article based on the thing.I sell the article to the client, but not the thing itself that's not considered part of the work product. And while this might seem like a technicality, it is an important thing to think about in terms of how you're delivering value not only to the client or to your employer, but also to the reader. Because if you're going to create something useful they've got to be able to use it. Open source can help.Jeremy: [00:39:36] That's an interesting example of having this open source library. Having this tutorial or document that describes how it works and that helps other people learn how they can build something similar. And at the same time, you got paid by CSS tricks in this case.So it's like everybody benefits.Philip: [00:39:55] Exactly. It's an indirect model of promoting and funding open source software. And in this case this library is a very small scale thing. It's not something I've dedicated a ton of time to. And the original creators of the library used it as content marketing for their product.But ultimately, I don't know if this model is scalable. I don't think you could support the development of Django by just writing a ton of tutorials about Django. But I think that at the smaller scale it is just another way that individual developers are able to monetize our own and other people's contributions to open source software in order to create useful things and pay the bills.Jeremy: [00:40:44] Yeah. I think an interesting possibility that I don't think I've seen before is, there are some open source applications, like for example, GitLab is an open source project that's written in Ruby on Rails. I wonder if somebody could based on GitLab's code write a book or a series of articles explaining this is how we wrote a real world application in Rails. Because I think when you have a code base that is as large as their's is, telling somebody to just look at the source code can be pretty daunting.I think there could be room to come in and walk people through parts of the source code and have something that they can learn and take to build their own projects with. I think that's a really interesting possibility.Philip: [00:41:38] Absolutely it is. There are two great examples that I can give about the marriage of open source software, technical content, and a sustainable business model.The first is Taylor Otwell's work with Laravel which is a PHP web development framework where he and a team have managed to not only create really great open source products, really great technical content. But also charge for various value adds that larger corporations, larger teams who are using these products are able to invest a lot in because they rely very heavily on this as a foundational part of their infrastructure and stack. Which in turn goes back to fund open source developments that make this whole product better for everyone.Another great example is Adam Wathan's work with Tailwind UI. Tailwind is an open source CSS library that is very, very popular. It's kind of like bootstrap in my mind although I know that power users might disagree. The way that this is monetized is through something called Tailwind UI, which is a collection of components that you can buy either an individual or a team license to which was an inspiration for the team license in my own product actually.You can buy an individual or team license. And it's I think $300 or $600 and this gives you access to a ton of components and a ton of technical content about how to use these components and Tailwind in general to make your websites look great. And if you think about it, $300 is what, three hours of developer time? Maybe less if you are hiring very experienced developers and you have fancy offices somewhere. So it's a total no brainer to purchase a piece of not just technical content, but also reusable example code that you and your development team can use to create something that looks great and drives business results.One of the things that I drew inspiration from them as well as Daniel Vassallo's book is the idea of price discrimination with a corporate license. When you buy an ebook, it's kind of like buying a physical copy of a book. You're not supposed to email a copy of it to everyone on your team. But I figured that some companies have documentation teams or might otherwise want to purchase a copy of this for everyone working there. But I didn't want them to have to go buy a bunch of copies and so I created a team license. A new tier of the product on Gumroad. I added a one page PDF saying, here are the three ISBN numbers that you can share with as many people in your company as you want and set the price a hundred dollars higher, figuring, Hey, this is a huge markup. This is amazing. So all that work took me about 10 minutes.So far, seven people have bought corporate licenses, which is really exciting and is a massive return on that time invested. So that's another facet of thinking about your audience during the product development cycle. How different people in different situations might use your product and then creating something that adjusts to their needs. Because for a company it's worth the hundred dollars to know that they have permission to use this book however they see fit, and to show it with as many people as they want.Having that kind of license both helps me out because it made me all of this extra money. It also helps out the people who are buying it because it lets them use this content in a way that they might not have been able to otherwise to better achieve their goals.Jeremy: [00:45:12] That's a great point. The threshold for what somebody thinks is expensive is very different for the person vs the company. If I look at something and it's a hundred bucks and I'm buying it for myself, it's a little expensive for me to buy. But if it's a business paying for it then it doesn't even register.They'll put it on a card. You don't have to think about it. It makes a lot of sense to price differently depending on who's buying.Philip: [00:45:35] Exactly. It allows me to provide the value to the people who are able to pay for it while also keeping the main product cheaper for everyone else because I'm not trying to hit a midpoint between individual and corporate use.Ultimately what both of these examples show to me is that where a lot of the money is in technical content is first, you make something great, and then you save people time. So you've made some really great open source thing that people want to use. And then you take the businesses who can really invest in saving their teams some time and you say, Hey, I'm going to make you a great resource. It's going to cost a few hundred dollars, but it's going to increase your development velocity so much that you won't care at all. I know that both of these people have also done a lot of other stuff in technical content, written great books, built a big audience doing interesting work in public. And so I'm not sure I can attribute all of their success to this business model or even the majority of it. But I think that this business model is the sort of thing that I'm looking at very closely as I consider longer term more sustainable work in technical content development.Jeremy: [00:46:46] Adam Wathan with Tailwind is a great example because Adam has put together screencasts, he's put together tutorials, and the quality is so incredibly high. Just going through those tutorials I learned a ton about CSS beyond just learning about Tailwind. Then you have this product that helps you move even quicker it's a very easy decision.Philip: [00:47:11] Exactly. And this goes back to what we were quoting earlier from Cassidy Williams saying developers don't like to be sold to. We don't like to be sold to but we love to be taught interesting things. And that's why content marketing, really good content marketing does so well in the developer space. And it's actually something that gave me some pause and some doubts when I was thinking about launching my book is I was thinking I have what, 20 Twitter followers? My website gets a nice dozen hits a day. Half of them are from my friends. So I was thinking, do I have the established credibility to launch a product like this?And are people gonna feel like I've created something that is valuable or they're going to feel like I'm selling them something? And part of that was solved using the investment of credibility for my interview subjects I was talking about earlier, but part of it was a experiment. Everyone says you have to build an audience before you launch a product. My hypothesis was, I can build an audience by launching a product.It happened to succeed. It's a single data point. I don't know if it's repeatable. I can't guarantee it'll work for anyone else, but a lot of people might look at a business model like Adam's, and say, okay, this is great for him, but he already made however much money selling books. He already invested years and years in building this open source library. I need money now. I can't do that. But I think that an iterative approach where you build a thing, you launch a thing, you use the momentum, you build a bigger thing, you launch a bigger thing might be possible and might be a way to introduce more people to this business model and this space.Jeremy: [00:49:02] Like you said, a lot of people do say you should build an audience first. I had a conversation with Ben Orenstein, he did a lot of public work in terms of, conference talks, blog posts, he ran a podcast for a long time, mostly in the Ruby on Rails community.And when you ask him how did you build your audience? He said well, I was just helpful to people for a decade. And you took a very different path where you didn't have that following, but you were able to have a successful launch out the door.And what you were saying makes sense in terms of having other people vouch for you. Having people like Patrick McKenzie, post on Twitter and say like, Hey, there's this, book, writing for developers, I think it's really great. And if you like what I post on Twitter, or you like my blog posts, I think you're really gonna like this. For me as a developer or just as a person, when I see something like that, then I'll go like, Oh yeah, I trust Patrick. I'm definitely gonna check out this book. So I clicked the link and then I get the landing page and that's the second step, right?You get the person to the landing page, And then what's on there is attractive enough where you go to the next step, click the buy button right? So I think that social proof or having somebody else make the recommendation is pretty huge.Philip: [00:50:24] It really is, and I will not sit here and pretend that I would have sold the number of books that I did if I didn't have the buy in from these very successful people with the interviews and the promotions.However, there are some steps that I made that I think anyone can replicate for their own products that I'd like to talk about sort of going through the funnel.So at the top of the funnel, like we've been talking about, it's Twitter and Hacker News, and I had no following on Twitter, but no one has any following on Hacker News. The concept of a follower doesn't exist. Every single post goes into new. So for a week I focused on, I'm going to create a great Hacker News post.I revised it like I would revise a article for a client or even the book itself. I wrote a top level comment that I posted immediately after posting the Show HN. I looked through all data and statistics to see, okay, I'm going to want to launch this on a Tuesday, 8:00 AM central time so that people in Europe are going to be awake to see it, cause it's afternoon there. I'm going to get that East coast and Midwest morning traffic, get it to the top by the time the West coast wakes up, then they're all going to see it. And that's where I'm going to get the massive influx in traffic.So taking the time to really consider how you present yourself on these platforms and that did rely on the fact that I've been a member and user of this platform for years, so I know it well. I know the sort of unwritten rules and what the audience is going to respond well to and not respond well.To contrast that with Product Hunt where I posted the same thing and got 10 up votes and no views. Having an understanding of the platform that you're launching on is really, really important. And is the difference between staying at the top of Show HN for a day vs getting ignored.That's the top of the funnel. Creating a post that really targets the specific audience of the platform that you're using. From there, my sales page, I spend a lot of time working on that. I based it on the advice from Nathan Barry's book Authority, which covers a lot of this stuff and made sure that I included not just the social proof, but also things like a free chapter so that people can look at the formatting and the content and make sure it's something that they want to read.I focused on having a good bio listing my clients which is a form of social proof that I owned for myself over the last 15 months or so writing for people. And then with the Gumroad page itself I spent a lot of time thinking about the layout, the pricing, eventually landing on $36 as the right price because on the one hand, the book teaches a lot of valuable skills. On the other hand. A lot of the people who might be interested in reading it are going to be students like me or other early career people, which is why I also put on the website that anyone who can't afford it can just send me an email. I'll send them a copy for free. So I've sent out dozens and dozens of free copies as well.And all of that just serves to get more Twitter followers, get more people excited about the book, spread the word. Even small things like making the book DRM free. A lot of people worry about what if someone's going to pirate my book? Well, I'm worried about exactly one thing. What if someone goes and lists it on Amazon? That's why I've registered copyright so I could issue a takedown notice. Other than that small authors such as myself have to worry about things like obscurity much more than things like piracy. I'm mostly concerned about getting the word out there about my ideas, of my content, rather than making sure that every single person who looks at the book has done so in the approved manner.Looking at all of that, there are all of these little steps that you can consider in a launch that are going to provide a big impact on the way that your product is received. People are going to be able to perceive that you are really, really invested in this thing that you've created and you've thought through all these things, and thus the thing that you're actually selling them is pretty good too.In summary, absolutely. The people who I interviewed were a massive factor in driving sales, and I'm incredibly grateful to them for that. But none of that would have meant anything if the underlying product hadn't had that time investment that anyone can make.Jeremy: [00:54:51] The way that I found out about the book was through Hacker News. It was seeing the post and the title was very to the point. It's writing for software developers. I do writing being a software developer and I would like to get better at that. I click on the comments, I see your post, which, was very targeted, to hacker news. It understood people want to hear about the process or they would want to hear about Patrick, who is well known on Hacker News. You put a quote from him in your post. It was very effective at getting what I think the average reader would pique their interest and get them to click.Philip: [00:55:31] Throughout the lifetime of that post on Hacker News. It generated almost 10,000 page views on my site. I'd imagine more because I think the average Hacker News user has more trackers and analytics blockers than the average web browser. But as a lower estimate 8,000 or so people looking at the page and converting at remarkably high rates.To anyone who wants to launch something technical content related or anything that the audience might like investing some real time and effort in a great Hacker News launch is really worthwhile for sure.Jeremy: [00:56:10] Another person that you interviewed for your book is Daniel Vassallo. Something that he's been really effective at is distilling thoughts into tweets. As you start to think about the next step of building your audience, how do you approach writing tweets versus more long form content?Philip: [00:56:33] Yeah. This is something I've really been thinking about and struggling with over the past few days because I have this brand new audience and it's the biggest platform I've ever had. Over 750 people follow me on Twitter and that's a sign of trust that I take very, very seriously.I want to focus on quality tweets over quantity and focus on creating threads to get around the 280 character limit. Because I mean, let's face it, I'm a long form sorta guy. I speak and write in a lot of words. And so what I focus on is taking something I've been thinking about or working on that day, distilling it into something actionable, and then providing that on Twitter and just hoping that people like it.The other thing that I've been focusing on as I engage and grow this new audience is responsiveness. I'm trying to answer email really, really quickly. I'm not necessarily always doing a great job. Anytime someone responds to me on Twitter, I want to be answering them right back or at least liking the tweet or something.I've sent so many direct messages. I actually had to look up Twitter's guidelines on how many you could send a day because I'm DMing people who follow me, who have interesting profiles and saying, Hey, you know, what do you want to read about on here? What do you want to know about the topics that I'm working with?All of that is in the pursuit of figuring out what people want and then figuring out what I can provide and operating in the intersection of that.Jeremy: It'll be interesting to see how you grow that audience and how you use Twitter in general.Philip: [00:58:15] Yeah. The thing that I wake up thinking about every morning and go to bed thinking about every night is at this point, more than 20,000 people have looked at the writing for software developers landing page, which is super awesome, and ~560 of them have bought a copy of the book, which is also super awesome.My question is, how do I get the rest of those people interested in this book? Or if it's not the right thing for them what is the right thing for them? What will get them contributing to this online community creating their own technical content for publishers, for themselves in their own learning, for the companies that they work for, the open source projects, and just expanding this idea because one thing that Angel Guarisma said in our interview right at the beginning that's been a real motivation for me over the past few months is there was a lack of a canon or accepted literature on technical content development. There's not a lot of books besides this one and a few others about the things that we do. There was that sense of responsibility as I was writing this book, living up to the title, writing for software developers.I don't think I've made it there yet, but in my pursuit of this field, I have the opportunity to be definitive, and that's something that I take very seriously and want to do a good job of and take my time and do right. But in order to do that, I have to engage with the people who like what I've done so far and ask them what's next?What can I make for you that will help you make the things you want to make? That's an incredibly privileged position to be in and I'm very grateful for it and I just want to live up to it.Jeremy: [01:00:08] Earlier we were talking about when Chris told you that people skim through articles that surprised you. Is there anything else from the interviews that really surprised you?Philip: [01:00:25] One thing I was surprised by is that many of the people who I interviewed were so strongly against advertising, which at the time I was a little confused by. I've never really minded ads all that much but I've also grown up steeped in them. Today with the sharp decrease in the number of advertisers on the market right now and thus the decrease in rates a lot of publications that have been relying on advertising are struggling to find sponsors and get the sort of revenue that they've been previously accustomed to. That was surprisingly prescient of my interview subjects. That's just what comes from all of those years of experience and seeing dips in advertising rates in the past. I've always thought of it as a very solid foundation to build a business on where in fact, it turns out it is not. That was one thing.I wasn't necessarily surprised but it was really interesting to see the level of passion and commitment that everyone puts into the work that they do. That was very inspiring to me because sometimes it can feel like, okay, I've just got to churn out another article. Just got to get something that's going to get the editors approval and be decent and people won't hate.And seeing the amount of effort that all of my interview subjects put into their own work, rekindled my passion for the field and really reminded me that every single article that I create has the opportunity to be my best work and that I should not waste that opportunity.Jeremy: [01:02:06] One thing about your journey learning how to write and publish professional content. This all happened while you were in university. I don't think there's many people who think I'm going to write professional technical content. What was the path for you finding out that was an option and deciding that was something you wanted to do?Philip: [01:02:29] So the great thing is that on the internet, no one knows you're a cat. That's a famous quote and I think that in technical content I have a picture and a byline and maybe a short bio, but that usually shows up at the bottom of the article, not the top.And the great thing about publishing with some of the big names that I've gone with is their aura around my work gives the work the ability to stand for itself and people are able to judge the work rather than the author.How I got into it was almost by accident. I started out doing some freelancing for a company based in Germany who was trying to localize their site for a United States audience. I rewrote their web copy in native English, to improve the perception of the site.And then I noticed that they publish technical content. I asked if they wanted to pay me for some tutorials about how to use Django and they said absolutely. I wrote a couple of tutorials and it was a lot of fun. Then I thought, Hmm. If this one tiny company out in Germany publishes technical content maybe other companies do too, that could be cool. So I went around and I started looking. I found FloydHub, I found Smashing Magazine. I found all of these places.This is a metaphor that I use in the book. Imagine if a college kid being me woke up one day and said, you know what? For my first attempt at journalism, I'm going to write the headline story in the New York Times. That would just go, nowhere.But what's amazing about technical content is because there's so much demand for it even the biggest publishers are willing to give you the time of day if you write a good pitch. So after doing some research and writing some outlines I started pitching people and people said yes a lot more than they said no.I've done a lot of other attempts at freelancing and entrepreneurship and such before this. Building products, all of that stuff. And reaching out to two people to have one say yes instead of reaching out to a hundred people to have one say yes is a massive change of pace.This is what I'm going to wake up over the summer and do at 5:00 AM before work because just that cycle of feedback and validation was really, really inspiring and got me to commit to creating technical content as a important facet of my overall work.Jeremy: [01:05:00] What's interesting is a lot of people when they see a page like Smashing Magazine or CSS tricks, not even just people in school, but people who work professionally. Most of them probably think to themselves there's no way I could write for this publication. You have shown that it's a lot more accessible than people think.Philip: [01:05:22] Yeah. I think unfortunately a lot of people struggle with imposter syndrome working in technology and that's a huge issue in our field. It limits the people coming in. It limits the diversity of the field. It limits the things that we can accomplish together. But I do hope that seeing, Hey, this college guy was able to publish with insert big name here is helpful for people as they decide that they want to either do something similar in technical content or the equivalent in their own field.Jeremy: Cool. I think that's a good place to start wrapping up, but is there anything that you felt we should have talked about or you wanted to mention?Philip: No, I'm really happy with everything we talked about.Jeremy: If people want to check out writing for software developers or see what you're up to where should they head?Philip: [01:06:16] So my central hub on the internet is my website, which is philipkiely.com the book is available at philipkiely.com/wfsd. I maintain an email list that you can sign up for on that website. I'm also increasingly active on Twitter and occasionally post content to YouTube. I'm working on a video right now going through my first week's analytics on this book and looking at where the traffic came from and who converted. I think that's going to be a really interesting thing for people who are interested in their own product launch to look at. And I'd be really thrilled if you purchased a copy of the book or followed my other work.Jeremy: Thanks for chatting Philip and congratulations on the launch of writing for software developersPhilip: Well, thanks, Jeremy. It's been a great time talking to you today.Jeremy: That's it for my chat with Philip you can get the transcript for this episode at softwaresessions.com. See ya!

Sara Leen is a localization programmer for XSEED Games on titles like Ys, Trails in the Sky, and Corpse Party. She got her start reverse engineering and fan translating games.We discuss:What makes games differentHow games store and encode textRewriting Corpse Party (and not being able to run it for a year!)Porting and modernizing gamesReverse engineering raw assemblyThis episode originally aired on Software Engineering Radio.Related Links:@SaraJLeenPersonal SiteCorpse Party PC Programming BlogTrails in the Sky Second Chapter Localization Blog 5Mojibake (Garbled text when text is decoded using the wrong encoding)Shift JIS (Japanese character encoding used in older games)Hot Soup Processor (Obscure programming language used for Corpse Party)XSEED Games Music by Crystal Cola: 12:30 AM / VHS SkylineTranscriptYou can help edit the transcript on GitHub.Jeremy: Hey, this is Jeremy Jung. This episode, I'm talking to Sara Leen, a localization programmer for XSEED Games. Sara has over a decade of experience localizing games from Japanese to English. And we talk about how games are different than other types of software, storing and extracting text, porting games to different platforms, and rewriting a game from scratch.A game written in an obscure programming language called... can you guess it? Hot Soup Processor.Pretty good right? This was a fun episode and I hope you enjoy it.Jeremy: [00:00:35] The first thing I want to start with is you have experience with building other types of software. I was kind of taking a look at your resume. What makes games different than working on a typical software application?Sara: [00:00:48] With games, there's a lot of things that you have to worry about that you otherwise wouldn't like the frame rate of the application becomes very important. For example, you need to make sure that the timers of the application are coded just so that the game will always run at a steady pace. And there are various applications of timers like delta time that are usually used to ensure this goes perfectly smooth.Jeremy: [00:01:17] When you talk about timers, I've heard of this concept called a game loop. Is that what you're referring to?Sara: [00:01:27] Essentially a game is made of graphics, audio, and input. But all of these have to be checked every frame. And so you have your timer going so that it is basically looping through the game 60 times a second or whatever other frame rate your game is running at a lot run at 30. There will be various steps in this process, like game logic and drawing the various models. And of course you have to keep the sound system updating.Jeremy: [00:02:01] As opposed to a web application where a lot of things are in response to something, right? Somebody's going to a webpage, clicking a button. With a game it sounds like there is this consistent loop that's tied to the frame rate. Is that correct?Sara: [00:02:20] Right. And instead of reacting to things, you are occasionally checking the state of course so many times per frame and you will still have to respond to the user input but it's all going to be one part of this large set of functions.Jeremy: [00:02:38] So you're in this loop and let's say somebody was pressing a button on their controller or moving their mouse that would affect the state. And when you're in that loop it's going to look at that state to determine what to do next in the loop?Sara: [00:02:58] Yes, exactly. For example, in your input system you are usually going to be saving every frame, the state of the input system at the moment, like is this button being pushed? Is this button being pushed? Was this button pushed last frame and by comparing it against the previous frame. You know what the user is doing right now, and then many parts of the game logic will have various branches that check exactly what the user is inputting and what the current context is.It's certainly more complex than the idea of I push a button and it calls this function.Jeremy: [00:03:34] That sounds like like an entirely different model of thinking. The structure of the application has to be almost completely different.Sara: [00:03:42] Absolutely.Jeremy: [00:03:43] In terms of skillsets working on a game versus working on a regular application. What parts are really different and which parts are kind of the same?Sara: [00:03:54] Regardless of what you're working on you're going to be working with a lot of external APIs, but these are probably going to be entirely different for a game versus other software.For example, you need to be familiar with APIs like DirectX and OpenGL and all of these typically have different ways of handling graphics, handling audio, handling input.There's also SDL of course that you could use and depending highly on the programming language as well there may be different ways of handling certain things that would be different from handling a normal application.But most importantly you need to be familiar with graphics and audio APIs above all. I think those are typically less important when you're working on normal software because you're more concerned about how to build your window and how to get all of the elements displaying in the proper positions.Jeremy: [00:04:51] With a typical application the actual act of rendering to the screen is abstracted away from you so you don't have to worry about how to draw to this screen. It's more like, I want a button or I want a window and something else is handling that for you.Sara: [00:05:08] Yes, that's pretty much it. And when it comes to video game graphics, you have to consider a lot of different things like differences between graphics cards. Can this graphics card handle square textures or does it need a different shape? As well, of course, as being concerned with the quality of how it's drawing. Like is this texture supposed to be filtered? Will this graphics card filter it correctly? The APIs simplify that, but you're working at a lower level.Jeremy: [00:05:36] And when you were giving examples of APIs you were talking about DirectX and OpenGL and SDL. How low a level are we talking? If I wanted to draw something to the screen at the most base level what kind of work am I looking at?Sara: [00:05:55] Well, for SDL in particular this is somewhat of a simplified process because it handles all of the initialization of devices and such for you.But with DirectX and OpenGL, you have to actually get the list of devices you have to initialize each process that you need. Like you need Direct3D, you need DirectInput, you need DirectSound, and then you have to make sure that you have all the device information that you need, including stuff like the current resolution or the capabilities of the device. Device capabilities is a big thing when it comes to that.And then to actually draw the image, you are going to need to create a texture buffer. You're going to need a back buffer for the screen, and then you will have to load the texture and you will have to draw the texture to the buffer and then present that to the screen.Jeremy: [00:06:53] When you refer to the buffer, at kind of a high level, can you kind of explain what that's doing?Sara: [00:06:58] Essentially you have the screen that you're drawing to, and you usually don't want every single draw command to go directly to the screen. It's better to have a buffer that you can operate with so that you can display things in the correct order at all times. You can have them on the correct Z level, and of course, any sort of processing you need to do, you can do before it's drawn to the screen. And so everything goes onto the buffer after it's ready more or less.Jeremy: [00:07:29] And when something is in the buffer is that what you plan to actually have the video card render? Or is that after it's already been rendered? Where is that in the stage?Sara: [00:07:43] Well, rendering is basically the last step of presentation, but it depends. If you are drawing something 3D naturally you are going to have to do some processing on that via the graphics card. And the APIs allow for things like that, but typically rendering to the screen itself is always going to be the very last step of the frame.Jeremy: [00:08:06] And then elsewhere in your application even though you're putting things into the buffer, you're deciding that there are certain frames that you don't need to render?Sara: [00:08:17] Yeah, and of course that depends highly on the game, but often there may be no difference. So you don't really need a new frame or there might be a case where the game is overall running a little too slow. So skipping some of these frames will get you more CPU cycles free to work with.Jeremy: [00:08:36] We were talking about the loop earlier where all the logic is happening at a set rate, but in terms of the way people see the game of how smooth it is, you may have the game logic running at the same rate, but only choose to render certain frames in the buffer so that it still runs the same it's just that it's a little choppier?Sara: [00:09:03] Yeah and essentially you need your game logic working at the highest that your gameplay can really go. But when it comes to the graphics, it's a little more arbitrary for sure. There are situations where you might intentionally want your game logic running at 120 frames per second and then the player input and such will still come at that speed regardless of the actual refresh rate of the display.Jeremy: [00:09:32] Interesting. I want to talk a little bit about how games can have a lot of binary assets. Typical software development you would use git as a version control system. Would you use the same for games or is there a different system that's more suitable?Sara: [00:09:49] Well naturally, this depends on the company. Quite a few do use git and simply store the binary assets even though there is no real comparison system for them. There are also various internal systems that companies may use that are specially made for handling their specific assets. Essentially version control is not that different in the gaming industry. It just occasionally uses different applications.Jeremy: [00:10:22] And so when somebody uses git even with the binary assets, it's not an issue. You just can't see the diffs properly.Sara: [00:10:32] No, it's typically not a big deal because it's not difficult to look at two different pictures and visually compare them. And generally you're going to be changing the binary assets less as development goes on rather than more, whereas the code that's always going to be changing.Jeremy: [00:10:52] The title you use is localization programmer. What would you say is the main role of a localization programmer?Sara: [00:11:02] Essentially, when a game is being localized from one language to another obviously the goal is that you get all of the text inserted and you make sure that it all looks correct. At base levels this is generally the accepted position of a localization programmer. You are getting all of the texts into the game. You are making all of the code work with the correct fonts and other asset changes that you may need that are different from the original language. However there are a lot of things in this process that you wouldn't normally need as a skill. For example, if you are working with Japanese games, all of the comments and everything are going to be in Japanese of course.Jeremy: [00:11:43] When you're translating games, do you usually have access to the original source code?Sara: [00:11:49] I usually do, but there are plenty of cases where we are working on a game and the original developer is actually the one to handle the localized version. It depends a lot on how much other companies feel they are able to release to other parties, whether it's because of rights issues or they're just kind of nervous about their code quality, which I mean, I would be.Jeremy: [00:12:12] And when you get access to the code do you usually have access to the original developers or are you on your own?Sara: [00:12:21] I would say that completely varies. You never know exactly, but it's usually going to be the same with each partner. Like, they will usually be more than happy to help you if they have the developers on hand to help you. But game companies are constantly developing new titles themselves.So when someone like me is working on a game, we may not have as much access to the developers or they would be working on the game.Jeremy: [00:12:48] You were talking earlier about being able to extract text or bring text into the game. What are some examples of ways that the text would already be stored and how would you bring that out?Sara: [00:13:03] This also varies a lot and it's very interesting. Many games use script files, essentially lists of commands for their events, et cetera. And in these cases, as long as you know the format of the script file, you're usually going to be fine. But some games like to hard code their text, and yet others have been in binary formats that have been compiled in some way, assembled rather.And when that happens, you often have to take a look at the code and figure out exactly what this format is, which may or may not be documented, and basically do the process in reverse unassemble it.Jeremy: [00:13:46] The first example you gave was script files. So would that be a plain text file and you could replace that text with instead of Japanese put in English?Sara: [00:13:57] Essentially, yes, like when there's a script file format, you're going to see various commands, like there will be message and then it will have the text for a message box, that kind of thing. And when that is the case, it's usually pretty easy to change since it is just text. But you might run into some encoding issues.Jeremy: [00:14:17] By encoding issues, would that be where let's say you replaced the Japanese with English and then the game itself tries to load the texts from the script file and it can't due to a different text encoding?Sara: [00:14:29] Not so much that as you might get garbled text, what in the fan community that I used to work in, we would call cave speak.Jeremy: [00:14:38] Could you explain what you mean by that?Sara: [00:14:40] Imagine that you are playing a game in Japanese, but your system doesn't have Japanese fonts loaded. The font not being there for Japanese, you might instead see jibberish like ekvgh, that kind of thing. And that is cave speak.If the encoding is very specific for the game or the font simply doesn't have other symbols, you might get complete jibberish in what font was already there. In Japanese, it's called mojibake.The way that encodings work of course, is that the text is interpreted from its binary or hexadecimal format into characters we can understand. And since there's a lot of different encodings out there, sometimes you don't know exactly what you're going to get. So, for example. If a game was made for an older system, it may have a completely custom encoding where a value like "2" means "a" and in this situation just replacing the text you have no idea what the game's going to output. You're going to have to change it all yourself.Jeremy: [00:15:55] So the original text in the script files, or perhaps you're talking about the binary files, they're not necessarily using a standard encoding like ASCII or Unicode. They may have made up their own encoding format.Sara: [00:16:12] Yes, and this certainly varies. When it comes to Japanese, you'll usually be able to find that it's in the standard Japanese encoding Shift JIS. And Shift JIS largely supports English letters, but because Japanese text tends to include characters that are all the same width, it doesn't often account for the fact that in English you have very thin letters like the letter I. And so when you simply insert the text into those games you might get a situation where all of the letters are spaced really far from each other. And that does not look good.Jeremy: [00:16:52] You were talking about how there's the example of script files or the example of code being hard coded or even binary files. What is the end result you'd like to get this text out for a translator to work on?Sara: [00:17:25] Typically the text you want to get into an Excel sheet because that's the industry standard for translation and keeping texts like that. Sometimes the developer will provide this themselves and that will of course, make the process of translation a little easier because somebody like me can work while the game is being translated but usually it's more a case of what is the translator comfortable working with?Are they comfortable working with script files? Do they need an Excel sheet? Most of them do and that's fine. But obviously this results in more steps. You need to create a tool chain to both extract the text and get it into a sheet, and then you have to make the call of whether you can get that text directly back into the game or if you would need to include more information, like the exact lines it's from.Jeremy: [00:18:21] The simplest would be the script file where you might build a tool to convert the script files to rows in an Excel spreadsheet. A translator works on that to translate it to English. They give the spreadsheet back to you and you run some kind of application to convert their spreadsheet back into the script files.Sara: [00:18:43] Yes, exactly. Although granted if a game is very small, you might just do it all by hand.Jeremy: [00:18:49] In terms of these script files are the lines of dialogue or the text-- are they in their original context? If somebody in a game was having a conversation would all of the sentences be grouped together or is it sometimes just spread all over the place?Sara: [00:19:14] This varies a lot. Sometimes you won't even know which character is speaking unless you actually play through the game while you're working on the script. And obviously this poses some interesting challenges because in general, Japanese is not as context heavy and context apparent as English is.You can't always tell who's talking just by the tone of their voice, so to speak.Jeremy: [00:19:41] Going back to when you have something where the text is hard coded. Is that a case where you basically have to do it by hand or have you found a process to deal with that?Sara: [00:19:53] There are a variety of ways that this can be approached too, which is part of the beauty of games. It's an art form. But what I personally like to do is simply make something that extracts all of the apparent strings, like surrounded by quotes from all of the code and gives that information along with the line number. And just like with any other script, you can usually automate this.Jeremy: [00:20:20] So you create your own version of a script even though they didn't exist originally.Sara: [00:20:29] Yeah. And then you have this list of all the strings in the game, and you can easily put those into a sheet.Jeremy: [00:20:37] When you work on localization, how much of the code base or the game engine do you feel like you need to understand?Sara: [00:20:45] It depends on the scope of the task, but typically I would say that I don't need personally to understand much of the game to start working on it, but by the time that the project is finished, I should understand almost everything except for the raw graphics the APIs being used. Even then, I might need to depending on how the game is functioning and whether we need to do something differently, like making an older game use widescreen.Jeremy: [00:21:15] And what's your strategy when you first start work on a project? You get the source files and you need to find out where the text is stored and how it's stored. What's your strategy for figuring that out?Sara: [00:21:30] First I will look through all the files I received and make sure that the text isn't somewhere extremely obvious. And if it's inobvious, I will then start looking at the source code and getting it to compile. And I will look through that to see exactly what files it's loading. And from there I can usually figure out where the text is stored.Jeremy: [00:21:52] In terms of when you get a project, do you just look at the code or do you figure out how to to run it and all that first?Sara: [00:22:01] Usually I prefer to start by looking at the code itself and getting that to compile, but typically a developer will provide a working copy of the game that you can use to see exactly how everything functions. I do like to know what kind of game I'm working on and the basics of the gameplay, the basics of the setting, because that will help me motivate myself to work on it. It will help me get into the right mindset for it.Jeremy: [00:22:31] You were talking about how there were easier ways of localizing in terms of script files and the hardest being hard coded or binary. What are some ways that if a developer is making a game that they can facilitate the easiest localization?Sara: [00:22:51] The easiest localization would definitely be first you want to make script files for everything, whether it's just the list of strings that the game is using, or a proper script file for all the events. And then you want to make sure that the game is capable of loading a different sets of these based on what language is currently in use.And if the game is already capable of loading different folders or different files for different versions of the scripts then it shouldn't be too hard to localize, but you also need to make sure that there are easy ways to work with the graphics and of course, absolutely all of the assets can be loaded differently based on language.Another problem of course, is when it comes down to stuff like fonts. Because to make a very pretty looking font, you probably want to make the font graphical rather than using a true type or open type font. And of course, the challenge with that is including every character you could need for every language as well as issues like the font could be badly spaced in certain languages and to make a game as easy to localize as possible, you probably want to use Unicode and Unicode fonts and just include everything you possibly can.Jeremy: [00:24:17] And because unicode has most languages used in the world, then you won't have the text and encoding problems.Sara: [00:24:25] Exactly. You would completely skip that step. You wouldn't need to worry about the fonts. You would just be able to put whatever language you need and you won't get tripped up by stuff. Like in Japanese games Shift JIS is the encoding used right? And if you try to insert accents like an E with the little dash over it. It will instead draw a Japanese symbol if you don't change the encoding.Jeremy: [00:24:52] You were talking about graphical fonts, so would that be a raster image, like a giant PNG or a JPEG that has all the different characters on it?Sara: [00:25:06] Yes, exactly. Usually this also has some sort of table containing the geometry of the font so that you can draw it with proper spacing and everything.Jeremy: [00:25:18] And then when you're talking about assets, that would be like textures. For example, if you had a sign in the game, that would be something that somebody would have to go into Photoshop and have a different version for each language?Sara: [00:25:34] Yes, and ideally you want to keep all of the original layered versions of this with the text on a separate layer, because that will make it as easy as possible to change the text that's on it. But of course that isn't always available. So when it's not, you need somebody who's actually able to remove the text and put something new in.There's also things like title screens of course, and any sort of special menu texts you have like that's drawn in a different font or has a special design.Jeremy: [00:26:06] Because something in Japanese versus something in English can look significantly different when you're doing a logo. So you need someone with the artistic expertise to be able to recreate something that looked like the other thing, right?Sara: [00:26:22] Yes. And of course Japanese logos are often written entirely in Japanese, so you will want to come up with a localized title for these games and then put it in the correct localized language, and that will involve basically designing a new logo that looks similar. It gets the same point across which is the big thing in localization. You want to get the point across.Jeremy: [00:26:47] One of the games you worked on was a game where you decided that the game's code needed to be rewritten. Can you talk a little bit about what the game is and why you decided that you needed to rewrite the game?Sara: [00:27:00] Well, in this case, we are talking about a game called Corpse Party. Corpse Party's origins were as an RPG maker game on an obscure Japanese computer (PC98). Obscure over here at least. And when the developers revisited it many years later, they decided to remake it in a programming language called Hot Soup Processor or HSP for short. HSP was taught in many Japanese schools because it was free and it was made for people wanting to get into things like this.Now Hot Soup Processor resembles basically the child of Java and C and in this way, everything is a little different. I wouldn't even compare it to C, honestly more like BASIC. And as a result, your game looks more like a scripting language than a programming language. And the bigger problem is that you don't have as much control over the graphics and sound APIs for games that are coded in HSP, and so to do many things with the graphics properly or get the timers right it may be better to change over to something else. Especially if you're going to put this game on something like Steam, because Steam does not have a version of their API for Hot Soup Processor. I know. Shocking.Jeremy: [00:28:30] About it being like BASIC, is there a high reliance on GOTO statements?Sara: [00:28:38] Unfortunately, yes, there is.Jeremy: [00:28:40] Was this statically typed? What was it like to kind of work in that language?Sara: [00:28:49] Well, when it comes to typing. You don't always know what type it's going to be. You don't know if this variable is necessarily an integer or if it's a float, and the compiler of the scripts largely doesn't care, and obviously that means you can run into some typing issues because you have completely misunderstood what a variable actually was. That definitely gave me some headaches.Jeremy: [00:29:16] So it's sort of like Javascript or Ruby where you can define a variable and that variable, you could put in a string, you could put an a number, you could put in pretty much anything and there isn't really a way to check. I guess it just runs through the interpreter and then you find out at runtime whether something's going to blow up.Sara: [00:29:37] Pretty much, I definitely would compare it to Ruby in that regard.Jeremy: [00:29:42] What was your strategy for rewriting a game? Were you porting function by function? How did you get started?Sara: [00:29:50] Of course, I started by just copying all of the code and then I began rewriting each function one at a time. Trying my best to understand the purpose of each function as I was going along. I knew what some of them did already because we actually inserted the translation before all of this, but the thing is that I was never going to be sure exactly what functions relied on each other and what variables were needed and how exactly they needed to be typed until everything was already basically put together. And so I honestly was pretty scared during this process at times because I had no idea if it was going to work until it worked.Jeremy: [00:30:36] You were rebuilding it function by function but because it wasn't really clear which functions relied on other functions you couldn't run the game. There wasn't this iteration of getting part of it working and moving on to the next part, you had to build everything and then you found out if there were issues?Sara: [00:30:58] Yes, exactly. Because for the most part with games, you are not building the code level by level. You are programming something that will be able to run all of the levels.Jeremy: [00:31:09] Wow, that sounds terrifying.Sara: [00:31:11] It is.Jeremy: [00:31:12] Were you able to on a function by function level, have an idea of what the function was supposed to return so you could at least test certain functions?Sara: [00:31:26] Yes. There were definitely cases like that. However, the most complex parts of the game are often more in the graphics and audio engines and such if you're working at this kind of level. And I wasn't always sure how the graphics functions worked. No fault to the developers, just that it was so much.Jeremy: [00:31:46] Since you didn't quite understand how the graphics functions worked, when you rewrote that portion of the game did you take a different approach rather than trying to recreate what they had done?Sara: [00:31:59] Well, I initially tried to recreate what they'd done, although for certain API functions, et cetera, I basically took my best guess and so when the game was finally running, it definitely didn't look or sound right. At first, there were problems, and these were entirely problems with how I had understood the code rather than anything to do with the game itself of course.Jeremy: [00:32:24] How long did it take from getting the code to actually being able to run the game at all?Sara: [00:32:31] The original version of the game pretty much worked out of the box and compiling it I just needed the version of the HSP tools that the game had been made for. But when it came to actually making my version of the game run it took about a year.Jeremy: [00:32:50] Wow. So a year before you could launch the game. That's intense.Sara: [00:33:00] Believe me, I hated myself for a lot of that project. I was like, why didn't I just use the original code? Why did I put myself through this? But the result was that the game runs very well.Jeremy: [00:33:12] One of the common pieces of advice people get when they work on software is quick iteration in terms of build the thing, see if it works, then you can build the next thing. But in this case it seems like that wasn't possible to do.Sara: [00:33:32] Yeah. Unfortunately. With games everything is going to be more tangled and depending on the way you approached creating the game you can end up with either something that's very clean and parts don't depend too much on each other, more abstracted, or it can be a plate of spaghetti. And just like any application, this is a matter of designing the application before you start building it.Jeremy: [00:34:01] Something you managed to do when you rewrote the game was make it platform agnostic. What are some examples of ways that when you write a game, you make something that can be easily run on different platforms?Sara: [00:34:15] Regardless of the platform, you are going to be dealing with different graphics and audio API APIs. Because you're not going to run DirectX on a PS4 and essentially the approach you need to take with this is mostly related to creating wrapper functions for all of the graphics stuff you need to do so that you can easily change these functions and not have to change any other part of the code.For example, you want to have functions for drawing a sprite. You want to have functions for drawing some text, and you want these to be things that you can easily change the contents of without changing any part of the actual games code directly.Jeremy: [00:35:00] And the parts that are going to be different from platform to platform, would you say that's the graphics, the sound, the input that sort of thing?Sara: [00:35:10] Yes. Basically everything that deals with IO. Aside from the fact that on pretty much any platform you're going to be able to use, fopen() without any issue.Jeremy: [00:35:21] When you rewrote the game, I think you had said that when you got the original version of the game, you had replaced the text. Was that hard coded into the game or was that through scripting files?Sara: [00:35:34] Mostly scripting files, but it took me quite a bit of effort to understand these scripting files. There definitely was some hard coded text in the game. Mostly related to the menu functions. Chapter names for example that you would encounter in the menus were hard coded.Jeremy: [00:35:52] And did your rewrite load in the original script file format?Sara: [00:35:58] Yes, actually. Although interestingly enough with this particular game, all of the original scripts were in Japanese. Even the commands and this is not common. It does happen, but usually the commands and such will be roughly in English because when you're working with programming, you're going to be writing some of it in English no matter what language you speak.But since all of these scripts were completely in Japanese, I decided to replace all of the commands such that the game could still load the original format, but it was using something sort of new in that the script commands were simply translated.Jeremy: [00:36:37] And when you refer to a script command could you give an example of what some of those would be?Sara: [00:36:42] Stuff like message, wait, display picture. Move character to the left, stuff like that, or check a variable.It's a lot like programming. It's just that you are working in a language that is created for the game and is being used by the game without compilation usually.Sometimes there's assembly involved, but when you assemble the script like this, you are just converting the commands to one, two, three, et cetera to make sure that you know what the commands are without having to use as much text and space.Jeremy: [00:37:19] So it's kind of like an enum, is that a good example?Sara: [00:37:24] Yeah. EssentiallyJeremy: [00:37:25] And those commands would use the English letters but they would be written in Japanese words?Sara: [00:37:33] They were actually written in Shift JIS full Japanese.Jeremy: [00:37:36] Wow.Sara: [00:37:37] I never encountered any other project that did that, although I understand it must've made the script files really easy to understand for them.Jeremy: Yeah. It's interesting how we sort of make the assumption when we look at code that there's going to be English in it. I guess sometimes you get surprised.Sara: [00:37:55] Yeah, for the most part, there aren't many programming languages where you can do everything in another language, but if you're creating the language yourself, obviously you have plenty of control over that and you can do anything.Even though we sort of take for granted that there's going to be English there it doesn't have to be, and I find that extremely interesting.Jeremy: [00:38:17] I don't know that it happens very often, but I think a lot of languages do support Unicode for variable names and stuff like that, I just am not sure how many people actually take advantage of that.Sara: [00:38:28] Yeah. When it comes to seeing localized variable names and source that are not in the normal character set? I would say I hardly ever see it. I can only think of one occasion and you will usually instead find that it is written in English characters, but is in that language, which in the case of Japanese is what they call Romaji.So you would have the Japanese written entirely in letters. And you find that a lot when you're localizing Japanese games.Jeremy: [00:39:01] You said after a year you were able to run the game, but there were bugs or there were different issues. How were you able to track down what bugs were in the game? Was it a completely manual process where you just had to keep playing and keep trying things?Sara: [00:39:19] Essentially, as well as having testers within the company also handle that sort of testing, what is called quality assurance. Of course, we just call it QA. But by having people iterate through the game many times we would find all of the bugs with it, essentially like you would any other program. You just have to have people using it.Jeremy: [00:39:41] Were there any bugs or odd behavior in the original version that the game was relying on that you had to reproduce?Sara: [00:39:51] I wouldn't say there were any bugs that I had to reproduce, but since I had copied the game's code as closely as possible, some of the original bugs did carry over initially and I was later able to fix a lot of them.Jeremy: [00:40:08] We've been talking about the process of making games and rewriting games. What is the language that you rewrote the game in?Sara: [00:40:18] C++.Jeremy: Is that pretty typical for the projects you work on?Sara: I would say that the vast majority of professional games are written in C++.Jeremy: [00:40:31] What makes that the default choice that people would go to?Sara: [00:40:38] Frankly, it is widely supported. You have C++ compilers for consoles. You have them for computers, you have them for all kinds of different devices. Whereas you create a game in Ruby, you have no idea if you're going to be able to put that on a PlayStation.Jeremy: So it's just the fact that everybody's using it and there's so many tools, so much support.Sara: [00:41:04] Yeah, exactly. As well as the fairly robust standard libraries and since these are available on so many platforms, it's just always been the choice ever since it came to be. Originally back in the days of like the Nintendo Entertainment System, games were primarily written in raw assembly. But obviously that's not very practical, especially for the scale of games these days.So it eventually changed to C and then C++. There have been efforts to adopt C# in more platforms, but it's kind of a slow going because it's just so deep-rooted.Jeremy: Typically when I hear about C# and games it's within the context of the Unity engine. It seems like that would make it easier for people to get started and work on projects but I don't know what the trade offs are relative to C++.Sara: [00:41:58] Typically, I would say that C++ simply is the most easy to optimize compared to other options like this because of the nature of how it's compiled. Whereas with C#, they take a more native approach where all of the code is managed code, and the native code is easier to optimize, whereas the managed code that makes up most of C# simply isn't something you can optimize much beyond how it's handled for everything.It's a global optimization, whereas optimization specific to various functions in games are better than global optimization in a case like this. The trade off is simply performance. How much you can pull out of the system before it starts choking.Jeremy: [00:42:50] And one of the big differences between the two languages is that C# has a a garbage collector. For games, is that something that's seen as not desirable?Sara: [00:43:02] Rather than having garbage collection, you need to handle all of the games memory very, very strictly. You need to know exactly how much memory everything is going to take at any given time. The reason for this is because consoles typically have much more severe memory limitations than a computer does, and if you start running over that memory, you just don't have a program running anymore. It's problematic.Whereas on PC, there are things like virtual memory to pick up if you end up running out of memory somehow, and as a result, if you have any sort of garbage collection, you're going to be wanting to do it yourself. Frankly, all memory that you're locating, you should be doing yourself. You should have your own memory manager in the game.Jeremy: You said it's probably true for all platforms, but on consoles it's more important just because of the memory restrictions.Sara: [00:43:58] Yes. And of course that is also why when you are porting a game to a console, you have to be very considerate of things like memory space and of course video RAM. It was more important back in the older days of systems like the Super Nintendo and the Nintendo 64 but it still remains important today. If you start running out of memory on console, there's nothing you can do about it.Jeremy: [00:44:27] You've ported some games from low spec hardware like the PlayStation Portable to the PC. What are some of the unique challenges related to porting a game?Sara: [00:44:38] First of all, when it comes to games on an older system or a weaker system such as the PlayStation Portable the game is made exactly for the hardware specifications and you are at the mercy of those specifications at first. For example, the PlayStation Portable, its resolution is quite small. It is something like 480 by 272 and while that is almost 16:9 it's not quite.When it comes to PC games people want options. They want the ability to use different resolutions and customize their controls and the original game is probably not going to have supported this and more importantly if you're working in a game at a low resolution like that, that is not going to look good on a PC screen.With these projects we take as many of the original PSD assets and such as we can and just redo the games assets in HD and that obviously results in a much prettier game.But that also assumes that you actually made the assets in a higher resolution to begin with. Otherwise, they're going to have to be remade, and that is a whole different ballpark of problems.Jeremy: [00:46:00] Yeah, their resolution is so low that if you were to use the original assets, they would just look super blurry or blocky if you were to port it straight to the PC.Sara: [00:46:10] Exactly right. And frankly, that would make the game look a lot worse as a whole and less people would play it because at that point you're just playing something that looks old and you don't want it to look old because you're creating a new version of it. You want it to look new and shiny and sparkly.Jeremy: [00:46:30] How does the process of porting a game differ from your experience rewriting Corpse Party? How much of it is a rewrite versus something different?Sara: [00:46:43] When you're porting the game, hopefully it's in a language that you can still use. And if that's the case then the first step is going to be ripping out all of the stuff that you can't use.For example, when you're working on a console you have to use the console developer's development kit in everything. You have to use their software and while usually this supports the same programming languages that you would be able to use on PC anyway, you won't be able to use most of the graphics functions and the audio functions and you're just going to have to do all of that over instead of copying like I did when I was porting, one language to another.I instead had to basically take guesses at what all of these functions did and then reproduce them to my best. And occasionally this would result in something I didn't understand, just like when I was working on the original corpse party game, but more often I would find that I missed a functionality and that something wouldn't change color when it was supposed to or something wouldn't play from the correct speaker or things like that.One of the latest issues that I got when I was working on my first port last minute was that I realized there were audio files that were supposed to be looping and cross fading. And I had forgotten to implement any of this because I didn't realize what it was. When it comes to porting to PC, there are two main concerns. One is making sure all the functionality is correct in the first place. The second is making sure that the game runs well on a PC because there are different threading models and different timer models that simply don't convert well to a PC. That's the case with changing platforms in general.You want to make sure that everything runs as it should, even though it's on a different platform, which I know I'm just speaking in circles there, but at the same time, you have to realize if you just put a game directly on PC and you don't change anything it is going to run a lot slower than it could because it was optimized for something else.Jeremy: [00:49:04] When you talk about different threading models, could you give an example of something running on a console versus on a PC? What are these different threading models or different choices you'd be looking at?Sara: [00:49:18] Well, for example let's say that you are programming a game that is working on a Nintendo Wii. You will find that you have to handle the graphics at this time and the audio at this time and threading is often a better choice when it's available for simple performance reasons.However, it's also the case that a lot of older games don't use any sort of threads at all. Everything is just a single thread going through that single game loop and you have to deal with the results as they come. But when it comes to a PC game, you of course want threads. You want to be able to use more of the CPU and instead of using the exact CPU number and types that you have on the console like that, you have this audio and this CPU and this console happens to have a dedicated audio processor, stuff like that. You want to use something more general on a PC that'll work on anything.Jeremy: [00:50:20] So in terms of concurrency models on a console, the software development kit for that console may already have ways that you're required to do certain things. Like you were saying, a certain way to do graphics, a certain way to do audio, and once you bring it to PC, then you have more control over how you do concurrency, how you do graphics. And so you may choose to do it completely differently than how it originally worked.Sara: [00:50:52] Yes, and I think in many cases that's preferable because obviously concurrency in something like a game is best used for optimization and making sure that everything runs smoothly without any hitches. But on a console, you are optimizing to such a specific scenario that this optimization does not carry over well and is actually harmful when you are copying it directly to a PC.Jeremy: [00:51:18] Are there any specific APIs for example OpenGL that you would choose when you want to make something that you can consistently use across platforms?Sara: [00:51:29] I would choose OpenGL or possibly SDL because surprisingly SDL exists for quite a lot of platforms. That said, I would say that Vulkan is becoming something that would be very useful in that sort of way.Jeremy: [00:51:43] In terms of the projects that you've worked on, the example you gave with PlayStation Portable, were those games using some APIs specific to the console to do graphics, rendering, audio, that sort of thing?Sara: [00:51:59] Yes. The APIs for these things are typically provided by the development kit and whether they resemble an existing API or not is pretty much up to the console creator's choice. For example, if you're working on a Nintendo 3DS, you can probably use OpenGL straight up with just a few changes. Although the optimization is still going to be a nightmare by comparison because it is completely different.Jeremy: [00:52:26] When you're working on one of these ports have you ever had to go the other way around where you had a less constrained environment and then had to move it to a more constrained environment?Sara: [00:52:40] So far, I would say no, because for the most part, PC is about the least constrained you'll get. But I still try to make sure that when I'm working on a game, it'll run on as many systems as possible, and that means trying to make it run on a toaster if I could. And when your PC is a potato, you don't really expect a fully 3D game to work right.But sometimes they can. Sometimes if you are very careful about the way you optimize the game and take a lot of constraints in what you're doing, you will make a game that looks a lot better than it should on an older PC. And I try to make that a goal of mine. I try to make sure that these games run in older environments, Up until last year I was even making a point of supporting Windows XP with all of my games and that would be more of an issue at times than you would think.Jeremy: [00:53:36] What's an example of something that you would have to do specifically for such an old operating system?Sara: [00:53:42] For supporting old operating systems you essentially are limited when it comes to your compilers and your libraries. You can't use newer versions of libraries because they weren't made for older systems. The windows kernel changed very dramatically with Windows Vista and it's only been evolving more and more since.And you are limited to using this older version of the kernel that is largely still supported, but there's often behaviors that are just a little different between a system and you won't expect them. Like you might be trying to get the size of your window rectangle and it will be off by one. Small things like that. Or you'll be drawing text and Vista might support outlining the text and XP might not.Jeremy: [00:54:32] So there's subtle bugs or small features that don't exist. So when you have to take into account whether you'll have those or not it just makes everything more complicated.Sara: [00:54:44] Yes, exactly. And you don't always have the ability to expect it because often the functions will look the same, but the behavior is simply different.Jeremy: [00:54:56] And these are operating system level calls that you're referring to?Sara: [00:55:01] Yes, largely. Although there is also the matter of APIs for things like graphics that don't exist on older operating systems. For example, you can't use DirectX 10 or later on XP.Jeremy: [00:55:15] One of the things about porting is we were talking about the things that you can't reuse. Are you able to use most of the core game loop and the logic as-is without modification?Sara: [00:55:27] Well, for the most part, most of the game code is going to stay intact like that. You will have to alter stuff like the coordinate calls for various textures because all of a sudden you're worried about different resolutions. Games running on consoles generally run on a single resolution that is their target. And if you are using a different resolution on your TV, the console is scaling the output to that.But when you are working on PC, you need to account for that and you need to account for different aspect ratios because while it's easy to say, Oh, I'm going to make my game this resolution, you might upset the people who have ultra wide monitors or the people who are still using old 4:3 monitors. They exist and ideally you want to support all of these scenarios as smoothly as you can. When you're creating a game, you can account for this better than if you're working on a game that already exists.Jeremy: [00:56:23] It sounds like a lot of work in terms of finding the sections of code that are expecting a specific resolution, a specific aspect ratio. And earlier you were talking about how your assets may change as well, because you need UI elements that can be rendered at a much higher resolution or at a different shape. It seems like there's a lot of work that needs to go into it.Sara: [00:56:51] Yes. A lot of the games that I worked on earlier in my career were older games that were made for 4:3 and in the process of working on these I realized obviously these games need to be widescreen, and so there were a lot of different approaches that could be taken. I could have converted the entire game to widescreen, but then what about the purists?So I made this approach where I could arbitrarily in the code set any graphic to be aligned to the left or right of the screen or the center. Essentially by dividing the screen into regions that were still 4:3. So for example, I would draw a UI element that was always in the bottom right of the screen on the right, and when the aspect ratio increases, this would still be drawn on the right of the screen in the corner.And by separating out HUD elements like this it gives a fairly consistent experience between different platforms without having to limit the user to a specific aspect ratio.Jeremy: [00:58:00] You've worked on a lot of different games that have been made a long time ago or more recently. What are some of the big differences code-wise you've seen in the oldest projects you've worked on versus the newer ones?Sara:[00:58:15] Well, I actually got my start in fan translation, working on things in assembly, disassembly rather. And obviously the challenges for that are very apparent. You're working with very raw code that you don't have any information on. But as I've gone along, I worked on various Japanese indie games as well as Japanese professional titles.As the time has gone along, I feel that on average these games are more abstracted and more object oriented. And back then there was very little object orientation. There were just these loops and loops upon loops. Nowadays, you'll have various objects that might be inherited from other objects.So a video game's character might be a single generic object that has most of the basic behaviors. And then on top of that, you will have the class for the player character or the class for a dragon. And they have their own unique stuff to them. And obviously the game loop has changed a lot over the years as a result of things like this.Jeremy: [00:59:27] I think that matches how software development in general has evolved because there used to be primarily procedural code, right? And now we're seeing more object oriented code, more code related to functional programming. Maybe that parallels the general software community.Sara: [00:59:49] Yes, I believe so, yeah. In this regard, the evolution does seem pretty parallel. The only limiting factor is that adoption of new programming languages is very slow.Jeremy: [01:00:01] I'm not even sure what they would have used before C++Sara: [01:00:07] C. But before that, they really did just program games in basic assembly. And occasionally in BASIC.Jeremy: [01:00:15] You worked on fan translations where you would have to reverse engineer the code. You said that was assembly?Sara: [01:00:27] Yes. Because when it comes to these games that are already compiled and you're working on them as a fan, you're not going to have any access to the source code ever.So there are two options available to from there. You can either learn how to work with the code that's already compiled by learning the assembly and learning how to debug this.Or you can in some cases to an extent decompile the code if it was made in an existing framework that's well understood like .NET or some other programming languages that exist. But usually you're going to be working with the raw assembly when you're working as a fan and you need a strong understanding of the processor that the game was developed for and the exact sort of quirks to expect.Jeremy: [01:01:18] When you're working as a fan you somehow have to determine where the text is in that game right even though everything is raw assembly. How would you even know where to start?Sara: [01:01:35] Well, initially you would have a debugger that works with the game and you would have to step through step by step until you actually see the texts being drawn. And then you basically start stepping back from there to see how the text was loaded. And so it's all a process of finding first something that uses the function you need to edit and then working your way back.Jeremy: [01:02:01] That sounds very, very time consuming.Sara: [01:02:05] It absolutely is, but it's an interesting process because you learn a lot about how different styles are used in different things.Jeremy: [01:02:16] Not only would you have to to find the text but you would have to find a way to re-insert it without the application breaking.Sara: [01:02:26] Yeah, and in some cases the text would be stored in the application itself, but usually it's going to be in some sort of container you don't have any specifications for. And then it might be in some encoding you don't know. It might even be encrypted and obviously all of the functions for you to fix that are right there, but how do you put it back in? It's easier to extract something than it is to create an archive that works exactly as the one before.Jeremy: [01:02:55] That's interesting. You have these files where you don't know how they're encoded but you're looking through the assembly code to maybe get a hint for how they were generated so you can modify or generate them yourself.Sara: [01:03:09] Exactly. And when it comes to compressions and encryptions. It definitely is much easier to get a file that's already encrypted or encoded or compressed open than it is to make a new one because you have to worry about the size of the file. You have to worry about getting the optimization of the compression right. You mess up one part of the encryption key and everything's wrong.Jeremy: [01:03:36] That sounds like pretty intense work for a fan project.Sara: [01:03:40] Yeah. I actually got my start working on just simple script engines in games like Furcadia of all things when I was young. But as it went along I discovered things like emulation and all these interesting games in a language I didn't understand. And all I could think was how could these games be English? Is there a way? And sure enough, there actually was.Jeremy: [01:04:05] That's really impressive and very cool. For somebody who's interested in modifying or localizing games, How would you recommend someone today get started?Sara: [01:04:20] Well, I would find a project that you like on something like GitHub and simply try changing things and improving things in that first. Make it your own. Essentially with something like this, you need to get comfortable working in other people's code. It's not about learning to program a program yourself. It's about learning to be comfortable in other shoes and being able to cope with the decisions they make.Jeremy: [01:04:52] I feel like general software development, a lot of it is also that right? So many of the projects people work on they're not brand new. They're something that a team has been working on for years and you're coming into that environment, living with the decisions that were made, and figuring out where to go from there.Sara: [01:05:16] Yes. With normal game development, you'll find all sorts of tutorials that teach you how to build a basic platform and game engine or such, but it won't necessarily tell you why it's built that way, and you can make games that way, but you can't really work with somebody else's games from tutorials like that.It has to all be about putting in the footwork yourself and just trying. You just have to try and you just have to keep doing it until it gets right. Just like any other skill.Jeremy: [01:05:43] After all the years of experience you've had are there certain things that you do really differently now than what you did before?Sara: [01:05:55] Oh, definitely. For one, I used to do absolutely everything by hand even stuff like text. I used to copy the text and paste it into files as I needed it and rinse and repeat. And when I was working as a fan translator, sometimes early on I would change the pointers for the text manually when I needed more space.And that is just an awful idea. Automate as much as you can, make sure that whatever you're doing, you're doing it the easiest way you can. But at the same time, don't sacrifice quality for this. But typically you won't.Jeremy: [01:06:38] So basically find the things that you are doing that are repetitive. Identify the things that you can automate and build that script or build whatever functionality you need.Sara: [01:06:52] Yes, exactly. Because if you're spending a lot of time pulling the text out or something like that, you obviously don't need to be. There are cases where you might want to do that, like if there's exactly one or two hard coded strings or just a single menu, but if the project you're working is at any sort of scale that makes that an hours long job? Do not do it manually. Please.Jeremy: [01:07:20] I think that applies broadly as well. How much pain do you need to feel before you just automate it and write that script.Sara: [01:07:32] Yes, exactly. The beauty of being a programmer is you shouldn't need to do much math because you can make the program do it for you.Jeremy: [01:07:40] Cool. I think it's a good place to start wrapping up, but is there anything else that you thought we should have mentioned or talked about?Sara: [01:07:50] I suppose another thing I would like to mention is if you're getting into localization programming for a specific language, like Japanese to English or something like that you should probably try to learn as much of the languages you can along the way. So not just the programming language, but the actual language as well.If you are working on Japanese games a lot, knowing Japanese makes your work so much easier.Jeremy: [01:08:15] What are some of the ways that you've found that knowing the language really helped you a lot?Sara: [01:08:21] Even though all of the texts for a programming language is just gonna be in like ASCII, you're going to be dealing with the fact that all of the comments are going to be in another language. A lot of the function names are going to be in another language. You're going to see a file named Zako and you're not going to know that means a small fry enemy.Jeremy: [01:08:42] Yeah, so just helping you get as many hints and context as you can.Sara: [01:08:49] Exactly. Because when you're working on someone else's code in another language, you are not going to be working with documentation you can necessarily read. All of the documentation is absolutely going to be in that language.Jeremy: [01:09:03] I guess code comments as well if those exist.Sara: [01:09:07] They do usually.Jeremy: [01:09:09] For people who are interested in checking out what you're working on or what you're up to how can they follow you?Sara: [01:09:17] First of all, obviously you should keep up with XSEED's work in general if you want to keep up with what I'm doing. But I also have Twitter @SaraJLeen and I also occasionally do Twitch streams, not of programming, but of games in general. And my Twitch username is @saralene. Sort of a corruption of my name.Jeremy: [01:09:45] You work in a very unique field of software development and I really enjoyed the conversation. Sara. Thank you so much for sharing your experience, translating and porting games.Sara: [01:09:55] Thank you for giving me a chance to talk about the technical side of things.Jeremy: Yeah, it was really fun thanks again for coming on the show.Sara: My pleasure.Jeremy: That's it for my chat with Sara. I hope you check out some of her projects like Corpse Party and Trails in the Sky. She put in a lot of work to make sure they run well on old PCs so don't worry if you've got a slow computer. As usual, a transcript for this episode is available at softwaresessions.com. Alright. See ya.

Spencer Dixon, the CTO and cofounder of Tuple, dives into the intricacies of WebRTC and its applications for remote pair programming. He explains how WebRTC facilitates low-latency communication and shares insights about video codec selection and NAT traversal. The conversation covers Tuple's shift from web to native app development, addressing challenges and optimizations along the way. Spencer also shares strategies for effective learning in programming and the role of TURN and STUN servers in enhancing connectivity for seamless user experiences.

Courtland Allen is the founder of Indie Hackers, a community for people who want to start bootstrapped and profitable businesses. It was acquired by Stripe in 2017. We discuss:Why a SPA was the wrong choiceCaching strategiesUsing firebase and algoliaFighting spamWhy he hasn't hired another engineerFocusing on the right problemsBeing a part of StripeRelated Links:Indie Hackers@csallenStripeHow to Build a Life You Love by Quitting Everything Else with Lynne Tye of Key Values (One of Courtland's favorite podcast episodes)Ember (SPA framework)Ember fastboot (Server Side Rendering)Firebase (Authentication provider and NoSQL database)Algolia (Provides search for posts)Render (Web Host)Dribbble (Site to find design inspiration)Egghead (Developer video courses, got started selling zip files of videos to mailing lists)Balsamiq (Wireframing tool, added inspirational quotes to their loading screen so that people wouldn't complain about loading time)Dan Luu (Popular tech blog with no styling)Music by Crystal Cola: OrionVisit the Software Sessions website for a full transcript of the episode.

Iris Zhang is a gameplay engineer at Riot Games on the League of Legends Champions Engineering team. She previously worked on backend services at Microsoft.We discuss:Working at Microsoft and Riot GamesFinding a role and team that fits youBackend services vs gameplay engineeringBuilding features, testing, and debugging gameplayUsing internal tools to create game logicRelated Links:Personal Site@riotnyanbunClean Code (Book used by Microsoft team)Spring Framework (Java web framework used by League of Legends)Chef (Deployment tool used at Riot)Hazelcast (Cache that Riot Games switched to)What it's like to work on League of LegendsSylas (Champion that can steal ultimates)Clash (Public facing mode worked on by backend team)Music by Crystal Cola: 12:30 AM / Orion