Aaron VonderHaar returns to the show for a deep dive on automated tests, test-driven development, and elm-program-test, a new high-level test framework for Elm. Thank you to our sponsor, Culture Amp. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 2 April 2020 Guest Aaron VonderHaar (@avh4) Show Notes 00:00:00 Intro Elm Town 50 – My favorite thing is when they don't even notice 00:01:42 TDD and automated tests at NoRedInk RSpec 00:02:19 elm-program-test elm-program-test elm-test Capybara Selenium Test.Html.Query (was elm-html-test) Test.Html.Event (was elm-html-test) 00:04:36 Why write automated tests 00:06:44 Test-driven development 00:08:55 Tests vs types 00:11:33 Test-driven development (continued) 00:13:25 Red, green, refactor TDD (Test-Driven Development) Traffic Light 00:16:18 Test-driven development (continued) “Make Data Structures” by Richard Feldman “Making Impossible States Impossible” by Richard Feldman 00:20:23 Testing at the right level 00:24:53 Testing culture in a team 00:26:43 The need for elm-program-test Robolectric 00:30:22 The elm-program-test API elm-program-test API documentation 00:32:12 Standing in for the Elm runtime 00:35:34 Testing Elm commands Elm Town 46 – You Get All Of The Chapters 00:37:49 Standing in for the Elm runtime (continued) elm-testable 00:39:46 Resolving and asserting on HTTP requests 00:43:08 Other supported effects 00:45:12 Modelling the user interface in your test suite 00:47:18 Smart DOM matchers 00:49:05 Keyboard focus tracking 00:49:48 elm-program-test vs non-Elm alternatives 00:51:53 Stability and feature-completeness 00:53:00 elm-program-test at NoRedInk 00:54:42 Testing the interface with the back end Pact 00:55:58 Related talks by NoRedInk colleagues "Writing Testable Elm" by Tessa Kelly "A Month of Accessible Elm" by Brooke Angel 00:56:53 Sign-off and outro Transcript [00:00:00] Kevin Yank: Hello and welcome back to Elm town. It's your old friend Kevin, and I am rejoined by Aaron VonderHaar back for his second episode. Welcome back, Aaron. [00:00:09] Aaron VonderHaar: Hey, Kevin, it's great to be back. [00:00:10] Kevin Yank: Aaron, you were with us a few weeks back now to talk about your work on elm-format, among other things, including a new refactoring tool that, we'll be releasing very soon, I imagine, by the time our listeners hear this. If you haven't caught that episode, [00:00:28] go back two episodes and have a listen to that first chat we had with Aaron, because we will be picking up on the threads of that conversation here today with a focus on elm-program-test, and testing in general for Elm. Richard Feldman is not the only person that, NoRedInk who cares about testing. [00:00:48] Am I to understand that right, Aaron? [00:00:50] Aaron VonderHaar: That's right. Test driven, and having a lot of automated tests has been pretty core and NoRedInk, for everybody there. So, yeah, we all get into it now. [00:00:58] Kevin Yank: I'm actually interested in hearing about the background of that, cause I would say that's maybe not true of every company that's using Elm today. But I'm aware that NoRedInk started as a Ruby on Rails application, where in that ecosystem, in the Ruby ecosystem, test driven development and automated testing in general is very strongly embedded in the culture of that programming community. [00:01:23] Is there a sense that NoRedInk's investment in testing and belief in testing flowed out of that background in Rails or, not? [00:01:30] Aaron VonderHaar: Yeah, I'd say that is definitely fair to say. RSpec is the big testing framework there, and that was definitely encouraged by Rails, the framework as well. [00:01:40] Kevin Yank: And so I guess a lot of developers who are used to having a strong testing framework on the back end, they are invested in the idea of having that on the front end with Elm. And that leads to people like Evan (sic.) working on elm-test and yourself working on this new package, elm-program-test. [00:01:59] Aaron VonderHaar: I mean, web apps in general these days, testing is highly thought of across a lot of JavaScript platforms as well. So it's not unique to Elm, but Elm kind of being a totally separate language, even though it works in the JavaScript ecosystem, kind of needs its own tools and its own way of doing things that supports the language itself. [00:02:19] In a nutshell, elm-program-test is an Elm package that you can use alongside elm-test, and it lets you write tests that are maybe at a higher level of testing than what you can do with elm-test out of the box. So the types of tests you can write in elm-program-test are similar to maybe what you might write in tools like Capybara in Rails or Selenium where you write tests saying, okay, the user is loading this page, [00:02:48] the user is going to click this button, fill in text in the input with this label, navigate to this other page, submit the form, and then check that certain things appear on the page afterwards. So writing tests at that high level of the user perspective, is what program-test is designed for. And I like to think I've made a pretty nice and easy-to-use API to let you do that in a powerful way. [00:03:10] Kevin Yank: So if elm-test by itself is for unit testing, elm-program-test is for the other kinds of tests you're going to write. [00:03:20] Aaron VonderHaar: Yeah. So certainly if you have a function or a module with pure functions in it, using the built in stuff in elm-test is the place to start. After a while, Noah Hall developed the elm-html-test, which is now incorporated into the elm-test library itself, that lets you check things on the HTML values that get produced by your views. elm-program-test goes a step further and actually lets you incorporate your init function, your update function and your view function and all the messages and everything into a single unit that you can just run user steps on and inspect the output of the view and kind of interact with your program and check what the user's going to see afterwards. [00:04:03] Kevin Yank: So tests at the program level. Thus the name. [00:04:05] Aaron VonderHaar: It took a while actually to get to that name, but I think it reflects the purpose now. [00:04:10] Kevin Yank: I can imagine, cause as you were describing the type of tests it's for, those tests I've heard called acceptance tests and feature tests and end-to-end tests. And I imagine all of those terms, were on the table to name this framework. And in the end, you landed on, “Well, actually, what do we call the thing we're testing here in Elm? [00:04:29] “We call it a program.” So yeah, it makes sense once I understand what it does. [00:04:34] Aaron VonderHaar: Yup. Exactly. [00:04:36] Kevin Yank: Hmm. Well, let's take a step back, here. We're going to get to the details of elm-program-test by the end, here, but I'd like to come at this, from kind of a first principles, “What are you trying to achieve and why?” [00:04:48] question here, cause I dare say there are a lot of people who use Elm, who are listening to this, who use it as a hobbyist or, as a first language of this type, are exploring the space and are maybe, maybe not necessarily feeling the need for a test framework as the first thing they want to invest their energy in as they're exploring Elm. [00:05:11] So tell me a little bit about your philosophy around testing. When should you start testing a piece of Elm code what should that look like? [00:05:20] Aaron VonderHaar: If you're working on a production system where you have a job that depends on code that you're writing working with other code, and many people are touching it, and it's a system that's used by real people in that environment, there's certainly value in testing in that it protects you from bugs, it protects you from future regressions, that sort of thing. [00:05:41] But there's another aspect to testing, which, I think last time we talked about some of my background, previously working at Pivotal Labs doing agile consulting, and the way I learned to use testing in development then, and specifically the test driven development practice, is an approach to kind of integrating writing tests and thinking about tests [00:06:05] into the way that you actually write the code. And you can do that through test driven development in a way that helps you think about the design process for designing the code, both the architecture of your code and the implementation. So I'd say if you're working on a hobby project that maybe doesn't have the correctness requirements of a paid job, it's certainly up to the individual. [00:06:30] But personally, I find a lot of benefit in the process of using tests to think about and educate and even help me find correct or even better solutions faster. So I kind of use it as a thinking tool as well. [00:06:44] Kevin Yank: Right. So for someone who hasn't practiced TDD, or maybe is hearing the term for the first time, how would you describe that process and the experience of using it as a thinking tool, as you put it? [00:06:56] Aaron VonderHaar: When you get into kind of the details of test driven development, one of the big things is to think about the context of the caller, the person calling your code if you're writing an API, or the user that's using your site if you are building an application. So the way it tends to play out for me is I'll have some feature that I want to implement. [00:07:19] Rather than just looking at it as a specific list of things to implement and make work, I want to look at that feature and think, okay, who is going to be using this feature? In the case of NoRedInk, we have a lot of teachers and students, so we might have. Like a student leader board for a class. We think about who is a teacher, who's going to be using this feature, what's going through their mind when they come to interact with it? [00:07:44] And then from that you kind of walk through a scenario of a real teacher in your mind, actually using the feature and try to write tests from that. Kind of the process of TDD is you think about, okay, if you tried to go through this scenario as a user, what's the first thing that the site currently doesn't do that you would run into in trying to go through the workflow and that's your first test. [00:08:10] You write a test saying, okay, as a teacher want to see the leaderboards for my students. That's your first test, and you'd say, okay, what page is that going to be on? The body of the test would be load the page where it's supposed to be, stub out some data about the class and maybe the grades in the class, and then check that certain things appear on the leaderboard, which are the things that the teacher would be looking for, [00:08:35] like maybe the name of the first student and the number of points that student has. [00:08:39] Kevin Yank: Assuming that page doesn't exist yet, that test will immediately fail. [00:08:43] Aaron VonderHaar: Yup, exactly. And in the case of a strictly type language like Elm, it may not even compile yet. So, you can often think of compiler errors as a sort of test failure in a way. [00:08:55] Kevin Yank: That does get at something, which I've heard said and said myself a number of times over the years, which is having a strongly typed language, like Elm means you have to write fewer tests. Is that something you believe? [00:09:09] Aaron VonderHaar: I would say that is sort of true. Yeah, I definitely know that argument of types versus tests. there was an interesting way I saw of thinking about this, where in a functional language functions are types of values in a sense, like you can have a variable that contains a function. So when you're writing tests, you can think of that as reducing the number of possible functions that exist that could be implementing the type annotation that you've defined for that function. [00:09:42] So in that sense, there's some similarities between tests and types in that by changing the type annotation, you could restrict what kind of inputs the function can take or what kind of outputs it can produce, that reduces the number of possible implementations you could have for that function, which hopefully reduces the number of possible incorrect implementations you could have. [00:10:04] And the same is true for tests. By adding a test, you are in a different way and using different tools, reducing the number of possible valid implementations, for whatever the function is. [00:10:15] Kevin Yank: All right. And so if your goal is to, first narrow the possibility space of what this code might do, if you're using a type to do that, maybe that's one less test that you have to write. But speaking for myself, those would be the, probably the lowest value tests I would be writing in another language. [00:10:34] And the higher value tests are still worth writing. [00:10:36] Aaron VonderHaar: Yes, exactly. So like an example is in JavaScript, let's say, you might want to write a test saying that when given these inputs, the particular function does not return null. It always returns some default value or something like that, in case of an error. In Elm, that's an example of something where the type can handle that. [00:10:58] You can enforce in the types that whatever the function will never return null. But also on the other side of the coin, in JavaScript, you aren't always writing a test that your function does not return null. For every function, certain functions, it makes sense because you know it's a risk depending on the inputs, but other functions, you know, okay, it's never going to come up, [00:11:19] or if it does come up, it's obvious what's wrong, in which case you don't necessarily need to write those tests. But certainly in Elm, there is a lot of safety that you can get basically for free or just for thinking about the types. Whereas in JavaScript, you don't have that protection. [00:11:33] Kevin Yank: So back to this TDD scenario, we've created a test to where the teacher wants to go to the page and see the scoreboard and, the compiler fails, or if you happen to luck out and the compiler passes, the test fails because the page doesn't exist. What next? [00:11:50] Aaron VonderHaar: I tend to think about different levels. So the test we just talked about is a fairly high level test. Maybe you heard the words high level when I talked about elm-program-test. [00:12:00] Kevin Yank: Yeah. I was going to say, this sounds like the kind of test you would write without elm-program-test. [00:12:04] Aaron VonderHaar: Right, exactly. And that's in fact kind of why I wrote elm-program-test to fill that gap. But even if you aren't writing that test, you can think about that as a test case anyway and say, okay, I'm going to start my local development server. I'm going to login. [00:12:19] I'm going to try to go to that page. You see it fail. In a sense, you can think of that as a manual test case. In either case, the next step would be, okay, why doesn't it fail? We've already diagnosed that is because the page doesn't exist. Okay, we'll go create the page. If there's any logic related to the creation of the page that's non-trivial, [00:12:39] we might want to write a test for a lower level detail. For instance, maybe we need to decode some page flags, for loading the page or something like that. This is getting a little bit contrived as a first page for a new test, but I think about it in layers where I write this kind of high level test describing the user behavior, [00:12:59] then I see what's failing. I think about why it's failing, and then I figure out, okay, what code needs to exist for that first step to be able to move on. And sometimes at that point I go down and write a lower level or a more unit test on some specific module maybe to implement some function that I'm going to need to wire up the test at the higher level. [00:13:24] Kevin Yank: Often when talking about TDD, we talk about the red, green, refactor cycle. Is that something that is still a real part of your workflow today or is that more like something you learn on your first day of TDD, but you kinda let go of the details of that process over time? [00:13:40] Aaron VonderHaar: It's definitely something I keep in mind, and I feel like I follow that pretty consistently, but also it's not like I have a stoplight on my desk, like, to remind me— [00:13:51] Kevin Yank: What mode am I in? Yeah. [00:13:53] Aaron VonderHaar: Right. Yeah, I think it's something I tend to maybe reflect back on if I am in a position where I'm getting stuck and I can think, okay, [00:14:04] let me run through kind of my checklist of what I need to do. Do I understand what behavior I'm trying to implement right now? Am I trying to take too big of a bite right now? Am I getting lost in the weeds? Is there some simpler version of the problem that I could focus on making work right now? [00:14:23] An example that comes up for me a lot is, okay, maybe I'll have a failing test and I know that to make that test pass, it's going to require both maybe writing some information into the database and then reading it out later. I guess this would be in the case of a Ruby test where we're on the back end. But often you'll have a test failure where to make it work, you have to do something to both change what's getting stored internally and something that's being used, reading that internal state to produce some outcome. And often what I'll do in that case is I will make the test pass by just hard-coding the output value somewhere in the code, which the way that helps is that it helps me find the exact location in the code where I'm going to need to have some logic to decide what to output. [00:15:14] And then often that'll make it obvious to me what specific value I need stored in the internal state that I need to be able to use to make that determination. So it kind of breaks the overall problem into. The output and the input step and you can tackle those separately. [00:15:30] Kevin Yank: For those who may not have heard of red, green refactor, the idea is you first write a failing test and your tests are then red because they fail. You then write the necessary implementation to make that test pass, and the tests turn green, and then having that passing test suite, you are then allowed to refactor your code to try to make the structure of it a better expression or a more maintainable version of the set of features that your tests guarantee will exist. [00:16:01] And as long as your test say green, you know, you haven't broken anything, and when you're happy with the shape of your code, it is time to write another failing test before you make your code do anything new. And so you go through that red, green refactor cycle all the way to success. Now, in your example of the scoreboard for the teacher, you mentioned hard-coding [00:16:23] a value. And the thing that came to mind was, okay, as we are writing this scoreboard screen, we might start with the empty state. So there are no students have taken this test. So there are no scores yet. And so the first test I might write is I go to the scoreboard screen and it says “no scores”. [00:16:44] And the easiest way to make that test that is currently failing pass is to hard-code a view that outputs the HTML “There are no scores” – at all times – and your test turns green. And the idea of red, green, refactor is before you then go on to add more features to this page, you need to add another test that says, oh, and if there happens to be some data in the database, then I get a different result on the page. [00:17:13] And you add that feature while ensuring that the old empty state continues to work. [00:17:19] Aaron VonderHaar: Yup. Exactly. And you were asking about how strictly I follow that. An interesting optimization for the example you just laid out is something that actually I learned to do from a product manager. But the idea is that, okay, the empty state seems like a natural starting case, but if you try to think about, okay, can I make my first test be a test [00:17:44] that is the most common test that's going to happen in real life. So in this case, okay, most of the time teachers are going to have students. So if you make your first test be having actual data, it can kind of be a shortcut in a sense because it forces you to drive out more of the implementation upfront, [00:18:02] and also gives you some flexibility from the product side to maybe descope the fine tuning of the empty state later, if you want to get a feature released sooner. [00:18:11] Kevin Yank: One of my pet peeves is opening a test file and having to scroll past two pages of really artificial, edge-casey tests before I get to the meat of what does this thing actually do, [00:18:23] so yeah, I like that approach. When I find myself leveraging the rigorous version of TDD is when I'm writing a piece of code that I don't necessarily fully understand yet, or I can't quite hold it in my head. [00:18:42] Richard Feldman has done a number of talks about how, you want to get your data representation right: he's given a talk about, you know, focus on designing data structures; he's given another talk on making impossible states impossible. And all of those things come back to getting the data structure right and everything flows out of there. [00:19:03] But every now and then I am solving a problem where I go, wow, I honestly don't know what the right data structure is going to be for this. There's just one too many features to support with a single Elm type definition, and how am I going to implement those constraints? How many of them will be provided by the type system? [00:19:23] How many of them will be enforced by the functions that act on that opaque type? And if I don't know, that's when I break out TDD and I start going, all right. One requirement at a time, here. Start with what is the most important thing this has to do. Get a green test that guarantees it will always do that, and then start thinking through edge cases, [00:19:43] start adding constraints one at a time. And invariably my implementation code starts to get ugly at that point, and forcing myself to wait until my test suite is green before I start messing with alternative approaches is really valuable. It kind of forces me to slow down, to prove to myself that an idea I'm experimenting with, actually satisfies all of the constraints that I can't hold in my head all at once yet. [00:20:12] Aaron VonderHaar: Yeah, that's a great example. And a situation that is easy to run into when you aren't quite sure what data structure or what algorithm you should be using, internally, a lot of times it's easy to do what I was describing earlier and jumped down to testing at the lower level too soon. [00:20:30] And you might start writing unit tests that are very specific to the data structure that you chose, in which case, if you later decide you want to totally change that data structure, you now have a whole bunch of tests that were tied to that old data structure that you now have to throw out or rewrite in terms of the new one. [00:20:50] So that's an example where if you have that uncertainty, it's good to be able to identify that and have your testing at a little bit higher of a level. Think about kind of the high level API that you're trying to implement and what behaviors and outcomes are being observed on the system as a whole. [00:21:08] So if you write your tests at that level, it gives you the flexibility to entirely change how it's all implemented, but still have those tests providing coverage without any risks the tests will be invalid. [00:21:20] Kevin Yank: So what I'm hearing is want to work from the outside in. Before you start defining the criteria for a tiny module in the middle of your program, you want to start by justifying its existence at the outer levels. And so everything starts by an elm-program-test test that says, this feature should do this thing for this user under these conditions. [00:21:43] You work your way in to the point where you are writing a single module in support of that. [00:21:49] Aaron VonderHaar: Yeah. Be able to think about and identify the different levels that you could test at, and make a choice that's the correct one for whatever situation you're in, about how many tests at which level you want to do. We actually have some variants at NoRedInk. We have at the moment, two different teams that are focused on building user facing features, and do most of the Elm work at the moment. [00:22:14] I'm on one of those, and my team tends to, on the one hand, have less involves features. Like we do a lot of user interface work for the teachers and changing the overall not very deep features, but features across the site. Let me give an example. So like, our team is responsible for like the dashboard that shows all the assignments or the grade book that shows a summary of all the grades, which can have a lot of Elm code, but it's not a particularly algorithm-heavy code. [00:22:48] Whereas the other feature team has been working on, like our new self-reviews feature, which is this entire interactive assignment with multiple steps. And you kind of worked through an essay section by section where there's a little quiz and you can highlight stuff. So there's kind of a state machine involved on the back end. [00:23:07] We're using a rich text editor that we had to integrate with Elm, be able to merge updates if there's conflicting changes in multiple browser tabs, things like that. So that's kind of like a deeper feature, and that team ends up doing a lot more of the more focused unit tests compared to my team where we have a lot more features that are— [00:23:27] We only need to go as deep as kind of thinking about the user experience because the code to wire it up isn't that complex from an algorithmic sense. [00:23:36] Kevin Yank: You were saying that informs the testing practice of your respective teams. [00:23:41] Aaron VonderHaar: Yeah I like to always start by thinking about what's the highest level test, but there are tradeoffs on the spectrum too. Like if you test too high, one thing is your test can be slow if, for instance, you're using Capybara or Selenium or something like that. [00:23:56] Luckily on the Elm side, even elm-program-test is very fast even for high level tests. But the other downside of high level tests is it can be harder to debug what is exactly wrong when a test fails. You might get a failure like, “Oh, the, the button with the submit text doesn't exist on the page.” [00:24:15] But it doesn't tell you why it doesn't exist. Whereas having some lower level unit tests gives you some brittleness in that a lower level interface is now protected by tests, and if you need to change the API of that module, your tests will break or have to be rewritten, but it gives you the benefit that if one of those lower level tests fails, it's going to be a much clearer idea of like where exactly the problem is in your code. [00:24:43] So there's kind of a balance you need to, you need to make between how many of each kind of test are appropriate for whatever your situation is and the experience level of the people on your team, and so forth. [00:24:53] Kevin Yank: How do you and your team think of and talk about that balance? Are there established norms? If you're reviewing a code change that deviates from those norms, it will raise your eyebrows and cause you to question it. Or is there any tooling around things like code coverage and things like that? [00:25:11] Aaron VonderHaar: Yeah, that's an interesting question. I believe there's a one or two Elm code coverage tools, but I haven't personally played with them, and we don't use them consistently at NoRedInk [00:25:22] Kevin Yank: The thinking behind the question is, for me, this is one area where front end tests and back in tests differ significantly. Maybe it comes back again to the difference between a dynamic language like Ruby or a statically type language [00:25:37] like Elm. But on the back end, it is way more common in my experience to have the, that tooling to go the code you are contributing must have a minimum level of test coverage before it is considered acceptable. Whereas on the front end. At least at Culture Amp, we kind of trust our engineers to think about and make good decisions about that and we trust them to hold each other accountable to that in code reviews. [00:26:05] But it is a much more informal process that is difficult to set clear expectations around in a growing team. [00:26:11] Aaron VonderHaar: Yes. Yeah. NoRedInk is definitely on the informal side there. Personally, my recommendation is just that aligning a team on that is pretty much like any other team process, it's going to require good communication, as much communication as possible, typically more than you think is going to be necessary. [00:26:33] And ideally pair programming is a great way to really get people to talk about differences of opinion that they didn't realize they had until they started working together. [00:26:43] Kevin Yank: Let's start to talk about elm-program-test a bit. I said at the beginning that this was new to me, but obviously this, this framework has been around a little while to, to get to a third major version, unless, you know, that is just semantic versioning at play and you had to put a couple of breaking changes in there, early in its life. [00:27:02] But, was there a time when NoRedInk was shipping Elm code without the ability to write program level tests in Elm? [00:27:12] Aaron VonderHaar: Even today, we still have a fair amount of our front end code does not have tests at that level. html-test itself was kind of an early attempt at answering the question of what kind of tests are appropriate and necessary to write for front end code in Elm. That turned out, in my opinion, to not be particularly useful for applications because you ended up unit testing your view function, which [00:27:40] by itself, the view function is very declarative. You tend to not have a lot of complicated logic in your view function. So where Elm programs get interesting is the interaction between the messages that can get produced by the view or through other means and how the update function processes those, and in particular when you start having a sequence of interactions, how the state of the program evolves over time, as a sequence of messages gets played back. [00:28:10] So being able to kind of test the view function and the update function together as a unit, and have a single test. So in our case, the origins of elm-program-test came when we were redesigning the assignment form at NoRedInk where teachers can create assignments and we have seven different kinds of assignments, all with different kinds of contents. [00:28:32] So the page itself was pretty complicated. And having only unit tests kind of available in elm-test at the time, you tend to get very, a very confusing set of tests when you have a test that reads like, okay, given that the teacher has already selected these 10 options in this way, then we're rendering a view that has a button that when clicked, produces this message, and then you have a separate test saying that, okay, given that you have this initial state, when this message is processed, here's the next state. [00:29:06] And it's just very hard to read and hard to understand and just grows very, very fast to write tests that way when you have a complex page. [00:29:14] Kevin Yank: You end up with a bunch of isolated tests about a single transition from one state to the next, and the big picture of what is the system supposed to do gets lost. And bugs can fall into those cracks. [00:29:26] Aaron VonderHaar: Yup. Exactly. [00:29:28] There was kind of a choice of like, okay, do we just reduce and slow down on the amount of tests we're writing and kind of step away from having as much coverage of this page? Or do we continue to write tests that way? And to me there was the other solution of how can we write a testing API that makes tests that we want to write and that are easy to write and read easier to produce? And actually my background at Pivotal Labs, I've worked on a bunch of test framework API in the past. There's the Robolectric framework for Android, and we did an experimental one while I was there for iOS apps. That was kind of a similar type of high level API where you could specify interactions with the user interface and see what kind of effects it produced. [00:30:15] So having that background, it was natural to me of like, oh, we just need to start writing a new API to, for this. [00:30:22] Kevin Yank: So let's talk about that API. How do elm-program-test test suites look different from the test suites that people might already be writing with elm-test. [00:30:31] Aaron VonderHaar: Typically in elm-program-test, you are going to typically define a top level definition called start, I like to call it, which basically sets up your program. It needs to specify the init function, your update function, your view function – and we'll talk about, commands and how to test commands and subscriptions in a minute – [00:30:52] but you set that all up in the first place and that produces a value of the type called ProgramTest, which basically represents your program and also its current state in the test world. And it also tracks other things like, any pending HTTP requests or other effects that are being simulated, by elm-program-test. So your typical test case is gonna say start, and then the next line you'll pipe it to some test program command, typically clickButton or fillIn and you'll say like, oh, clickButton "next", or fillIn and you'll give the label of the input field to fill in and the value to put in. And then there's another set of functions, that all start with the word expect, or there's an ensure version if you need to do multiples. [00:31:42] So you can say, ensure page has, and then give some html-test selectors, like ensure that the page has a certain piece of text on it or ensure that there's a heading with certain text in it or ensure that there's a button with this class on it, and so forth. So you basically just string a bunch of those together, [00:32:00] and if you want, you can alternate – click some things, check things on the page, click other things – and kind of go through an entire workflow as much as you feel is appropriate for whatever particular test you're trying to set up. [00:32:11] Kevin Yank: It sounds to me like there would have been a lot of work in modeling the outer layer of that system that you are testing there. like most Elm code that we write, we get to assume there is an Elm runtime there doing all of the things that we ask it to do by returning it commands, and we also assume there is a browser there that is, you know, maintaining the DOM and, sending us messages when the user interacts with it. [00:32:40] And in a way to write the kind of tests you write, your testing framework kind of needs to step outside of that system and poke at it from the outside. So does that mean you have had to basically emulate the Elm runtime itself and, at least a simplified version of a web browser? [00:33:00] Aaron VonderHaar: Yes, to some degree. And a big answer to that is something I learned from working on the Android test framework I mentioned, Robolectric, is you can kind of be as a TDD approach here to where your test framework only needs to implement the features that people actually need for their tests that exist so far. [00:33:20] So there's a lot of, for instance, web APIs that are available in Elm that elm-program-test doesn't support yet, but it's kind of set up in a way that that can be added, hopefully if people like this and have unusual, effects that they need to simulate and so forth. Hopefully people will start contributing some PRs to add and flush out the API. [00:33:42] But it's definitely been an incremental approach in developing it. And I think towards the end of last year, I decided, okay, I'm going to make an effort to add good documentation to that, clean up the API, and it was actually version three that was the first that I started publicizing it. [00:33:57] Kevin Yank: I'm trying to get my head around some aspect of it that I think you're going to clarify when you explain how you test commands and update functions and subscriptions. But the thing that's going through my head is, if my program inside of an elm-program-test suite makes an HTTP request, does that HTTP request actually occur by default? Are those things, mocked or stubbed by default or by exception? [00:34:23] Aaron VonderHaar: Oh yeah, that's an interesting question. It comes down to a subtlety in Elm where when you use the HTTP library in Elm and you call get or post or whatever, you get back a command. What's interesting to note is that the commands in Elm, when you call a function that returns a command, it hasn't actually done anything yet. [00:34:48] The command basically represents a piece of data that when the Elm runtime receives that command from your update function, it interprets that and does the actual effect. So, stemming from that, when you run elm-test, there's basically no mechanism for commands to get to the Elm runtime. Kind of, the test runner is hiding all of that. [00:35:09] So to answer that question, any commands you produce, HTTP requests, when you run them in tests, there's in fact no way for them to actually get executed and actually performed as real HTTP requests. [00:35:23] Kevin Yank: Okay, so elm-program-test isn't breaking any new rules. There are no runtime environment exceptions specifically for this test framework is what I'm hearing. [00:35:34] Aaron VonderHaar: So being able to test commands, program-test provides a way to do it, but it is certainly not the ideal. So the way it works now, and actually, on the elm-program-test documentation, I kind of wrote a guidebook of walking through step by step how to do what I'm about to describe. [00:35:51] But basically at the moment, because Elm has command as an opaque type and there's really no way in Elm code to be able to take a command and look at it and figure out what it represents – only the Elm runtime can do that. So out of necessity at the moment, the way elm-program-test works is that if you're going to use it to simulate commands or subscriptions, you basically have to refactor your program to define a new data type that you define, that lists out all the possible, commands that your program can produce. [00:36:31] I tend to call that type Effect and it's just a union type that you would define. And then you have a separate function that takes values of that new type and turns them into the real commands. And then on the testing side, you have to provide a separate function that basically looks the same, that takes your new type and turns it into the simulated commands that elm-program-test can read. [00:36:57] So that's the way it works now, but I believe Richard actually talked about this in the episode he was in, about looking at ways for elm-test itself to provide some JavaScript code that would be able to eliminate that step and allow elm-test to be able to inspect commands, [00:37:15] which would support features of elm-program-test and be able to do everything you can do now just with a bit less boilerplate and without having to refactor your current program before you can test commands and so forth. [00:37:27] Kevin Yank: For listeners who want to go back and hear the last time Richard was on talking about elm-test, that's Elm Town episode 46, in September 2019. He was on to talk about the release of Elm 0.19.1, and we talked a little bit about what was coming in elm-test and it definitely is forming a big picture now, seeing elm-program-test landing. [00:37:49] Aaron VonderHaar: You asked earlier about the complexity of simulating the Elm runtime. With the current solution, it's relatively straightforward. Basically, I have, a union type defined internally in elm-program-test that defines all the possible commands that can be simulated at the moment, which is a growing number, but basically HTTP, the sleep timer and a couple of others. I guess you can deal with tasks and things like that. [00:38:17] But actually previously, another precursor to elm-program-test, I was separately looking into how to test commands and have this thing in called elm-testable, where I was actually trying to write native code that could inspect the internals of commands and figure out what they were and destructure them. [00:38:36] And in that version of testing, I actually went through the work of writing Elm code that simulated the effect managers that Elm has internally and, like, queuing up all of the events that would happen in the event manager queue, and processing those. So that was actually a lot of complexity, but ended up not really being needed [00:38:59] with this new approach, as long as you provide a way to tell elm-program-test what commands you're trying to produce. It's relatively straight forward to implement that internally now. [00:39:10] Kevin Yank: I like that pattern of kind of replacing all of your commands with your own custom inspectable commands that get converted to real Elm runtime commands kind of at the outer layer of your program. I'm understanding that you are able to make these kind of transparent-to-you versions of Elm commands, and that leads you then to a process of stubbing out the responses to HTTP requests, for example. What does that end up looking like in practice? What are the kind of features that you need to provide test versions of to a typical Elm program? [00:39:46] Aaron VonderHaar: So inside of the Elm program, state or like the internal state of a running program, there is a couple of things that are tracked now. So one is kind of a list of, or a dictionary of all of the outstanding HTTP requests. So whenever your update function produces an effect that maps to an HTTP command, all elm-program-test does at that moment is track it as a new pending request. [00:40:15] What that means is then after you say, click a button that results in your update function, posting an HTTP to an HTTP endpoint, then after that, at any point – and you don't have to resolve it right away – but at any point after that, in your test, you can either assert that a request was made to a particular endpoint. If you want to, you can assert on things in that request. If you want to assert on the body of the request or the headers, something like that. And there's also a mechanism for stubbing a response. So you can either say that the request errored with some particular HTTP error, you can provide an okay value, with the status code, with the response body, [00:40:58] and basically at any moment after that. So if you have multiple requests in flight, you could have a test that resolves them in different orders, if that's the case, if you want to test for and so on. [00:41:07] Kevin Yank: Do you need to explicitly handle requests after they have been queued or is there the ability to, for example, set up some request handlers that say, if I get a request that goes to this URL, it will receive that response. I don't care in any given test if that has actually occurred or not, [00:41:27] I care about the effect of that response having come back that the user can observe. That's a way I've seen these kinds of tests written in the past is you set up the system first and then you knock it down by playing at the user's interactions and expectations. [00:41:41] Aaron VonderHaar: Yeah, that's a good question. That was actually the alternate API that was considered when I was first looking at what kind of API to design to expose this. In looking into it, I actually determined that that type of API, where you define your handlers upfront, can actually be built on top of the API that currently exists, where you let things get queued up and then you can resolve them later. [00:42:08] So the API you're describing doesn't exist right now. It could. There's some interest in it. I think I have probably a GitHub issue, tracking that. So that's something I'd like to build in the future. But, it does not currently exist, but totally could and could be built in a particular test helper, for an individual project if you needed. [00:42:26] Kevin Yank: Yeah. If I really wanted that, I could model a set of request handlers and then at a point in my test suite, say resolve all requests or something like that. [00:42:36] Aaron VonderHaar: The tricky part in making an API for it tends to be that it needs to be flexible enough to allow, for instance, dynamic responses depending on content in the request, but not everyone needs that level of complexity. So how do you make an API that's simple for regular usage, but also is flexible enough? [00:42:56] So that's kind of the API design issue there in making a general solution. But yeah, certainly I'd encourage anyone that wants that, it could be done, as you just described, using helper functions within your project. [00:43:08] Kevin Yank: I'm sort of mentally going down my internal list of effects that I know that the Elm runtime can handle. And is an HTTP request an example of a particularly complex one? Are there harder ones that you've had to support? I'm thinking, okay. There's random numbers. There's ports. There's viewport queries now in Elm 19. Are you having to work through those one at a time, or is the same pattern applicable to supporting all of those? [00:43:39] Aaron VonderHaar: Well at a high level, the same pattern's gonna work in that there's some internal state tracked with the running of the program. And then there's some new simulated commands that manipulate that internal state. And then there's a couple of new public functions on elm-program-test that lets you write assertions [00:43:58] that inspect that internal state or produce new messages based on the internal state. HTTP is the most complicated one that's been implemented so far. The only other big one I have at the moment is, Process.sleep or delay, I forget, which it's called an Elm. And I've started implementing a couple of the browser navigation ones like back and forward and having a back stack and integrating that with the Program.Application [00:44:28] like onRouteChange-type stuff. So all that works. So that, and the HTTP have been the most complicated so far. So, I do have a list of possible APIs, like for instance, even for HTTP, doing stuff like tracking the progress of ongoing requests and canceling in-flight request is something that isn't implemented yet. But it's on the to-do list. Random number generation, I think would be relatively easy. [00:44:54] The internal, runtime state for that is just that there's a seed value somewhere that would be used and updated. So yeah, I think probably the depth of complication has been tested out. I think it's been proved as a concept, but certainly there's a lot more APIs that still could be added to support more things. [00:45:12] Kevin Yank: I have not used elm-program-test myself, but I have written extensive test suites in something equivalent like Capybara in Ruby, and something that has happened in a couple of projects is that as soon as you start building up a nontrivial number of these tests, you start to want for a more abstract representation of your application's user interface, from the user's point of view. I've gone through the process of modeling a couple of times in my career with varying levels of satisfaction as to the result. [00:45:49] Is that something that you have had to do at NoRedInk in your test suites, and how does that go with Elm? [00:45:55] Aaron VonderHaar: NoRedInk is still largely a bunch of individual Elm apps. So pretty much every page on our site for the most part is still a separate Elm program, which means there's a limited number of interactions you can want to go through. [00:46:10] You kind of start at the loading point of that page and only do actions within the page in any given elm-program-test. We do have a couple of things like within a container, like we have these little, raised rounded corner containers that can have things in them, [00:46:27] so we have a helper that like checks and you can say, okay, look within the container that has this title, and then click the submit button within the container that says "class1" or whatever. So we have a couple of those, but I guess at NoRedInk, we're still relatively lean in our like building new features kind of fast rather than have we, we haven't shifted yet to focusing on having a comprehensive design system across the site that's hard to work with. [00:46:55] Kevin Yank: What's got me smiling is I'm thinking back on those libraries of abstract descriptions of pages and their features and I'm realizing they were very object-oriented code bases. The pattern that I've heard this called is the Page Objects pattern. And just the fact that it has object in the name makes me wonder… I suddenly want to explore what that API could and should look like. Were there any other unique challenges in getting a elm-program-test to do what it needed to do to be useful? [00:47:25] Aaron VonderHaar: Well, on the lines of UI components, elm-program-test actually does a lot of stuff behind the scenes, to work with whatever DOM structure you have. I've tried to make the API as simple to use and in the user terms as much as possible. So for instance, the clickButton function that we keep mentioning, it actually can check for a lot of different situations. [00:47:48] It can find an input of type button. It can find a button with text in it. It can find a button with an image in it where the alt text is the text that you want. It can find a div that has an ARIA label and ARIA role button. There's some functions to help you write your own custom DOM matchers if you want to, [00:48:09] but the built-in ones actually do a lot of extra work and also help provide nicer error messages in a lot of cases. [00:48:16] Kevin Yank: User interfaces get refactored as well. It's not uncommon to start with something as a text link and then decide it should have been a button, for example [00:48:26] and having to rewrite your tests when you make that change would be tedious. [00:48:30] Aaron VonderHaar: Yup. Exactly. So I've tried to also help encourage good accessibility practices as well, when possible in the default helpers. But implementing that stuff has been kind of a pain. Like the way elm-html-test is implemented at the moment, it doesn't give you a lot of flexibility, for instance to find an element and then find another element nearby, or that has the same ID, or that is the parent. [00:48:58] So I've kind of had to do a bunch of hacks internally to make the selectors work, to do some of those more sophisticated matchings. [00:49:05] Kevin Yank: I imagine things like keyboard interactions are really hard to model and reproduce as well, especially since they can vary from browser to browser as well, what user input does in a particular widget. [00:49:19] Aaron VonderHaar: Yeah. Well, that reminds me that doing focus tracking, is something that's not implemented. That's going to be a pain, someday when somebody gets interested in it. [00:49 it's for,] Kevin Yank: So right no, do you sort of fudge it by saying, “and this thing gets focus?” [00:49:35] Aaron VonderHaar: Huh? I believe at the moment. If you care about focus, you'll just have to do that manually. So you'll have to simulate a focus event. Then do the fill in to do the change event, and then do the blur event if you want to. [00:49:47] Kevin Yank: What would you sayto someone who is already testing their Elm at this level, using another language like a Capybara or something like that, and they're just going, all right, spin up my actual Elm app in an actual headless browser and interact with it. What is the value of moving these kinds of tests into Elm itself? [00:50:09] Aaron VonderHaar: Yeah. So the reason would be speed. And, fast compile time and getting the type safety within your test as well. But speed's the main reason. I guess the reason to still use Capybara would be for certain cases where you're using ports to interoperate with JavaScript and, you want to actually test the interaction between Elm and JavaScript. [00:50:33] Or with a real server backing, your code instead of just a simulated server. But yeah, I would say at NoRedInk, our CI builds are now around 24 minutes, but most of that is taken up by Capybara tests. So if we could convert all those to elm-tests, or most of them, that would cut a huge percentage off of our test time. [00:50:51] Kevin Yank: That's about the same amount of time it takes the builds that Culture Amp. I would be very interested in seeing that bell curve, for companies of a certain maturity, what is the, tolerated slowness of CI, past which engineers revolt and make the necessary changes, [00:51:11] but below which they kind of live with it? [00:51:13] Aaro,n VonderHaar: Yeah. Which I think. In my mind That's why it's good to get started with, program tests kind of right off the bat so that you get used to learning, okay, when do you actually need a higher level test that's slower and flakier? But I think given that Elm is a very stable language and that the Elm architecture is very well designed, and easy to understand, [00:51:38] Most things you're doing in Elm, you don't need Capybara tests because you can trust that Elm is going to compile to working code. So if you have tests and you feel confident that if your tests pass, your page is going to work, then that's the ideal state. [00:51:53] Kevin Yank: Last time you were on we talked briefly about the version number of elm-format and how there were a couple of, kind of, must have features that were still on the roadmap before you would call it a 1.0. elm-program-test is at version 3.2. Do those version numbers, line up with each other? [00:52:11] Is that number an expression of the maturity of elm-program-test? And I guess, would you advise people to go out there and pick it up today? [00:52:19] Aaron VonderHaar: Yeah, I would say that the 3.0 release was— A lot of effort went into that to improve the documentation, to publish some guide books about how to pick it up, how to test commands, how to test ports if you need to. And I got a couple of my coworkers to help on reviewing that. [00:52:36] Kevin Yank: As a fan of good documentation, seeing a section called Guidebooks with multiple links to fully worked examples, that makes me smile when I'm encountering a new package. [00:52:47] Aaron VonderHaar: So 3.0 definitely stable, but there's definitely some missing APIs, certain commands you might be using that aren't testable, but that shouldn't get in your way of testing the things that are currently supported at the moment. [00:53:00] Kevin Yank: This is a tool that is, part of the everyday at NoRedInk? It's not unusual to be adding some program tests to the feature you're shipping? [00:53:10] Aaron VonderHaar: Yeah. I mentioned we have several different teams now. My team uses it pretty consistently for every new feature that has front end code. The other teams use it to some degree, and I'm starting to roll out some internal documentation about what tests look like at different levels and how to think about choosing what level is appropriate for your particular feature, to kind of have more consistent approach to that across the company. [00:53:35] Kevin Yank: If you can share that, even if parts of it only apply or are the right tradeoff for NoRedInk, I think just seeing how a company of your size thinks through those tradeoffs would be super valuable to the community. [00:53:49] Aaron VonderHaar: Yeah, I'm sure I can get some version of that available publicly. [00:53:54] Kevin Yank: Your point on the speed of the way this is implemented is especially interesting to me here because we long ago got to the point where we said, okay, we can't afford to write a Capybara test for every single thing these screens do because the test suite would run forever. And so we need to test the critical features that we want to make sure never break. [00:54:19] And so that is often the happy path and any especially sensitive error states that we want to make sure remain error states. But, it means that, you know, in practice we have less than 50% of the actual features of our product covered by these or described by these tests. And it feels like you could get much closer to 100% with this. [00:54:42] Aaron VonderHaar: Yeah. I mean, the concern you will have switching to elm-program-test is that the interface between your front end and back end is not implicitly covered. So for instance, at NoRedInk, we do this by having some Rails controller tests that actually output the JSON that they would be producing and save that in a file. [00:55:04] And then our elm-tests— Well actually we have a script that like converts those JSON test fixtures into Elm files that just contain that JSON as a string and we load those in the elm-test. [00:55:17] Kevin Yank: I'm smiling because I've seen that wheel reinvented so many times now. There's the behemoth that is Pact, which is this collection of frameworks for different languages that say, “Okay, we are the framework for testing that, that contract in two directions between your front end and backend.” [00:55:36] But in my experience, it is so heavy and so featureful that it ends up being no fun to use. And at the other end of the spectrum, there are these, like homegrown, quick and dirty solutions of like, let's script together some JSON. And that tends to make people happier. But, the programmer in me hates to see that wheel reinvented over and over again. [00:55:58] Aaron VonderHaar: Well, I did want to give a shout out to, uh, two of my coworkers who both did some strange loop talks in the past about testing. Tessa Kelly has one specifically about kind of designing your Elm code to, be more testable. And she actually works on the other feature team I mentioned here at NoRedInk that tends to do more unit testing in their Elm code, compared to my team. [00:56:22] And the other is Brooke Angel, who gave a talk about accessibility in Elm and kind of using some testing approaches. And she actually worked with me on the assignment form project that I mentioned earlier where we started the initial prototype of building what became an elm-program-test, into NoRedInk's code. [00:56:41] Kevin Yank: You're doing my scouting for future guests for me. I've got to sign them both up to do an episode in the future. But, in the meantime, those talks will be linked in the show notes of this episode. So go have a watch. I'm definitely going to be doing so. [00:56:53] Well, all right. Thank you, Aaron. It was great to have you on a second time in so short a period. I feel like we have wrung the sponge that is your brain of everything that you've got going on right now. And I'm looking forward to seeing what it refills with next. And in the meantime, keep an eye out for elm-refactor that, if it's not out by the time you hear this, it will be out soon. And as Aaron said last time, he's eager to hear from people who might want to test drive it in the meantime. [00:57:24] Aaron VonderHaar: Absolutely. I've been doing some work in the past couple of weeks, so it's getting closer. [00:57:28] Kevin Yank: All right. Well, thanks again, Aaron, and thank you, listener for joining us once again in Elm Town. It's always great to have you here. Especially in these isolating times, it's nice to send a message in a bottle out into the ether and know that, it's finding your ears. Please hit me up [00:57:45] on @elmtown on Twitter or on the #elm-town channel on the Elm Slack. Let me know what you're thinking of this new batch of episodes, and if there's anything going on in the universe of Elm that you would love to hear from the person who created it, [00:58:02] please let me know. I'm filling my calendar with new recordings as we speak, so, eager to hear what you want to hear about. Until then, I'm Kevin Yank and this is Aaron VonderHarr. Bye for now.

Joseph Ni (aka Mordrax) shares the twisting tale of how an obscure 90s video game got him his first job with Elm, and how it brought him face to face with 10,000 lines of legacy code. Thank you to our sponsor, Culture Amp. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 1 April 2020 Guest Joseph Ni (aka Mordrax) (GitHub: mordrax) Show Notes 00:00:00 Intro and sponsors 00:01:42 Mordrax's history with programming Castle of the Winds InstallScript Language Reference Honeywell Enterprise Buildings Integrator Hawley Honeywell Mouse 00:05:16 Getting into front end web development TypeScript Angular Meteor Ember React 00:07:45 Getting into Elm Blake eLearning CoffeeScript Ruby Haskell 00:11:40 Castle of the Winds Castle of the Winds Python TypeScript jQuery Meteor React 00:15:10 First impressions of Elm… 00:18:00 Playing Castle of the Winds in Elm Castle of the Winds in Elm source code 00:19:36 Pacific Health Dynamics (PHD) Pacific Health Dynamics 00:22:35 Flattening state in 10,000 lines of Elm Elm at PHD ebook 00:27:17 Growth of Elm at PHD elm-jobs job listing 00:28:41 Sign-off and Outro Transcript [00:00:00] Kevin Yank: Hello again. It's your old friend Kevin, and we're back with another episode of Elm town. This episode. I'm joined by Joseph Ni. You might know him as Mordrax in the various online communities we frequent. Today he's gonna be here to talk about his journey with Elm and, how it's been used at a little company here in Australia called Pacific Health Dynamics. [00:00:22] Before we get into it, just a word of thanks as usual to Culture Amp for paying my salary while I record this. they support the show with my time and, it feels great to be able to do this in the middle of a Workday when I'm wide awake. And thank you to Culture Amp as you have probably heard by now, listener. We build a web app that companies use all over the world to invest in their people and their workplaces – to make the workplaces of the world better places to work is how I like to put it. We build our front end, at least in part, in Elm, in production. [00:00:54] And so if that is the sort of thing you would like to do for a job, you should reach out. My email address is kevin@cultureamp.com and you can visit https://cultureamp.com/jobs to see what open roles we have at any given time. [00:01:09] I'd also like to thank Xavier Ho for editing this podcast every single episode. He makes us sound great. We were just chatting the other day about some new software. He was looking at to do some live transcription of the rough cut so that you could then edit the transcript and that would cause the episode to be edited in a rough form that he could then tidy up. So he is always looking for ways to make this show better. [00:01:35] And I really appreciate that. If we sound at all good, it is mostly thanks to Xavier Ho. Joseph, welcome to the show. [00:01:42] How long have you been writing software for a living? [00:01:46] Joseph Ni (Mordrax): So writing software for living, pretty much ever since uni. And even before that, you know, I fell in love with the 286 with, QBasic and I'd be writing games and, and various calculators and whatnot pretty much my whole life. [00:02:02] Kevin Yank: So that's early. nineties, if not late eighties? [00:02:07] Joseph Ni (Mordrax): That's early to mid nineties. Yeah. Early to mid nineties. Floppy disks, five and a half inch. Those, those were the days. Yeah. [00:02:16] Kevin Yank: So what, what was your first language? [00:02:19] Joseph Ni (Mordrax): QBasic actually. Yeah. [00:02:20] Kevin Yank: Wow. Okay. Yeah, I came up on the Apple Basic. [00:02:24] Joseph Ni (Mordrax): You know, back then it was, it was all brand new. And, I was just amazed that I could actually get the computer to do something. And I was very much into kind of the D&D pen and paper role play, and, you know, and I've kind of tried to make a few games out of that. I was making along the lines of Grand Strategies, kingdom management and kingdom simulations. And then I moved on, to my first Windows machine, and then fell in love with couple of RPGs. Castle of the Winds was one of them. And that will come up again and again. (laugh) [00:02:59] Kevin Yank: My partner Jessica was a big fan of Castle of the Winds. It was her first, addiction as a video game. there are lots of fans out there who still remember that very fondly. So the programming bug, but you early, you were writing it as a hobbyist, as a gamer, as many of us were back in those days. [00:03:17] And then at a certain point, the web took over the world. What form did that take as it entered your life and, and I guess eventually became a career for you? [00:03:26] Joseph Ni (Mordrax): I kind of resisted the web for quite a long time. I was, you know, doing back ends, and I was doing a Windows desktop development for a while. [00:03:35] And before that I was, you know, at a big company, Honeywell doing a very, little known software called InstallScript. You know, when, whenever you're on Windows, then you double click on an MSI file to do an install. Powered by InstallScript, in the backend. It's a basic scripting language, and it's got, you know, basic kind of programming concepts. You've got functions, you've got, you know, variables, conditionals, and it's meant to be moving files, but, with the install, you've got a certain level of configuration of the system. [00:04:12] And within Honeywell I was in the, the EBI Enterprise Buildings Integrator. There is a lot of configurations required and depending on what version of Windows you're on, depending on what previous, I guess, updates have been installed, things can differ drastically from one install to the next. [00:04:32] So at that level of complexity, I found and discovered, was really hard to manage when you don't have proper tools and you've just got like a basic scripting language. I would've preferred to use QBasic. [00:04:46] Kevin Yank: When you said Honeywell, my mind went immediately to the favorite mouse I have ever owned: the Honeywell mouse. I had it for many years and I didn't want to give it up because there was nothing as good available to replace it with. It's the only mouse I've ever had that had two wheels on the bottom, a horizontal and a vertical wheel rather than a single ball. [00:05:07] It had two wheels and it just felt so accurate. And so reliable compared to everything else I've ever used. So what pulled you away from Honeywell? [00:05:16] Joseph Ni (Mordrax): Well, actually it was pretty much all about necessity, to be completely honest. I was writing, you know, Windows services and SOAP web services, I was writing Windows desktop applications. And my partner in crime at that time, at that startup, was doing most of the classic ASP web front end that was used in the offices. [00:05:40] And then, business didn't really go anywhere. My partner in crime left. And then I took over, took over the front end as well I kind of had to learn that. [00:05:50] Kevin Yank: So out of necessity, you mean there was no one else to do it? [00:05:53] Joseph Ni (Mordrax): Yeah. [00:05:54] At that point in time, I was exploring TypeScript, TypeScript was the stepping stone into JavaScript. I tried to kind of ease into JavaScript with the type system. Early on though, TypeScript wasn't particularly well known. [00:06:11] A lot of libraries didn't have TS, you know, definition files and so— and, and types was really easy to get around with a get around as well. So it just felt like there was a lot of, you know, kind of things in your way that we didn't contribute in the end. [00:06:30] At that time, I hadn't gone into JavaScript that much at all. So I thought, okay, let's. Let's put that aside and really get into it. Loved the language though, because that was the first language that I used where functions was considered as first class citizens. [00:06:46] Now I've got things that I can just call. Like it's just the function. It has no baggage of or concept of. Well, okay, you did have context. Yeah, you did have closure within JavaScript, but, you could explicitly just declare a function, pass it around as a parameter, and just say, “Hey, use this method or use this function, you know, for these parameters.” [00:07:10] And it didn't come with a class. It didn't come with any state of its own. That was brilliant. And I loved that. [00:07:18] Kevin Yank: And how quickly from there where you sucked down the functional programming wormhole? [00:07:24] Joseph Ni (Mordrax): Not very quick. It took TypeScript, Meteor, Ember, React, to eventually get to Elm. So I pretty much bumped into, you know, every popular framework. Drank every Kool-Aid. “Hey, this is great. Angular's gonna win.” And then 2013: “Yay. Meteor is great.” You know. [00:07:45] Kevin Yank: So you were telling me that, when you started at PHD, your Elm expertise was one of the things they were looking for. So, how did that expertise come about? where did Elm start with you and what did you cut your teeth on? [00:08:00] Joseph Ni (Mordrax): So right before PHD, I was at a place called a Blake E-Learning, and they made educational games for kids. While I was at, you know, Blake E-Learning. These guys are really, really supportive of technologies and, you know, explorations. [00:08:17] Kevin Yank: I've met a few people from Blake over the years at, like coding camps and things like that. And they always seem very open to experimenting with the latest tech. [00:08:26] Joseph Ni (Mordrax): They had Haskell groups, Elm groups, Ruby groups. They had, Elixir groups. So all of these lunchtime learning groups, and, you know, there were a great bunch of people, don't get me wrong. It was a really, really hard position, leaving it, because really, they fostered and really encouraged me, to, to learn and to really grow and, and to kind of analyze, you know, languages as well as concepts, you know, like what was good, what was bad. [00:08:58] So, you know, within Blake, that was kind of where my Elm journey really took off. Work at Blake consisted of CoffeeScript and Ember and Ruby. [00:09:07] It was actually quite interesting because I had already had, heard about Elm, you know, quite a few times just within the last year within the community, you know, listing out the new things. And then when I first joined Blake during the interview process, I was talking to Martin, the tech lead there at the time. [00:09:27] And you know, he had said, he had started talking about Elm and he said he'd built some things in it, you know. I was like,“What, what? That's, that's crazy. That's amazing.” And so, yeah, so I really started exploring that on the side. The last time that I had done any functional programming, was in first year in uni. [00:09:49] Kevin Yank: I had a similar experience that FP is something I learned at university and then did not use for a decade. [00:09:55] Joseph Ni (Mordrax): And it was in Haskell. Loved it. Absolutely loved it. I had a lot of fond memories from Haskell, especially how it unpacked complex problems using recursion. It almost just makes the problem go away, you know, by kind of reducing your, your problem space from, “Oh, how do you solve this whole problem?” to just “How do you solve the current issue that you have at hand?” On the larger scale as well, it's like, “Oh, how do you solve this, this company's whole issue?” Well, how do you solve the current issue you have at hand? And typically that is, you know, in my view, the most optimal way of moving ahead: by solving the issue that you have at hand rather than worrying or holding within your memory and, you know, spending your time and resources, [00:10:42] right, within this cycle on everything else that needs to be solved in the future [00:10:47] Kevin Yank: Well, the really satisfying thing that I think you're getting at about recursion is that you are able to set aside the big problem, solve a small problem, and then realize that by solving that small problem, you have actually solved the big problem. [00:11:05] Joseph Ni (Mordrax): Exactly. And so the big problem that you thought was a big problem, is just really the same pattern of small problems over and over and over again. But I mean, so that was, that was the one really notable theme that I had from learning Haskell. [00:11:21] Kevin Yank: I'd venture to say that combination of functional programming that you had fond memories of and the type system that ever since you had moved into JavaScript land, you were looking for something as strong as the type systems you'd worked with in .NET. So that was, I guess, an irresistible combination. [00:11:40] Joseph Ni (Mordrax): So what I'd been doing in React was writing or rewriting a game called Castle of the Winds and as many who you know, have met me would know, I keep talking about this, this is the game that I keep rewriting over and over again. [00:11:54] Kevin Yank: So for people who have not heard of Castle of the Winds, can you describe the game? [00:11:59] Joseph Ni (Mordrax): So Castle of the Winds is a top-down RPG. So tile based RPG, made by Rick Saada in the late eighties, I believe. And it was targeted at Windows 3.1. It was a, it's a very old game. But I love it to bits. It was my first RPG that, that's what got me into RPGs. You're going into towns and randomly generated caves, you know, turn-based. And you're just clicking around. It's got, your basic warrior and your mage, really simple in its concept. And that's why I thought, okay, it'd be awesome if I could recreate this, you know, relive my childhood dream in some ways, but as I've found over and over again, RPGs, you know, and, [00:12:42] this is to kids at home: don't pick RPGs that you first game to remake! I'd been remaking this game, you know, but at every opportunity using a different language. It actually started, first with Python 2, and then, you know, I was doing it with TypeScript and jQuery of all things, and that was horrendous. [00:13:04] And then I tried to remake it in Meteor, and finally I moved to React and then replaced the Meteor backend. I reached a point where it was kind of like the momentum was sapped by continual changes. Continual refactorings continual type changes. I had test cases in place to test for when things breaks, right. [00:13:30] Did this item in the inventory gets stowed away properly or did a purchase of a potion, was that purchase successful. And whenever I made refactoring changes, you know, you may break some things and I knew about it, but it kind of inhibited me from making those changes because I knew every single time I tried to do a system wide change, [00:13:52] it would then, you know, take me a few more days of debugging to fix everything again. Right at the beginning as a project ramps up, you're like, ah, okay, I'm making a lot of progress. Everything's great. You know, 1,000 lines, 2,000 lines, 4,000 lines, 5,000 lines, 6,000 lines, and then eventually you hit a point where you don't want to make system wide architectural changes anymore because it's just, you just don't know what's going to happen. And then you're constantly adding tests. [00:14:22] So it was at that point that I joined Blake, with my Castle of the Winds. All right. Let's derail with this one last time. This thing called Elm, what is, what is this thing called Elm? Let's just try it out. And so that first two weeks I was absolutely, absolutely gobsmackedly hooked onto it. It was really amazing. [00:14:48] Kevin Yank: It's good to have a specific project that you're craving the existence of, to drive your learning in something like a new language. [00:14:56] That's advice I give to junior engineers all the time. They'll say how should I learn a new programming language? And the first question I'll usually ask is like, find something you really want to build with it. Because that will motivate you through the hard parts. [00:15:10] Joseph Ni (Mordrax): What really kind of struck me was the workflow: the process of getting something working. At the beginning it was actually a frustration. So what I mean by that was, you know, when you're in Elm and you're coding, you're constantly getting these compile time errors, right? And you're like, that's slowing me down. [00:15:32] What, what's all these errors? Why are you throwing errors at me? You know, I'm just like writing a few lines of code. What is this type inference issue that you keep talking about? But then, when you read into it, or when I read it back into it at that time, it's, oh, okay, well that makes sense. You know, I'm trying to pass a, a an Int attribute into a String type, well, fine where you can't do your own coercion and I'll, I'll do it. [00:15:56] So that was kind of like slowing me down a bit. And then once I got past it and it compiled, you know, things just kind of just worked, right? I didn't have to debug it. And on the flip side, it was actually hard to debug during run time as well. So one of the things that you know, has really changed in the process, and this is future me talking, is my debugging process has completely changed from, analyzing issues during run time to just thinking about systems. [00:16:27] Thinking about the problem. And nowadays, whenever a bug comes in, I'll pause for a moment and think, well, okay, so this bug affects this subsystem. There are only two places in the code that changes this state, and it's either A or B. It's not 100% of the time that I get it right, right? It is 99% of the time that I get it right. To me, that's like one of the hidden strengths of Elm that I don't hear people talking about it. But it's there for big systems, right? And it's like, okay, so you've got, you know, it routes to a page, but that data doesn't exist. [00:17:03] Well, so the data exists in one state is the, you know, it's kind of like initialized in two different places. These are the only three places that it can ever come or disappear from. [00:17:13] What blew me away was that when, when it compiled it just worked, right. I'm not used to that as, as a JavaScript programmer, you know. When it compiles, you then kind of settle in for an hour of debugging. You think, Oh geez, what about regressions? And if you have a set of test cases, great. Well, first you spend like half an hour writing the test and then you settle in for debugging, and then you think, well, okay, if all these tests pass, then surely everything must be kind of okay. [00:17:46] Kevin Yank: You get to a moment where success requires that you outsmart yourself in order to detect the problems with the thing that you just wrote, trying to be as smart as possible. Yeah, I know that feeling. For our listeners, if you happen to be in front of a computer as you're listening to this, you should go to game.castleofthewinds.com and join me in tapping around this little world, written in Elm. [00:18:11] I have a, as you were speaking there, I went down a well, I walked around and killed a goblin and now I'm face to face with a giant rat. [00:18:19] Joseph Ni (Mordrax): You went down the well? That's the cheat way! [00:18:24] Kevin Yank: (laughs) Is it? I have— I must confess, I walked around the town a little bit and then I went, oh, there's a well there. I bet if I try to walk into it, it won't let me because there's a well in the way, and then I was, I was in a cave. [00:18:34] Joseph Ni (Mordrax): Then you were in the cave. Yeah, that was, when I was doing, what is it called? [00:18:38] A randomized dungeon generation way of getting into a random dungeon. [00:18:42] Kevin Yank: Ah, okay. It's just there for developer convenience. [00:18:46] Joseph Ni (Mordrax): I'm glad that that's still up. There's a story behind that site as well. I was remaking, you know, this game in Elm, and I thought, oh, I got to host this up somewhere, you know, and tell the whole world about it. And I just looked up www.castleofthewinds.com. Someone had the domain name and, I contacted him, Sean in the US and, you know, just kind of, laid out, you know, it's like, just splurge: [00:19:12] Oh, Castle of the Winds, so awesome. I'm a massive fan. I'm doing a remake. And he said, yeah, that's great. Look, I mean, if you ever want to host it as a subdomain here, you know, just give me the link. [00:19:24] Kevin Yank: This port of Castle of the Winds, this third remake of it, if the README as accurate, it started while you were at Blake e-learning, is it safe to say the Castle of the Winds got you your job at PHD? [00:19:35] Joseph Ni (Mordrax): Yeah. [00:19:36] Kevin Yank: I think we need to maybe set the scene for a moment and like, what is PHD? What do they do? What are they make? What is their tech stack like? [00:19:46] Joseph Ni (Mordrax): Right, so PHD is a, is a private health insurance software vendor, and we write software for a line of business app, you know, for, health insurance staff members who's on the phone talking to their clients, their members. And so our software helps, you know, with claims processing, with creating and maintaining covers payments, with distributing health insurance cards so it's, it's all around kind of the office management of, you know, the day to day's kind of use of private health insurance for members. So where they're at PHD and you know, they're a sort of corporate-y, you know, a company. Their backend's in C#, you know, SQL. So very, very much a Microsoft shop. Then this guy rings me up, and he goes, well, Hey Joe, you know. [00:20:43] You do Elm right? So what I learned was that there was a greenfield Elm project in progress and they were looking for someone to continue that project to basically lead the front end. [00:20:55] Because what had happened was that the engineer had left, so they had no one on the front end. They had about 10,000 lines of code. And so they were looking for anyone who has even vaguely heard of Elm. There was a project manager there whom I had worked with in the past, and he contacted me for that role. [00:21:15] From my perspective, that was the only Elm role in existence in Sydney. And at that point in time as well, you know, at Blake, I had spent about a week debugging this issue, you know, and so, eventually I worked out that the problem was caused by a change in some back end, field naming. [00:21:37] That then propagated to the front end and was missed, you know, and then was passed into the game state, and then, you know, it caused the game not to get that value. [00:21:50] Kevin Yank: So you were at peak wishing for the guarantees that Elm would give you? [00:21:54] Joseph Ni (Mordrax): Well, yeah, it was, it was kind of like everything, [00:21:59] every bug that I looked at, you know, just reinforced that voice in my head. This class of bugs would not exist had I had a strongly typed language. This isn't even ADT, right? I'm not even getting into ADTs yet. This is just strongly typing an endpoint so that, you know, your fields that you're passing in the JSON object exists. Okay, we can't even have that. Right. I was at peak wishing, and this opportunity came up and I was like, ah, I have to go for it. And I did. [00:22:35] Kevin Yank: And 10,000 lines of Elm code. That's not small. [00:22:40] Joseph Ni (Mordrax): No, it isn't. And you know, the thing about 10,000 lines of Elm code is, [00:22:46] you can write Elm code really, really badly, right? And I discovered that, you know, at that place, and you can write it so badly that it's actually not feasible to untangle it in some ways as well. I really had to isolate quite a lot of the systems. And then kind of build things from scratch and then put things back. [00:23:14] When I first started with PHD, I had a tendency to model my state, like the UI. And so what I mean by that is, you know, you've got pages and then each page will have their own state, and then within a page you might have some forms. [00:23:30] And then those forms' state is actually within the page state. And so when you've got that going on, then you want to communicate between that form and something else somewhere else, then you know, it's kinda like, visually, you're representing, or from a state point of view, you're representing your state, like what you see visually. [00:23:52] So you have to kind of climb out of that form, out of that page, into your menu system, into a sub menu, into something else to get communication going. So, and that's like a really hierarchichally nested, you know, architecture. So what I'm advocating for, and what I'm doing now within our system, is to flatten pretty much everything. [00:24:17] Moving forms out of pages, moving things that aren't, you know, related from a UI, like they're nested from a UI point of view, but their state doesn't have to be nested. When you have nested state, your messages also have to kind of jump, you know, the hoop to communicate between those two nested states, right. [00:24:39] But if all of those states are at one level, then you don't have to do that. That's kind of the ultimate, moving away from child to parent communication. Your UI can still be as nested as you want. You can nest your views, right? But the state that you refer to, you know, is flat, and you can kind of like, just talk between any parts of that state. [00:25:03] So that's kind of like roughly what where we're heading towards. We've still got, you know, two and a half years worth of legacy code. And it's, it's constantly evolving. It's ever evolving. But another great thing about Elm, you know, evolution is awesome, right? People say this all the time. [00:25:21] I dunno if anyone realizes how awesome it is at 120,000 lines of code, right? Like, you can, you can make a change without fear. And I imagine it would exist at 500,000 lines of code as well. [00:25:34] Kevin Yank: You've got the legacy code that has some of those older patterns, but you have managed to carve out these new patterns that you are building new things with this flattened state representation. [00:25:45] The team knows what that healthy pattern looks like and is gradually adopting it, moving things to it opportunistically, I imagine. is that kind of where you find yourself at the moment? [00:25:57] Joseph Ni (Mordrax): It does take a bit of convincing and you know, within Elm there is kind of no right and wrong, or maybe there is right and wrong. It's kind of like Lego pieces, right? And if, if you've never built yourself into a hole before, you end up building yourself into a hole to learn about it anyway. It's almost like being guided by intuition. I have to formulate, the reasons down the line for why you would not go to this extreme, but it's a lot easier within Elm to talk about these concepts because there is a lot less ways to get things wrong than there is in JavaScript. [00:26:33] Let's put it that way. Like, if you've got you know, state within a module and the module's opaque, then that really forces your hand quite a bit, you know? So it's kind of like some part of the argument would be working out where state exists and where that state exists would determine where those types and messages exist. [00:26:57] And that will really determine how, you know, other modules use it. So, you know, the, the conversation can be quite structured around scoping of module scoping of states, and then, you know, then we can get the team on board to, to solve problems in a certain way. But really that's, that's still up for discussion. [00:27:17] Kevin Yank: You told the story of how out of nowhere who you assumed was a .NET person and had never heard of functional programming, offered you the only Elm job in Sydney. Is it still the only Elm job in Sydney? Is PHD looking for more Elm fans to join the team? What's the ecosystem and what's the growth of PHD and its Elm team look like? [00:27:40] Joseph Ni (Mordrax): It's definitely not the only Elm job in Sydney anymore. We've got four strong now. you know, we're based kind of, I'd say all around Australia, around, in India as well as all around the world. [00:27:52] And we're still hiring. The product has been a pretty big success within the industry. And we need more, you know, hands on deck. We need more people to flesh out ever-growing requirements. So, definitely, if you're interested in Elm and you'd like to see first hand what this legacy system, is all about and how to write Elm at scale and think and approach problems as scale, in Elm in Sydney, [00:28:22] then yeah, contact me on Slack, and we'd like to hear from you. [00:28:27] Kevin Yank: Yeah, great. And you don't have to have attempted a port of a obscure nineties video game in order to get a job there. Just to be clear. [00:28:34] Joseph Ni (Mordrax): No. No, but we will look favorably on that. (laughs) [00:28:39] Kevin Yank: Good. (laughs) Well I think that is a great place to leave it. [00:28:41] Thank you, Joe for, the walk down memory lane. [00:28:46] Joseph Ni (Mordrax): Thanks Kevin for having me. And it was lovely having a chat about this. [00:28:49] Kevin Yank: Yeah. I can't wait to hear what's next for you. And for PHD. [00:28:53] Joseph Ni (Mordrax): Cheers. [00:28:53] Kevin Yank: And thank you, listener, for joining us once again in Elm Town. One of the nice things about being locked inside at the moment is I'm getting a lot of time to record new podcast episodes, so watch for the next episode of Elm Town coming real soon. [00:29:07] Bye for now.

Aaron VonderHaar shares the origin, history and plans for the future of elm-format and a new, related tool! Thank you to our sponsor, Culture Amp. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 19 Mar 2020 Guest Aaron VonderHaar (@avh4) Show Notes 00:00:00 Intro and sponsors Using Elm in Production at Pivotal Tracker (article) Elm San Francisco (meetup) elm-format 00:04:22 How Aaron got into Elm "Controlling Time and Space: understanding the many formulations of FRP" by Evan Czaplicki (video) 00:06:09 The origins of elm-format “A Farewell to FRP” by Evan Czaplicki (article) go fmt your code (article) Prettier 00:12:15 Avoid configuration options 00:15:53 elm-format's evolution Issue #209: Should single-line if expressions be allowed? 00:17:50 elm-format's parser parsec: Monadic parser combinators 00:20:33 Auto-correct syntax errors 00:21:55 Learn Elm with elm-format 00:23:41 Major “internals” projects 00:25:30 elm-refactor coming soon Migrating to elm/http 2.0 Html.Attributes.style 00:32:03 The Elm version flag 00:35:06 Controversial formatting 00:41:08 Where changes in elm-format come from 5 core principles issues on GitHub 00:47:05 Still beta issues by milestone on GitHub 00:49:00 Inclusion in Elm core elm-test 00:51:12 elm-refactor coming soon (continued) 00:52:50 Teaser: elm-program-test elm-program-test 00:53:54 Sign-off and Outro Transcript [00:00:00] Kevin Yank: hello, and welcome back to Elm town. It's your old friend Kevin, and I'm here today with Aaron VonderHarr. Hi, Aaron. [00:00:07] Aaron VonderHaar: Hi Kevin, how's it going? [00:00:09] Kevin Yank: Very good. Good to have you with us. Aaron is, if you don't know the name, the author of elm-format and, I think, a lot of listeners will be perking up at that because I have yet to meet someone who doesn't love elm-format. After using Elm for a little while. [00:00:27] So I think we'll have a lot to talk about today. Before we get into that though, as usual, I want to acknowledge our sponsors, Culture Amp. They pay my bills and they are letting me record this in the middle of a work day. So thank you, Culture amp. You've probably heard of all of this by now listeners, but as you may already know, we make a web application that businesses all over the world [00:00:50] we're talking 2,500 plus companies now around the world, use our web application to collect feedback from their employees, [00:01:00] to run their performance reviews and to mush the data that comes out of those things together in order to understand their people, their culture, how people are feeling about work and, and how to improve their workplace. [00:01:13] that's a mission I am very excited to be writing software for. And if you would like to write Elm in production on that kind of web app, you should reach out kevin@cultureamp.com is my direct address. Or you can visit cultureamp.com/jobs in order to see what roles we have open at any given time. [00:01:32] And also thank you to Xavier Ho, our producer who makes us sound great by cutting out all of the mistakes and trimming the boring bits from these episodes, so you get the very best Elm Town that we can make, dear listener. [00:01:47] Aaron, welcome to the show! [00:01:49] Aaron VonderHaar: Thanks for having me. [00:01:51] Kevin Yank: for those who might not be familiar with you or might not have heard your story before, tell us about yourself and what led you to Elm. [00:02:00] Aaron VonderHaar: Sure. well, let's see. When I first was getting into programming, I really took an interest and kind of had some mentors that got me interested in user experience and kind of that aspect of development, but web development was kind of the big thing when I was getting out of college. [00:02:17] And, I did a fair amount of that. I did a lot of mobile app development after the iPhone came out and, did some, a lot of big work for some Android projects. [00:02:27] Kevin Yank: Are we talking about native apps for those platforms or still on the web? [00:02:31] Aaron VonderHaar: Yep. Native apps. Actually, I did one project for this Android phone. I was actually in charge of the team that was re-skinning and developing new core apps that were going to be shipped with the phone itself from the manufacturer. I've done a lot of Android and iOS stuff. But yeah, I eventually ended up doing some consulting work at Pivotal Labs and started doing more web development there and I guess got interested in Elm… [00:02:57] …kind of on the side. I started going to the meetups in San Francisco, got to know Evan and Richard Feldman… [00:03:03] and got more and more interested in it. And eventually when I started looking around for a new job, NoRedInk was looking for people to do more Elms. So I ended up working on Elm almost full time, at NoRedInk, after that. [00:03:17] Kevin Yank: Pivotal Labs was one of those names that, when Culture Amp was first looking at Elm and you know, you do the thing of, well, who else is using it? And NoRedInk was obviously on the list, but, Pivotal was one of the early big names, I would say that, you know, they were on the Elm website… [00:03:35] …there were one or two people from Pivotal who had given talks about stuff, that they were doing there or had open sourced projects. Were you exposed to Elm at pivotal at all? [00:03:47] Aaron VonderHaar: I think I was the one that did a bit of the exposing. I'm trying to think back. We did a project, I think for Twitch where we used a bit of Elm, and I had actually brought Richard in to do a lunch talk for a few people who were interested in it. [00:04:00] I think the next thing when they started using it for was the front end for the Concourse CIS system. [00:04:06] And later, I think, I believe it's now used in Pivotal Tracker itself. But yeah, I definitely heard about it outside of Pivotal and started talking about it with a few people there. We had like a functional programming club where we would talk about Haskell and then other functional programming languages like Elm. [00:04:22] Kevin Yank: So tell me about how you got into that FP stuff, cause I do find there are two kinds of Elm people. There are the people who got into FP first. And then Elm was a cool way to, to stretch their FP muscles. And then there were the people who got into Elm first and then realized they had to learn some FP in order to use it effectively. [00:04:41] Aaron VonderHaar: Yeah, well, I would say Java was my language of choice before Elm. I guess largely because a lot of the great, IDE integrations and tooling and automated refactoring stuff that you can do with Java. So I'd say I was aware of functional programming, but had never really used it much, professionally, other than playing around with it. [00:05:03] And, kind of following. Or just reading interesting articles. Actually, the talk that I mentioned however long ago, at Strange Loop that Evan gave was kind of about the nature of functional reactive programming that was back around like Elm 0.15 or something like that. [00:05:20] Over the years and hadn't really found anything that was good for like a work environment until Elm came along. But yeah, I was definitely in the, object oriented worlds, not the functional world until Elm, and didn't really know any Haskell until after I started working on elm-format. [00:05:41] Kevin Yank: Right. Okay. So the, the FP club came later. [00:05:45] Aaron VonderHaar: We didn't really have any advanced functional programming practitioners. There were actually some guys from Walmart Labs who were working out of Pivotal's office for a while and they used Haskell. So they would occasionally stop by and, and, give us some pointers on how to learn Haskell. [00:06:01] But that was kind of more just an educational thing. Nothing that we ever really brought into our work for any of the Pivotal clients that I worked with. [00:06:09] Kevin Yank: how long were you working in Elm before elm-format started to be an itch you wanted to scratch. [00:06:16] Aaron VonderHaar: Yeah, that's a while ago. Let me try to remember here. I definitely started going to the meetups, kind of more heavily than having any big project in Elm when I was first playing around with it. So I was interested in kind of more just toy experiments rather than building any particular side project in Elm. [00:06:38] I think I probably was going to the meetups for maybe six or eight months, maybe even a bit longer before I wanted to start doing something bigger. I guess my motivation for choosing elm-format as a project was that I was really interested in editor tooling, like I mentioned, as something in Java that appealed to me. [00:07:01] And I thought Elm format would be a good opportunity to get familiar with how the Elm parser worked and how the compiler worked, without really having to commit to like maintaining the compiler itself. But I could learn something about the parsing, and have a useful product that other people could use as a side effect. [00:07:20] Kevin Yank: Right, right. I'm less interested in the specific timelines, but it's more what you're getting at there is that not everyone's first reaction when picking up a new language is how can I make the tooling better? [00:07:33] Especially for something like Elm where the compiler and the tooling around it was already considered at least at the time, to be quite cutting edge… [00:07:43] …and quite a highlight for it. most people would go, okay, I've got this new language. It's a toy. I want to play with it. Let's start building things. It sounds like you very quickly kind of gravitated towards a sort of meta project where you would get to learn about the thing by helping to build the tools for it. [00:08:00] Aaron VonderHaar: So one of the things that attracted me to Elm in the first place was similar principles, like the idea of the usability of tools that developers use in their daily work. Especially coming to the meetups where I got to talk to Evan a lot, back in those days [00:08:17] one of his core interests, especially at the time, was really looking at the experience of people trying to use the language and almost like doing user interviews with them, seeing what they struggled with and trying to solve problems. I'm a big example of that was when Elm went from using signals back in 0.15 or 0.16 to the current Elm architecture, which everybody knows now. [00:08:43] The big project of Evan’s to understand, like, what was it that was confusing about signals and made it hard to understand and how could that be solved potentially in a, in a drastically different way, which turned out to be the case there, but make it a lot easier for people to understand, especially for web developers to understand. [00:09:02] And it's that focus on usability that really attracted me to Elm, and also Elm being a relatively simple and straightforward language. And like the compiler is relatively easy to understand compared to like the JavaScript VM or Ruby, or even Python or something like that. I think that aspect of relatively simple tools for the power they're giving you, and also the usability and the experience of using those developer tools. [00:09:30] it's kind of been like a common thread for me. [00:09:33] Kevin Yank: is it a part of it that Elm is a simple enough language that contemplating writing your own parser for it, it felt within reach in a way that it didn't usually for other languages? [00:09:44] Aaron VonderHaar: Yeah, I think that's definitely the case. And having used the language as well, the syntax of Elm is relatively straightforward. Like there's a lot fewer weird edge cases, and weird syntax in it, as a language. So, yeah, I think that was part of it. [00:10:00] Kevin Yank: So did you go straight to the idea of an auto formatter? was that like immediately a need that you perceived and it was the inspiring idea that you were pursuing from the beginning or did you go through several iterations of interests before you landed on elm-format? [00:10:16] Aaron VonderHaar: That's a good question. My memory from then is kind of fuzzy, but Evan and Richard and I were talking fairly often and were having a few lunches, and I can't remember, it might even have been Evan or even Richard who thought that it was a good thing to have in the Elm ecosystem. And one of them had written up a Google Doc kind of listing the rough idea of what it might do. [00:10:42] gofmt was around at the time. So the idea was kind of based on that, that it would be basically no configuration, and kind of one way to do things. At some point, yeah. I think that the idea just appealed to me. I was kind of looking for a bigger project to do and I jumped on it and started flushing it out, started building it, started working on it in my spare time, and went from there. [00:11:05] Kevin Yank: I'm not familiar with gofmt. I would say apart from elm-format, the big auto formatter that I'm aware of that most people will have now heard of is Prettier. And at least as I experienced it, Prettier came after elm-format. But can you talk about what was inspiring about the prior art of gofmt? [00:11:22] Aaron VonderHaar: Yeah. First of all that it was a blessed tool of the official Go tools, and I haven't actually followed it that much in the recent years. So I don't know what has changed since then, but the appealing aspects were that it was an official tool supported by the core Go organization, or I don't know how they're organized, also that it didn't have configuration options. [00:11:48] So there was no customization between projects. And part of that was because it was an official tool. Basically every published Go package, I believe, was, supposed to have their code Go formatted so that any Go code do you looked at, would follow the same conventions. So that was an aspect of it. And the other aspect of it was trying to do the right thing without [00:12:15] getting in people's way. So that's a big thing I've tried to do in elm-format is not have configuration options, but also as much as possible, make everything you type format in a reasonable way without having tons of different choices for you as the person typing the code. [00:12:33] Kevin Yank: What's an example of a choice? [00:12:34] Aaron VonderHaar: One thing that elm-format is restrictive in is if you have a function call and you have the name of the function and then you have a bunch of arguments, each are separated by white space, there's a choice in those white spaces of where do you put line breaks? Do you put a line break between every item between some of the items? [00:12:53] Do you try to group certain items with line breaks? So elm-format is extremely restrictive. In that there's now three options. You can either have all the arguments on the same line as the function call. Or you can have the function name and the first argument on the same line, and then everything else is on separate lines. [00:13:13] Or you can have all the arguments on separate lines. So those are the only three options. So there's no choice to, for instance, like have the first argument on a separate line, then have the second, third, and fourth arguments on a line together, and then the fifth argument on a line. So that's an example of [00:13:30] a choice, like it's restrictive and that you can't control how it formats in some sense, but it also removes the choice, so as you're typing code you over time, learn to not even think about it, not spend time thinking on it. You just type it out, let it get formatted and move on. [00:13:47] Kevin Yank: if you are imagining more flexibility there, apart from like leaving it in the author's control, which things are grouped on one line or not. I mean, there's other things like how indented are the function parameters, are they all lined up with the first one? [00:14:02] If the first one's on the same line as the function call? Yeah, I can see all of that. so reducing the number of things, I imagine it makes your job easier as the author of elm-format? [00:14:14] Aaron VonderHaar: I guess so. I mean, one of the interesting things is that I've kind of, I've tried to drive those design choices by the code implementations. So I've really developed elm-format in kind of a test-driven way, where I've made it essentially do the simplest thing possible first, and then as we've discovered cases where that format really doesn't make sense… [00:14:41] First of all, I've, I've tried to make a pretty high bar of pushback and say like, okay, let's make sure that this is a really common use case before we start adding special cases to the formatter. And then once it is, let's look at where does this structure of code occur the most often in real world code and what formatting makes the most sense for the most number of cases where this type of code would occur in the wild, and kind of optimize the design of the format for that. [00:15:14] and from there, I've written, I don't know how many hundreds of test cases for elm-format where it's basically an input file and the expected output file. So, building over time. It hasn't really been hard to maintain because of the way I've done that, but if there's been a choice about formatting, I've tried to have the initial version do whatever formatting is simplest to implement given [00:15:42] how everything is currently factored and see how that works out and if it doesn't work out, then kind of intentionally design a specific format for how that code appears in the wild. [00:15:53] Kevin Yank: elm-format's been around long enough now that I imagine you are probably much happier with the current version of it than you were with the initial version of it. [00:16:01] Can you talk about some of the things that were janky earlier in the life of elm-format? Some things you're glad are now left in the past, or even some lessons learned, mistakes made? [00:16:14] Aaron VonderHaar: The first things that come to mind are things that are still janky, which we haven't implemented yet. The single line if statements, so it still breaks if statements out onto like five lines even if they're really short. Huh? Let me think back. I think actually, I've, I've been relatively happy with even the initial versions of elm-format. [00:16:37] And actually in the test suite still, you can find some really old Elm files that I copied in from some open source libraries from probably Elm 0.15 even, where I put some real code in there to make sure that when I ran elm-format on some real files, that the results was at least, well usable, certainly, but also better than what was there before. [00:17:06] So, yeah, I think for me, elm-format has been pretty usable almost from the beginning and has just been incrementally improved. One pain point is when a new version comes out that has some changes to the format, how do you apply that to your repository that has 40 people working on it, and they all have different branches in Git that all need to get merged. [00:17:30] So that's kinda been a challenge to deal with. But for elm-format itself, I've actually been surprised about how stable it's been to work with over time. I've been glad that I chose to write a very comprehensive test suite for basically every new formatting rule I've added. Cause that's really helped avoid issues with bugs and regressions happening. [00:17:50] Kevin Yank: What is elm-format written in? [00:17:53] Aaron VonderHaar: So it's written in Haskell which I started mainly because I forked the Elm compiler in the beginning, and that just ripped out everything that wasn't related to parsing, and started from there. And it has diverged drastically in the meantime, but that was the biggest reason to write it in Haskell. [00:18:11] Kevin Yank: So that was going to be my next question. Did you find yourself writing or implementing an Elm parser from scratch or were you able to reference the compiler’s implementation? It sounds like you grabbed the compiler's implementation and ran with it. [00:18:25] Aaron VonderHaar: Yeah, the biggest shortfall of the compiler's parser, which works great for the compiler, but for elm-format's, purpose, we need the comments and the whitespace information to be retained. [00:18:41] So that was the biggest change initially that I had to do to the parser was to kind of modify the abstract syntax tree to track all that information and make it available for formatting. [00:18:53] Kevin Yank: And I can imagine keeping that code base in a state where you could then consume further developments to the official parser probably was off the table pretty quickly then. [00:19:04] Aaron VonderHaar: That's interesting actually. So most of the parsing files that define the parsers are still quite similar, or when I've had new changes to the syntax and the official Elm compiler, they've been relatively easy to fix up. But the big issue has been, I think in Elm 0.19 or, I forget which version exactly. [00:19:29] Evan rewrote the entire, underlying library or— The old parser used to use this Haskell library called parsec, and Evan basically wrote his own new parsing library that's a lot faster, which is one of the reasons for speed gains in one of the recent versions of Elm. [00:19:48] Kevin Yank: That sounds like an Elm 0.19 thing to me. [00:19:50] Aaron VonderHaar: Yeah. So that new parser, has still not made its way to elm-format. I've kind of talked with Evan and made a strategy of how I can basically write an adapter layer, that implements essentially the parsec API on top of Evan's parsing library. And then I can just swap that in and then over time, migrate the rest of the code to use the new parser directly. [00:20:18] But that's going to be a big project and it's something that's kinda been on hold because I've ended up having other big refactoring projects within elm-format that I've wanted to do to the code. So that's still a big pending project that has been on hold for a while. [00:20:33] Kevin Yank: Okay. Are there any other kind of interesting major features of the internals of elm-format? [00:20:41] Aaron VonderHaar: Well, there's one of my favorite features is that there are certain syntax errors that elm-format will correct automatically. [00:20:50] And one of my favorite things is pairing with someone and they make one of those mistakes and it gets fixed and they don't even notice. The biggest example is if you mix up colons and equal signs, either in a record expression or in a record type, it'll fix those to the right ones. [00:21:08] Kevin Yank: Yes, and becoming aware of those, suddenly it's like you have a third hand to help you type. It's you— You realize, okay, I've just defined a record type. Now I need to write an initializer, like for the default value of it, I can just copy, paste and save and it'll change all of my colons to equals signs for me. [00:21:29] Aaron VonderHaar: Yup. Exactly. Yeah, so that type of feature is something I'd, I'd like to add more things like that. For instance, when you paste that, if you have arrows for type functions, those don't parse correctly. So maybe doing something with those. Also, like removing things like trailing commas, and what are some other good examples? [00:21:50] Oh, fixing indentation of case expressions, is another good one. [00:21:55] Kevin Yank: My favorite thing early on about Elm format was not anything to do with, you know, getting consistent code formatting or anything like that as a learner of the language Elm format was my first feedback on whether the thing I had just typed made any sense at all. Like was it syntactically correct? [00:22:15] Because if the formatting snapped into place, it was an immediate, instant feedback that yes, you did something right. You typed the Elm correctly, and it was like just a squirt of serotonin of “Good boy. You did something well here.” And that, for me was my first kind of, experience of the developer delight in the Elm ecosystem. [00:22:38] So it was even before my code got to the compiler, it was elm-format that was telling me, Nope, try again. Nope. Try again. Yes, you did it! [00:22:47] Aaron VonderHaar: Yeah. That's great to hear. that's an interesting, because basically all editors and tool plugins make use of, so it's something that's consistent across editors. Whereas other things like getting syntax errors highlighted, kind of have a different experience across different editors. [00:23:05] Kevin Yank: It was nice to have the two tiers of feedback as a learner where it would say, no, that's not quite right. And if you want to know what you did wrong, you can go and read the compiler output. Or if you want to challenge yourself to realize what you did wrong and try and fix it yourself, you can just keep [00:23:21] tweaking and saving and, and try one more thing to see if, if elm-format is happy, and at a certain point you go, okay, I give up. Tell me, compiler, what did I do wrong? But it was nice to have that quicker feedback loop of, “Oh, I'll try one more thing. I'll try one more thing. Oh, I got it right by myself! [00:23:38] “I've learned something.” [00:23:40] Aaron VonderHaar: Yeah, that's great. [00:23:41] Kevin Yank: Let's see. Where were we? We were talking about internals. [00:23:44] Aaron VonderHaar: Oh, yeah. Well, so there's a few big, big projects within Elm format. Adopting the new parser code is one that I mentioned. Another that's been going on has been having a way to transform Elm source code into JSON files and back and forth. And the idea for that is that it would allow elm-format to kind of be [00:24:11] a translator or a reusable parser where you can turn your own files into JSON, you can write tools in what other language you want to transform the JSON and do whatever kind of fancy code transformation or linting that you want to do, and then turn that back into Elm code if you want to. [00:24:30] Kevin Yank: Would the JSON be kind of like a static version of an AST? [00:24:35] Aaron VonderHaar: Yup, exactly. [00:24:36] So that's something that's been in progress and I've been trying to make the JSON representation of the AST be something that is easy to modify without making modifications that produce invalid code. So that's been kind of an interesting project. And another kind of internal feature that elm-format had when Elm 0.19 came out is that I wanted to [00:25:03] experiment with the ability to automatically upgrade code. So in Elm 0.19, there were a bunch of changes to like the core library, the Http library, the Html library. So I started writing some code hidden in elm-format that could basically be an upgrade mode that would transform your code from the old API to calling the functions correctly in the new API. [00:25:30] And I've been continuing to kind of expand that, and I'm actually going to be spinning off a new tool currently called elm-refactor. You know, creativity in naming for Elm. [00:25:41] and the idea for that is you'll be able to define definitions of how to upgrade code. Say if you're working on a library and you have a major version change, you'll be able to write an upgrade definition that will allow elm-refactor to automatically upgrade people's code from the old version to the new version. So that's something that's getting close to working. We've been using it a bit within NoRedInk for some UI style guide upgrades. And, I'm hoping to get it ready for people to beta test, in a month or so. [00:26:16] Kevin Yank: Is the Http 1 to Http 2, is that kind of like an example of the ambitious version of what you are trying to solve for? [00:26:24] Aaron VonderHaar: Yeah. I think that would definitely be the ambitious one. Actually one of the weirder ones in Elm 0.19 was in Html where style attributes went from taking a list of styles to being a single style, which is now you have a list of items at the attribute level. So kind of doing that transformation was one of the weirdest ones where you have like a single expression that in a certain context now becomes a list of expressions that have to be spliced into the containing list. [00:26:53] Kevin Yank: It's just tedious enough that you don't want to do it by hand. The thing we had to do a lot of, in our transition to Elm 019 was top-level destructuring in Elm modules where at the top level of a module instead of saying function name equals you would actually [00:27:10] have a tuple or a record destructure and saying the function returns two values in this structure, and you can reference those names elsewhere in your module. And that was a feature that was removed in Elm 0.19 and we were using it for our style sheets integrations. [00:27:27] So a CSS module would return a className function, a classList function, and a third one. And you could, you could grab any one of those from the top level declaration. So every single CSS module in our code base, we had to manually convert to three separate functions, which, return a field of the styles function. [00:27:51] Aaron VonderHaar: So, yeah, that's the type of thing where being able to convert your files to JSON, transform them and convert them back, could allow people to build quickly, hopefully some custom tools to do whatever they need to, transform a huge code base. [00:28:06] Kevin Yank: Yeah. I wanted to ask you that. What is driving that desire to be able to have that JSON representation to do transformations on? Are there specific use cases at NoRedInk that are driving that, or is there some particular tooling you have in mind for the Elm community that you'd like to enable? [00:28:24] Aaron VonderHaar: Yeah. I think this comes back to tooling for Java where there's still a lot of things I miss from doing Java development, specifically automated refactoring tools. Elm tools have a bit of support for things like renaming things. But there's a lot of other interesting tools like introduce a parameter to this function and across your code base, set this as the default value, as an example, or select some expression in the body of a function and say, extract this expression to be a parameter to the function, or extract this to be in a let block, or take a particular definition and inline it to everywhere that it's used. [00:29:09] Those kinds of transformations are the type of things that I constantly find myself missing from using Java, for instance, and figuring out a way to provide tooling authors the support that they need to be able to quickly develop those features. Cause the problem right now is if anyone wants to implement any of those, they first have to implement a parser. [00:29:35] And that parser has to be written in whatever language they're using. Because like, hey, they could copy the parser from Haskell, from Elm compiler like I did, but that's not gonna work if they're writing a VS Code plugin that has to be written in JavaScript. So that was the idea for Elm format was that, hey, if we can have an existing tool that editors already use and leverage [00:29:58] be available to just convert code into a safe data structure that can be easily parsed in whatever language you want to write further tooling in, then that would really open the door for people to write more advanced tooling or even custom tooling specific to a project. [00:30:15] Another example for the elm-refactor tool that I mentioned: it looks like one of the options that I want to add to that is being able to normalize your imports across the code base. So you'll be able to say something like, refactor import Maybe, for instance, exposing some specific things. Html, for instance: say you can say, import Html exposing everything, and then it'll be able to go through every Elm file in your project, change however those files import Html to match the way that you specify, and then also update all the references throughout the file where it uses something from Html, and if it used to be Html.div, but now it's exposing everything in the import, it'll just change it to div since it can be accessed directly. But also have the tool be smart enough to know that, oh, if something else is in scope defined locally called div, it's not going to change that cause that would break your code. [00:31:17] Kevin Yank: Yeah, right. Wow. So it's like kind of managing a virtual global namespace across your code base. [00:31:25] Aaron VonderHaar: Yup, and especially apply consistency when you have a large team of developers that having the details of how you expose each module on those habits are kind of not worth spending time on trying to communicate it across a large team. But if you have a tool that can do that automatically, it'll make things nicer and make a problem of communication about the fine details of a style guide go away. [00:31:50] Kevin Yank: And save those dozens or hundreds of times a day that an engineer in your team needs to scroll to the top of the module and go, how was that imported again? [00:32:00] Aaron VonderHaar: Yup, exactly. [00:32:02] Kevin Yank: Amazing. [00:32:03] There was a certain point at elm-format where you needed to introduce a version flag so that the invocation of album format would tell it which version of Elm it should be targeting. [00:32:15] can you talk about, you know, what pushed you over the edge to need to do that? And what was involved in implementing that? [00:32:22] Aaron VonderHaar: Yeah. So that is still there actually. It's just that now, almost everyone's using 0.19, so that's the default. And also, with some contributions from some other people, like Martin. There is now some automatic code to try to guess what version it is, and if it can't figure it out, it'll ask you to specify it. [00:32:43] But that's still there. That is because there were some incompatible syntax changes between certain Elm versions. I can't remember if I had to add that in 0.17 but certainly in Elm 0.18. It was actually the way characters are escaped in strings is different. So I think that was the main feature that kind of forced me to add it because I wanted to be able to just parse the file, irrelevant of what version of the language it was as much as possible, and then only when we format, based on what version we wanted to output, escape things in the correct way. Because the AST itself isn't different for representing strings; it's just how they're represented in the in the output file. [00:33:36] Kevin Yank: Did that have negative side effects on the internals? Like was there a splintering of, the code that supports old version of Elm versus the code that supports new version of Elm, whereas before there was just a single implementation? [00:33:50] Aaron VonderHaar: It's actually been surprisingly smooth. The worst part about it is that I have an ‘elm’ parameter that I've had to thread through a bunch of functions, but that's basically been the worst of it. I just thread that through where it needs to go and then case on it and then do the correct thing. There's only really minimal differences between how things are formatted. For instance, like there used to be direct syntax for range expressions, and I basically parse that into the AST and when we print it, I check the Elm version and if it's after Elm 0.18, or whichever version it was removed in, I just print it out as a call to List.range with the two parameters. And if it's still Elm 0.17, I print it with the square brackets and the two dots. So it's been surprisingly simple to maintain, and actually it's been more of a pain to remove old versions. Actually most of them format still supports Elm 0.16. I did actually remove that as an option from the command line tool recently, but most of the internals still support all the way back cause it's just been easier to keep maintaining it than to go through and remove it. [00:35:06] Kevin Yank: one of the things you talked about was that the importance of a tool like this, having no configuration options that it is the standard and the benefit of accepting that and shortcutting all of those potential conversations and hand-wringing over configuration is, way greater than the cost of adopting any one formatting decisions you might not, you might not agree with. [00:35:32] what aspects of the formatting that Elm format does felt like. You know, easy and uncontroversial. What aspects feel a little more controversial? what do you still get feedback about to this day that people are grumpy about? [00:35:46] Aaron VonderHaar: Yeah, that's been interesting. And, I think kind of the incremental approach of adding features that I described before has educated that a lot. [00:35:55] Certainly the most controversial thing currently and still is the amount of vertical spacing. That kind of means a couple different things. One is the spacing between top level definitions, which is something that was one of the big things that Evan talked about when I [00:36:17] first was going through ideas. And for him, like, what made code readable – cause he's been designing the language with a particular way of reading it and a particular way of writing it in mind – so that's one of the things that came from him early on. And, I mean, it's something that people bring up, but as also something that I've been trying to watch for. [00:36:40] Where does that actually affect and cause a real problem for people? Like, okay, it's very common to have a lot of stuff on the screen that you need to see all at once. But I haven't been able to find good use cases to support changing it. So that's something that stayed. One of the big controversial items was the indent level. [00:37:03] Two versus four spaces, and it's actually something we experimented with both ways. If you've used elm-format for a long time, there were different versions with each. [00:37:13] That's another thing where, I mean, occasionally you get people who come to Elm and they want things a certain way and they complain about it. [00:37:22] But in pretty much every case I've seen when people start using it, they get used to it and it's, it's not a big deal later on. And actually I did hear from one person who was like, “Hey, my company says that all code must be indented with two spaces. How do I configure that for elm-format?” But I said, well, you can't configure it. This is just the way it is. And he was like, “Well, okay, great. I'll tell that to my boss and we'll use four spaces.” [00:37:52] But yeah, I mean, that's something where there's always going to be debate, but I feel like luckily I'm someone who isn't bothered by a lot of criticism and I feel like I'm capable of looking for what merit there is and keeping that in mind and thinking about, okay, are there changes to the formatting I want to try making or experiment with, or do some research on and consider it and make the choice. [00:38:22] And it's kind of like my responsibility as the maintainer of elm-format to do that assessment. Not get stressed out by people having different opinions and making a choice. And an interesting thing to keep in mind, I think, is that while there are people who are vocal about not liking a certain way a certain thing formats, there's a lot of other people who are perfectly happy with the way it is and get a lot of value out of, for instance, having a single unconfigurable tool that just does everything without having to worry about it. So you have to keep in mind that people who are happy with the way things are, and it's kind of my job to help protect that as well and make sure that the tool doesn't evolve into something that becomes painful to use down the road. [00:39:11] Kevin Yank: So you're saying you have to remember that silent majority is there, even if they're not always telling you. [00:39:16] Aaron VonderHaar: Yeah, and it can be kind of hard to assess that cause you don't interact with most of the elm-format users day to day. And if they aren't interacting on the GitHub issues or via other means, then it's hard to know what they want, but you know, I try to do my best. [00:39:32] Kevin Yank: You were talking about the vertical spacing and the horizontal indentation, like those two things are not disconnected. I imagine if something, if code is very vertically spaced, then if it weren't also generously, horizontally spaced, then it would look a little out of balance. [00:39:49] To what extent do you feel like you are making decisions based on aesthetics versus technical considerations or some other benchmark of success? [00:40:01] Aaron VonderHaar: I would say that aesthetics would not be a concern, but the related concern of usability would be. I think if, for instance, I had evidence that a certain formatting choice, for instance, reduced the cognitive load for people reading it or made it easier to parse or, within a certain block of code, [00:40:28] what are the important things to notice when you're looking at it for the first time, or when you're scanning it, when you know something's there and you're trying to find it, like what formatting makes it the fastest and lowest cognitive overhead to find those things? I think those would be good reasons, in my opinion. [00:40:46] I mean, I haven't gone into that much depth on a lot of things, but certainly if there was some big controversial choice, and maybe over time I'll have more time to think about those usability aspects of some of the formatting and make changes based on that. Those types of concerns would be valid to me, [00:41:03] but I'd say aesthetics independent of that hasn't really been a concern. [00:41:08] Kevin Yank: I was wondering if we could drill in on a particular change that was made relatively recently in the life of Elm format and discuss, you know, what led to it, why it was made, how successful you consider it. The one that pops into my head, I don't know if you have a better example in mind, is the way let-in blocks were indented differently at a certain point for— I feel like for a long time, [00:41:32] the body of the in was further indented from the in and then a certain point it got out dented. You know what leads to a change like that? What would conversations started it? [00:41:44] Aaron VonderHaar: So that's an example of, a change that was motivated by kind of a core principle of elm-format. So even early on there was a list of five or six things, and I still in the elm-format README near the top. I don't have them memorized, but the one that was relevant here was that we wanted to minimize the noise. [00:42:08] When you are diffing files. So like when you have a version control change, we want to avoid unnecessary changes to lines that aren't actually affected by a change. In the case of the let block indentation, that was basically the reasoning behind it. if you have a body of code, or the function body of some function and it exists, [00:42:32] and then later you're going to refactor that code and extract some local variable, that variable has to be defined in a let block. So you're going to have to add a let block to a body of code that was already there. And in doing that in the old way, that would mean that the code that existed there before has to get indented, [00:42:52] which by default in most diffing tools is going to highlight those lines as being changed. Certainly there are options in diffing tools to hide white space only changes. But given that that's not the default in most places that people are viewing diffs, making the change to have the in body of the let blocks not be indented means that if you add a let block somewhere, it's not going to [00:43:18] cause the diff tools to by default highlight everything else that's inside there that didn't actually change other than some new variable is now referenced. [00:43:28] I think it was made in an experimental version first to try out and see what it felt like for use in the real world. [00:43:36] but I do also remember that, I was kind of tracking my own experience and other people's before making that change, as to how often that happened. And I did find for myself that I would quite often make those kind of changes and see that happen. [00:43:54] Kevin Yank: Can you recall whether for this particular change or more generally, do ideas like these generally come from the community and flow into your process? Or are you dreaming up these improvements yourself based on your own experience most of the time. [00:44:07] Aaron VonderHaar: I'd say it's about 50/50. So the GitHub issues page for elm-format. I'm happy if, if anyone drops any ideas they have, I'll tag it with the discussion label on there. So there's a huge list there. I probably tend to go back to things that people brought up, like years ago, even if they don't even follow it anymore. [00:44:30] So there are a fair number of those. My time that I have available to invest in elm-format varies a lot, month to month and even year to year. But there are a large number of those, and I try to get through them over time and, mark them for particular releases, and when I'm going to get to considering them. [00:44:50] And then there are a lot of things that just come from me. But I would say most of those are not controversial. Those are more things that are just such a small detail that no one else is even bothering to think about it. An example being like the colon and equals, automatically fixing that was something like, I was in the code. I was like, oh, that would be really easy to implement, and I bet that would help. So I tried it, it worked. But that's something that probably no one else is gonna be spending the time thinking about in detail to even to think to suggest it. [00:45:21] Kevin Yank: For me, a recent, discovery in elm-format that delighted me was the auto escaping or auto encoding of, characters in strings. [00:45:31] Yeah, the way you can, you can paste a rich string and then elm-format will automatically make more verbose and more clear, the encoding of the special characters in that string. I really like that shortcut. [00:45:42] Aaron VonderHaar: Yeah. And that that was an intentional one. specifically there's a list of, kind of special things that it intentionally will escape. For instance, like confusing white space characters, for instance, that aren't normal spaces. And the idea there was to make sure that if you're, especially if you're copying and pasting code, or even typing it accidentally in some way, that you will clearly notice if you put something there that you didn't intend. [00:46:13] Kevin Yank: Yeah. Like a non-breaking space or something like that. [00:46:16] Aaron VonderHaar: Yep. [00:46:17] Kevin Yank: I looked up those five principles in the README and here's what they are: “The benefits of elm-format: It makes code easier to write because you never have to worry about minor formatting concerns while powering out new code. It makes code easier to read because there are no longer distracting minor stylistic differences between different code bases. [00:46:37] “It makes code easier to maintain because you can no longer have diffs related only to formatting; every diff necessarily involves a material change. [00:46:46] “It saves your team time debating how to format things because there's a standard tool that formats everything the same way, and it saves you time because you don't have to nitpick over formatting details of your code.” [00:46:57] Something that was pointed out to me recently by a beginner in Elm at Culture Amp, is that it is still in beta. [00:47:05] And, the engineer who, I'm thinking of who started playing with Elm for the first time, like two weeks ago as we record this, he said to me, “I saw elm-format, but I saw it was still in beta. So I figured the right thing to do was to stay away for now.” And as a tool that, you know, if you ask me, this should be in your essential first install, especially as a new learner of the language. [00:47:28] Aaron VonderHaar: The main issue here is just time available to support an open source project in my free time. I think, elm-format is pretty much there. But you can actually— I think I have the issues organized pretty well, so you can actually go and look and see what are in the 0.9 and then the 1.0 release, the 1.0 would be the non-beta release. [00:47:56] So there is a finite number of things, but there are some things, like one of the weird things that bothers me is. There isn't complete and comprehensive, like new-line handling for types. So for all the expressions, everything works pretty well. But if you have a type definition, you don't have the proper control over [00:48:19] where you can put new lines. So that's one specific thing that I think I want to get fixed before calling it 1.0. But I mean, you're right, that it, it is something that is certainly stable enough for people to use. But I guess in my mind, and kind of given that I have the luxury to not need to hit milestones to make a living off of this, [00:48:44] I guess there's certain level of quality that I feel like it demands for me to feel proud of calling it 1.0. Anyone can take a look in the GitHub issues and see exactly what things are blocking that from happening. [00:49:00] Kevin Yank: The thing you mentioned about gofmt, earlier was that it was blessed by the core team of the language. And I feel like that is somewhat unofficially the case with elm-format, [00:49:13] yet, you know, someone installing Elm for the first time might overlook elm-format because it's not in the default installation, for example. [00:49:22] Do you have any thoughts about how you would, like the way elm-format is, packaged, distributed and discovered to evolve in the future? [00:49:31] Aaron VonderHaar: So the plan has always been from the beginning to eventually have it included. I'd say the main thing that's changed is that it hasn't really seemed that urgent. My impression is that Evan's been happy with the way I've been maintaining it and hasn't felt a need to take it over. For instance, like I've still had time, even after all these years to continue to dedicate to it. [00:49:54] So it hasn't been in need of a new owner. I mean, there are, there are other tools, in particular elm-test is something that ideally should be part of the core tools as well, and yeah, I don't know. It's just kind of that there's a lot of balls in the air. It's kind of a low priority to try to do that. [00:50:13] It seems like Evan's philosophy for Elm, if not for open source in general, is that he really sees value in having tools and packages kind of attributed to the author that develop them and using that knowledge and reputation of the person who's responsible for a tool to inform you about the quality of that tool, for instance. [00:50:39] So I think in kind of his philosophy, it makes sense to continue to have, for instance, elm-format maintained under my name. Rather than trying to merge a lot of things into Elm, which allows him to kind of be the main person for the main Elm stuff. And the other tools that are mainly owned by other people, those are still attributed to them. [00:51:05] I mean, at some point in Elm's growth, I expect that's gonna need to change, but it seems to be working fine for now. [00:51:12] Kevin Yank: If people are wondering what's exciting in the world of Elm format, what is there to look forward to? It sounds like that refactoring tool you were mentioning earlier is a big focus for yourself right now. [00:51:23] Aaron VonderHaar: Yeah. Like I mentioned, hopefully in the next month or so, I should be posting about that on, I guess the Elm Discourse and [00:51:32] the Elm Slack. Look for that. If people are interested in trying that out, even contact me on Slack sooner if you want, but I should be posting some stuff and looking for feedback on that pretty soon. [00:51:45] Kevin Yank: I don't know if this would be spoiling it, but like what is the user interface for that likely to be in its initial version? Is it a command line tool? Cause it sounds like the sort of thing that needs to be, integrated pretty deeply into an editor. Is it, is it editor tooling authors that you want to hear from at this point? [00:52:03] Aaron VonderHaar: Yeah. So this is actually more for bulk refactoring and, it is a command line tool. Basically the definition of your upgrade, essentially, it looks like code, that it just inlines and does simplifications on. So, for instance, at NoRedInk, we have, our style guide where we have, for instance, shareable components for buttons, for modals, for drop-downs, things like that. So we've been using the, alpha version of this refactoring tool to, for instance, if we introduce a new API for doing modals, we can write an upgrade definition for that and then run it across a whole code base, and then point everything to use the new module, even if it has a very different API. [00:52:50] Kevin Yank: I've heard that kind of tool described as like a codemod tool. The other thing I'm looking forward to is our second episode with you, Aaron. We were chatting before recording this and, you've got a whole other, thing you're working on there at NoRedInk around test driven development. And, did you want to, give a, you know, a sneak 30-second teaser about what we’re going to talk about there? [00:53:13] Aaron VonderHaar: Yeah. So folks might have seen this on, the, discourse and the Elm Slack. but elm-program-test is the name of the library. and I think we'll be talking about how we use that at NoRedInk and some of, my philosophy about. How to test Elm, what you should test, how to write comprehensive tests in a large project where you have front end and back end or even different back ends that are all interacting and being used by your own project. [00:53:42] And in particular how you can test, your commands in your update functions and the long-running interactions that may be making HTTP requests, interacting with the user, and so forth. [00:53:54] Kevin Yank: Ooh. that's heady stuff. I can't wait to, I can't wait to hear your take on it. Um, already looking forward to that recording. Thank you very much, Aaron, for stopping by giving us the inside look at elm-format and its development. I want to personally thank you for all of the time and effort you've put into that tool. [00:54:13] as I've said, it provided the first example of delight I had in learning Elm and it continues to do that every day for me to this day. [00:54:23] Aaron VonderHaar: Well. That's great to hear. Thanks for the kind words. [00:54:26] Kevin Yank: You're welcome, Aaron. Talk to you again soon. Yeah, and thank you, listener for joining us in Elm Town. We'll be coming to your ears with another episode before too long.

Ian Mackenzie shares the history of elm-geometry, elm-units and elm-3d-scene, and how he hopes to one day see them used to design a world-championship-winning robot – or a skyscraper. Thank you to our sponsor, Culture Amp. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 18 Mar 2020 Guest Ian Mackenzie (@ianemackenzie) Show Notes 00:00:00 Intro and sponsors 00:01:05 elm-geometry and elm-3d-scene elm-geometry (Elm Packages) elm-3d-scene (GitHub) "A 3D rendering engine for Elm" at elm-conf 2019 (YouTube) 00:02:02 How Ian got into Elm FIRST Robotics Competition “Now you're thinking in functions" at elm-europe 2018 (YouTube) Dart dart-sass V8 JavaScript Engine Flutter Qt 2008 World Championship – 1114 Simbotics Arup (website) MassMotion crowd simulation software elm-units (Elm Packages) 00:19:18 API design lessons from elm-geometry “Now you're thinking in functions" at elm-europe 2018 (YouTube) Matthieu Pizenberg (website) Martin Stewart (Tretton37) Joël Quenneville (Twitter) 00:30:37 elm-3d-scene elm-3d-scene (GitHub) "A 3D rendering engine for Elm" at elm-conf 2019 (YouTube) 00:37:46 Ambient lighting 00:44:50 Transparency and graphics programming hacks 00:46:50 Real-world vs screen units 00:48:23 Creating a generic framework 00:49:55 What's left before 1.0 Contributing to elm-3d-scene (GitHub) 00:53:37 Using elm-3d-scene today 00:55:02 Thankyous & Outro

Martin Stewart tells the behind-the-scenes story of how his game Circuit Breaker (built entirely in Elm, of course) came to be, starting with an ambitious plan to surprise his sister on her birthday. Thank you to our sponsor, Culture Amp. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 5 Jan 2020 Guest Martin Stewart (https://tretton37.com/meet/martin-stewart) Show Notes 00:00:00 Intro and sponsors 00:02:37 How Martin got into Elm 00:05:14 C#/WPF to Elm tretton37 00:07:05 Getting started in Elm 00:07:44 Remaking Lego Loco / server-side logic Lego Loco remake (GitHub) Lego Loco (Wikipedia) 00:11:29 “I was allergic to using case statements” 00:14:10 The origin of Circuit Breaker 00:17:57 A brief description of Circuit Breaker Circuit Breaker / source code 00:19:45 The original “hackman” prototype 00:21:30 The level editor 00:23:08 SVG to WebGL / presentation framework Elm Town 35 - Herzog Drei with Francesco Orsenigo sthlm.js #53 at tretton37 (meetup event) Stockholm Elm: Catchy Elm meetup title (meetup event) WebGL for Elm Martin's C# game engine / video demo 00:30:27 Hacking around Elm WebGL's limitations 00:38:45 Ready for his sister's birthday 00:40:16 Polishing Circuit Breaker full time between clients Circuit Breaker with Elm logo colour scheme 00:42:13 Deterministic except for floating point precision 00:44:07 The game's tutorial 00:47:49 Hidden features and Easter Eggs 00:49:50 Splitting a project into modules 00:53:22 Music in the game Meganeko Lain Volta 00:54:59 Playing music in Elm Crypt of the Necrodancer 00:58:30 Thankyous & Outro elm-ui elm-geometry

Stöffel talks about Jetpack, a simplified build tool that NoRedInk built to replace webpack, and how it started his journey to learn Haskell and eventually end up on the team behind NoRedInk's next-generation, Haskell-based server-side architecture. Thank you to our sponsors, Culture Amp and Joel Clermont. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 31 Aug 2019 Guests Christoph Hermann (Stöffel) (@schtoeffel) Show Notes 00:00:00 Intro and sponsors 00:01:41 How Stöffel got into Elm NoRedInk 00:07:07 Jetpack Jetpack Haskell webpack elm-live 00:12:22 From Elm to Haskell 00:14:59 Unsupported open source & making the repo public 00:16:30 The features of Jetpack 00:18:42 Smart development rebuilds with Shake 00:21:13 Other Haskell at NoRedInk 00:22:40 Full-stack type safety: Servant and Servant Elm Servant Servant Elm 00:25:31 elm-verify-examples elm-verify-examples 00:30:21 Running Elm as a command line tool 00:31:39 Auto-detecting tests in elm-test elm-test elmi-to-json 00:36:04 Assets in Jetpack 00:37:10 Engineering management separate from team leadership 00:40:09 One-on-one meetings 00:41:01 Success as a manager 00:42:58 Self-improvement as a manager 00:44:37 Puffins: Haskell-based architecture team 00:49:18 Outro

Richard Feldman joins us to mark the content completion of his book, Elm in Action. Thank you to our sponsors, Culture Amp and Joel Clermont. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 31 Aug 2019 Guests Richard Feldman (@rtfeldman) Show Notes 00:00:00 Introduction 00:01:05 Sponsors Culture Amp Joel Clermont 00:02:06 Richard intro NoRedInk elm-test elm-css elm-decode-pipeline 00:02:46 elm-test and Elm 0.19.1 00:06:33 Draft API: testing commands and tasks with elm-test 00:13:00 Recent Elm community events Oslo Elm Day Elm in the Spring Mario Rogic's keynote at Elm Europe 00:20:50 Elm in Action Elm In Action 40% off discount code: PODELMTOWN19 00:46:23 Outro

Jeremy Fairbank, the author of “Programming Elm” (Pragmatic Programmers), visits Elm Town to talk about his new book and how it teaches Elm, and about writing tech books in general. Thank you to our sponsors, Ellie, Culture Amp and Joel Clermont. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 4 May 2019 Guests Jeremy Fairbank (@elpapapollo) Show Notes 00:00:00 Introduction Programming Elm (website) Programming Elm (book) Test Double 00:00:51 Sponsors Ellie Culture Amp Joel Clermont 00:02:16 How Jeremy got started in Elm and writing books Solving the Boolean Identity Crisis (elm-conf 2017) Toward a Better Front End Architecture: Elm (Codemash 2017) 00:05:55 Elm as a hobby, writing things polished enough for yourself 00:07:46 Using Elm in Production and client stories Richard Feldman’s elm-spa-example 00:10:55 Experience as a technical book writer 00:13:16 Teaching patterns progressively 00:16:00 Writing hard chapters as Elm 0.19 comes out 00:17:53 Writing books assuming reader has no Internet 00:20:25 Test driven development and type systems 00:24:40 Writing is lonely / thank you Emily 00:27:27 Editor relationship and learning from them 00:29:18 Deciding where the end goal should be for a beginner's book 00:31:34 Sequel or cookbook for the Elm community elm-css elm-ui 00:34:01 Modelling transition states 00:35:13 Ports and subscriptions / web sockets chapter 00:39:46 Fuzzing and testing 00:42:25 Thank you, good bye and how to get the book

Members of the organising teams of all four major Elm conferences – elm-conf, Elm In The Spring, Elm Europe, and Oslo Elm Day – come together to discuss what goes into a successful Elm conference, the lessons they've learned along the way, and what we can expect at their next conference! Thank you to our sponsors, Ellie, Culture Amp and Joel Clermont. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 14 April 2019 Guests Blake Thomas (@dijjnn) Danielle Pham (@quelledanielle) Erik Wendel (@ewndl) Thibault Assus (@tibastral) Show Notes 00:00:00 Introduction elm-conf Elm In The Spring NoRedInk Elm Europe Not Only Meetings Oslo Elm Day Bekk 00:04:50 Sponsors Ellie Culture Amp Joel Clermont 00:06:53 Blake: How Elm In The Spring got started 00:08:43 Danielle: How they joined NoRedInk and elm-conf 00:10:08 Thibault: How teaching led to Elm Europe 00:11:53 Erik: Oslo Elm Day was “the ambitious approach” 00:14:30 Blind CFPs and curating for diversity Speak at elm-conf ElmBridge Global Diversity CFP Day 00:27:13 Doing it all again next year 00:29:35 The organising teams 00:31:04 Elm In The Spring coming up soon SPRINGTIME10 – 10% off Elm in the Spring 00:32:39 Blind CFPs and curating for diversity (part 2) 00:35:44 The organising teams (part 2) 00:39:06 Lessons: Respect the schedule 00:41:55 Lessons: Audience Q&A is usually a bad idea 00:45:37 Evan and Richard’s involvement 00:48:29 Elm Europe coming up in June ELMTOWN10 – 10% off elm-europe until the end of April 2019 00:49:01 Be there for the hallway track 00:52:35 Speaking at a conferece 00:54:38 Outro

Keith Lazuka and AJ Alt from Microsoft share their experience writing the amazing Elm language support package for the IntelliJ family of integrated development environments. Thank you to our sponsors, Ellie, Culture Amp and Joel Clermont. Special thanks to Xavier Ho (@Xavier_Ho) for editing and production of this episode! Recording date: 23 February 2019 Guests Keith Lazuka (@klazuka) AJ Alt (@ajalt on Medium) Show Notes 00:00:00 Introduction 00:01:37 Guest intro intellij-elm elm-hot 00:02:45 Project beginnings 00:07:10 About IntelliJ 00:08:04 Elm at Microsoft 00:09:20 Writing IntelliJ plugins with Kotlin 00:13:34 Parsing Elm to produce an Abstract Syntax Tree (AST) 00:16:22 So many features! 00:18:56 Type inference 00:19:55 How do you decide what features to build? 00:20:44 Tidying up dead code and imports since Elm 0.19 00:23:03 Supporting both Elm 0.18 and Elm 0.19 00:27:25 Parsing partial Elm programs with errors 00:30:38 Language servers and multi-editor support 00:32:57 Saving on scrolling 00:34:28 Effortless refactoring 00:36:19 Extensible records 00:37:43 Test-driven development & BNF grammar 00:38:39 How do you split work? 00:40:42 Outro