Raw Data By P3 Adaptive

Chris Rae has had quite the data journey!  In his current form, he's the co-creator of MyCoast, an app that helps government entities clean up hazards on coastlines. He's travelled many paths on his journey, from the banking space to working on Excel and other projects at Microsoft and all points in between and beyond!  His unique problem solving process is on full display when he tells the story of how he fixed an electrical problem with a piece of bread! Follow Chris on Twitter The Septic Companion by Chris Rae References in this Episode: The DoughPacitor (aka BreadPacitor) Video Chasing Company-Submarine and Drones Episode Transcript: Rob Collie (00:00:00): Hello friends. Today's guest is Chris Rae, co-founder of Blue Urchin LLC. Blue Urchin uses apps, computers, and the cloud to help government agencies keep the seashore free of hazards like derelict vessels. But that's not how I know Chris, I know him because he also worked on the Excel team at Microsoft. And like so many people that have been on the show with us, he never set out to have a career in spreadsheets, never intended it. In fact, never really even intended to have a career in computers, and yet here he is. He's also a devastatingly successful author having penned the Septic Companion. While we talk about that book and how it got its completely inscrutable name, he seems to think that title makes sense. But even after hearing his explanation, I'm still not a hundred percent convinced that he isn't pulling my leg. Rob Collie (00:00:51): He's an incredibly entertaining fellow and he's constantly cracking me up and entertaining me every time I open Facebook. Near the end, we even dive into one of those stories that he relayed on Facebook, where he uses a piece of bread to fix a piece of electronics. And it's a funny story, it's a silly story and it's true, but there's also something about the spirit of how he proceeded to diagnose that problem that I think really reflects that curiosity that most data professionals possess. That spirit of, we can get to the bottom of this is so perfectly exemplified in this humorous story. There's even a YouTube video to go along with it, if you're interested. A really funny guy, a really nice guy, we had a great chat, so let's get into it. Announcer (00:01:38): Ladies and gentleman, may I have your attention please? Announcer (00:01:42): This is the Raw Data by P3 Adaptive Podcast with your host, Rob Collie, and your cohost, Thomas LaRock. Find out what the experts at P3 Adaptive can do for your business, just go to p3adaptive.com. Raw Data by P3 Adaptive is data with the human element. Rob Collie (00:02:06): Welcome to the show. Chris Rae, how are you this fine morning. Chris Rae (00:02:10): I am very well. Thank you, Rob. Rob Collie (00:02:12): Well, I'm sorry that we pulled you away from the inflating of your boat. Chris Rae (00:02:19): I keep missing meetings these days, because the job I have does not actually have that many meetings in it, and so... When you work at Microsoft you're just constantly looking at your computer and thinking, "Oh, there's another meeting now, there's another meeting now." But I have meetings once every two or three days, and so I miss half of them. Rob Collie (00:02:33): Yeah, your head's not on that swivel, you're not as alert to it. Yeah, I agree. When you worked at Microsoft, it was, "Oh, am I missing a meeting right now?" But you also look at your calendar in the morning and get a picture of your day, and if you get out of that meeting habit, you really get out of it. We're Facebook friends, but that doesn't mean that I really know what's going on with you, what is your job these days? Chris Rae (00:02:53): You're right, we're Facebook friends with capital F's, it's a whole new category of friends. Rob Collie (00:02:58): It is. Have we ever even been in the same room together, maybe once? Chris Rae (00:03:01): You worked at Excel the same time I worked at Excel, right? Rob Collie (00:03:03): No, no, I was gone. I was either over on Bing or on Power Pivot at the time. Chris Rae (00:03:07): Oh, I wonder if I was in meetings with you too at Power Pivot, that might be true. Rob Collie (00:03:11): Yes. Yes. Chris Rae (00:03:13): I worked in investment banking for 10 years in the UK, and then I worked on Excel, and I can tell you how that switch happened, if you're interested, that was another 10 years or something. And then now I run the business with one other guy and we make... Basically, you know these apps that are kind of take a picture of a pothole and send it to the local council and they'll ignore it? We basically make that but for things on the coast. So like tidal flooding and abandoned boats and creosote covered logs and these kind of things, any kind of crap that you'd find that needs to be sent to some state agency, we make the app that you take a picture of it with and we sell it to state agencies. Rob Collie (00:03:50): Okay. And is that how you market it? This is the picture taking app where you get the pictures, you ignore them, is that the pitch? Chris Rae (00:03:55): I think this actually is the most marketing that we've done on this. It's all kind of word of mouth, [inaudible 00:04:01], once you get into one of these kind of niche-y markets, then people chat about your thing at conference and then you get a couple of emails from people. So yeah, we just marketed by word of mouth, really, and by not doing a terrible job. Rob Collie (00:04:10): So is the name of the app ignorecoastalshit.com? Chris Rae (00:04:13): It's mycoast.org, actually. Rob Collie (00:04:15): Oh, I was close. Chris Rae (00:04:16): You're not in a coastal state, right? Rob Collie (00:04:17): No, no, no, we are landlocked here in Indiana. Although, I mean, I suppose we have like a little tiny corner that's exposed to a Great Lake. Chris Rae (00:04:23): Yeah, I don't think they're going to pay for it. Rob Collie (00:04:24): Probably not. Chris Rae (00:04:25): Where do I live in Washington if I find an abandoned boat, I have to use my app. And actually, I've got to say, we get all these reports as they come in and there are some really nice abandoned boats. Honestly, some of them are just brand new boats. And so Wes, my business partner and I are like, "Oh man, how on earth did that ended up being abandoned?" And I think some of it is uncle Bob died and they sorted out the house and they sorted out the car, and then everyone just forgot that uncle Bob's boat was still floating around somewhere. Rob Collie (00:04:53): The marina owner eventually just unhooks it from the pier and says, "Hey, whatever happens, it's God's will." Chris Rae (00:05:00): Yeah, well and some of the abandoned ones are still hooked up and nobody's paid for them for a while. But basically the state needs to get rid of them because eventually they start leaking poisonous things into the sea and that's less good. Rob Collie (00:05:09): Maybe that's the pivot for y'all is you become a salvage operation. Chris Rae (00:05:14): It's not so much salvage as theft. Rob Collie (00:05:15): Okay, yeah. If you did it, here's the thing, you could do it with the state's blessing. These boats are just a problem for the state, right? Chris Rae (00:05:22): Actually the most common email we get from just random people on our website is, "Can I go and have this abandoned boat?" There's nothing we can do about it, we just send it on to the state agency. But I've seen the state agency responses and basically, are you willing to put up with six months of disastrous legal headaches in order to get a junky, nearly sinking boats? Or do you want to just get on with your life? It turns out you can't just go and collect a boat that you think nobody wants. Rob Collie (00:05:46): Right, okay. There's no squatter's rights or anything if you go live on the boat for a year? Chris Rae (00:05:50): I don't know about that part, you'd have to be pretty dedicated to live on like a sinking fishing boat. Rob Collie (00:05:58): Well, you got to prove that you want it. So this boat that you're inflating today is not of the abandoned variety. Chris Rae (00:06:04): Not yet, no. Do you remember when, I can't remember if this was the case when you worked at Microsoft, but Microsoft to give you some money a year that you had to spend on things that made you fitter. And the first year you'd buy some running shoes and a shirt or something, and then second year you kind of run out of things to spend this money on. And it ended up that both the wife and I were working at Microsoft and so we had a reasonable chunk of money to spend on things that made us fitter, and eventually persuaded the people who ran this program to let me and the wife use both of our allowances to buy one thing. And we bought this inflatable kayak that I already knew you can fit a motor to. So I bought the inflatable kayak on this fitness thing, and then we took the oars, and then I separately bought this motor and we had this power boat of sorts. Rob Collie (00:06:47): You don't ski behind it or anything? Chris Rae (00:06:50): No, it's not that much power, but we flit around in it with the kids. But the kids have got a bit big and they didn't really fit on it, so I bought a second boat. Rob Collie (00:06:56): A second boat, yeah. Chris Rae (00:06:58): When you're becoming successful in life, as I am, you can sometimes get a second boat. So I was just inflating that this morning to see if it stayed inflated. Rob Collie (00:07:08): I was going to say, because you're inflating the boat indoors, it can't be a 25 footer. Chris Rae (00:07:13): No, this is a big boat, well I've got quite a small house, but this is a 12 foot boat, it's in the back yard actually. I can probably see if it's staying inflated. Rob Collie (00:07:20): You didn't leave the compressor running, did you? Chris Rae (00:07:23): No, no, no, I stopped the compressor when you phoned me, and I thought, "Oh bullock, I've missed that thing." Rob Collie (00:07:29): Let's rewind, investment banking. Chris Rae (00:07:31): Oh, yeah. Rob Collie (00:07:32): You left all that glory behind for the life of the rest of us, how did you get into investment banking originally? Chris Rae (00:07:39): Glory, I did an English degree and at the time McDonald's wasn't hiring, and so my dad said, "Oh look, you like computers and stuff." And I'm like, "Oh yeah, but I didn't want to do the computer stuff for a living, I just kind of do that for a hobby." And he goes, "No, you can come to my office and you do that for a living now." My dad worked for this, he wasn't an accountant, or he isn't an accountant, but he worked for this accountancy company and they basically needed Word macros to do just weird stuff of theirs. And I just went there and wrote some VBA word macros and then they wanted more Word macros and more Word macros. We ended up with this kind of monstrous Word macro project that I was the grand master of. I did that for a couple of years in Edinburgh. And I did some system admin stuff. Chris Rae (00:08:19): And then I thought I'd go and get more money somewhere else. And I did a bunch of job interviews in London for a variety of things, initially for Word macros, mostly for lawyers. And then I started doing Excel macros interviews for mostly banks. And I basically got a job at UBS because the guy who was supposed to give me a technical interview was off sick that day, but they were in a hurry to hire somebody. So they turned out, we had a little nice chat about where I lived and what my hobbies were and things, and then they offered me a job. And then I was basically doing kind of IT support, but for spreadsheet stuff, but they were monster spreadsheets. Chris Rae (00:08:54): And I kind of switched from there and I ended up working on the trading floor for the department, and quant departments at investment banks are the people that do the complicated mathematics stuff. And actually at university, part of the reason I ended up doing an English degree was that I failed Math 101 four times in a row. And the last time, I basically started doing CS, but you had to pass Math 101 and I just failed it twice, so then I really actually tried to pass it and failed it two more times. And the guy who was my director of studies was this mathematics lecturer. And he's like, "Chris, I think you were future lies somewhere else." And so I did the English degree, and then of course I ended up working in a department full of mathematicians and me, but both used to keep referring to me as Dr. Rae because they assumed that I had a PhD in mathematics. And I'm like, "I'm just not going to disabuse you of this in case it results in me getting fired." Rob Collie (00:09:45): Dr. Rae, yeah. So that is of course fascinating, right? Chris Rae (00:09:49): It was cool. Yeah, it was a really interesting job if you were young. I mean, we were basically, I don't know how much you know about derivatives, but I'll give you the one sentence summary of derivatives, they're basically insurance policies. And so the reason derivative started is because farmers want to buy grain feed animals, but they have to buy it this year and they have to buy next year. And they don't care about the grain price, they're not interested in whether it's going up or down and so don't want to speculate on that. So they would buy insurance against the grain price, so they buy options to buy grain for next year for the same price you bought it for this year, or 5% more or something. And then you are guaranteed by the person you bought that option for that they will sell you that grain at that price if you wanted, but you pay a premium upfront to have that option. Chris Rae (00:10:30): And so derivatives are all just that kind of stuff, like there's people who turn it into finance, there's Chinese people who want to buy IBM shares, but they don't care about which way the dollar is going to go. So they'll say, "I want to buy these IBM shares in yen." And somebody else is going to deal with the dollar risk on this. So if the dollar goes up or down, somebody else is going to take the hedge on it, and I'll pay them a premium upfront because I don't know about where the dollar is going to go and I don't care about it. So we were doing all that kind of stuff. Chris Rae (00:10:54): And as you can imagine, a lot of that ends up at least being initially priced out in spreadsheets, because once you've traded a few of these or whatever, you have it in some backend system and it's updating every day and all that kind of stuff. But to work out what you're going to sell to someone or to chit-chat with someone about the kind of thing you're going to sell, that all happens in spreadsheets. And so I was making a few of those spreadsheets, lots of actual mathematicians in my department were writing C++ models around off the back of those. Then I wrote the wrapper thing that held all of their models and held this temple of library spreadsheets, and that kind of stuff. Rob Collie (00:11:22): What was that like in terms of having been someone who had discovered you weren't really that into math? And now here you are surrounded by math people in a department that is essentially doing nothing but math, I'm not getting the impression though, that you were terribly uncomfortable there. Chris Rae (00:11:40): There were occasions where it became apparent that I knew nothing at all about rounding up numbers, but usually it doesn't really matter. I don't even know why you needed math for a compsci degree, to be honest. Because, I mean, at the end of the day, you've got a computer that does most of this stuff. And as I say, there were people writing these derivative pricing models that were bonafide legit, solid mathematicians, that's the top end of mathematics jobs. But to be able to plug those in and say, "I want to buy this and sell that." You're just messing around in spreadsheets, you're not doing any complicated mathy stuff. And so I generally kind of got away with that. And also the more you are an expert at complicated math, the less you're an expert about talking to derivative traders. And so you often need a bit of a mixture of people who really know how the thing works and people who can somewhat explain it to a normal person. Rob Collie (00:12:28): This is something that I'm very, very familiar with is that, unlike you, probably, in high school I identified as a math person, I was captain of the calculus team and all of that. Chris Rae (00:12:38): Oh, wow. Thomas LaRock (00:12:39): There was no calculus team. Rob Collie (00:12:41): There was a calculus team, we went to the final four in Florida. Thomas LaRock (00:12:45): We just had a math club, we did more than just calculus. Rob Collie (00:12:48): We had that too. I was president of the New Alpha Beta chapter at my high school. And the first meeting I convened, I told her everyone, "Listen, we all know that this is just a matter of padding our resumes for college, right?" And everyone nods. And I say, "So meeting adjourn forever." The sponsor was happy. The teacher was happy. Everybody was happy. That allowed us to go do the serious thing like calculus competitions. My daughter was working on a calculus assignment the other night and I was completely incapable of helping her. I remember it was something that used to be really simple, it was a limit as X approaches, infinity type of thing. That used to be basic. It has no meaning whatsoever in normal everyday life. It is not relevant to anything that I do. There are many, many products that we use that I'm sure weren't possible without calculus. You don't need calculus to operate the product. Thomas LaRock (00:13:47): You guys are killing me. Rob Collie (00:13:48): I know, I know. Listen, Tom has an advanced degree in math, right? But even he doesn't use it and he knows it, it's okay. Thomas LaRock (00:13:54): Oh, I don't use it? I'm just counting the number of times you've insulted me in the past few minutes. Rob Collie (00:13:59): That's just arithmetic. In second grade, we were capable of counting insults, we don't need scientific notation for this. Thomas LaRock (00:14:05): Oh yeah, this is not an outlier in any way. Rob Collie (00:14:08): That's true. That's true. Yeah, you can probably anticipate how many times Rob will accidentally insult- Chris Rae (00:14:14): I mean, how many times does he normal insult you, probably just extrapolate. Rob Collie (00:14:16): Yeah, just don't even worry about it, it's just many. Yeah, it's just some number. Chris Rae (00:14:20): So basically it was a race to [inaudible 00:14:22]. Rob Collie (00:14:22): This has been a theme on this show sort of over and over again. There are so many people who are incredibly competent and successful professionals in data related fields who did not identify like that at all growing up, and I think that's wonderful as opposed to a problem. I had no idea that you were one of these people. I assumed, because I knew that you'd worked in sort of the quant shop at an investment bank, I assumed that you had come up through a math major. Chris Rae (00:14:50): Yeah, well so did my boss. Yeah. Rob Collie (00:14:51): Yeah, so there you go, I had something in common. Chris Rae (00:14:55): Yeah, I was going to head on to the segue for how I ended up working for Microsoft was, back when we shipped one version every three years, we'd get a year into specking the next version and then we'd trot around some PMs around a few customers to see what they thought of it. And then what always happened was you'd turn up at the customers and they're like, "Oh yeah, well, that's really fascinating, but can you fix the following really important bugs that we have for the current versus?" You kind of look them all up and it's like it won't fix. So the Excel team would send people to investment banking always, and the IT people would come onto the trading floor and say, "Look, who wants to talk to Microsoft about Excel?" And so we were the core of heavy use of Excel on the trading floor, so we volunteered to do that. Chris Rae (00:15:33): And I ended up being our guy who talked to the Excel team, and Dave Gaynor came out with a bunch of people and they chit-chatted to us about the new features they had in Excel, and we berating them about the list of 10 bugs we really want to fix. And afterwards we all went out for drinks and I thought, that'd be kind of fun, I could actually work at Excel and fix these stupid bugs. Chris Rae (00:15:54): And so I sent an email to Dave Gaynor just because he looked like the most important person there as far as I could see. So I sent him an email and I said, "Look, I can come and work for you if you'd like?" And he didn't reply. And I thought, it's a bit weird. You can't go poaching your biggest customer's employees. And so I didn't think anything of it, it was just kind of a random thing. And then about six months later, I got an email from him saying, "All right, I'm really sorry, this went in my junk mail. And yeah, you should come out and interview." So I came out here, I live in Seattle, I came out here and it was July, it was really nice sunshine, it was warm. There was that Seafair thing where they get boats out. Rob Collie (00:16:29): Oh yeah. Yeah, and the Blue Angels. Chris Rae (00:16:31): Yeah, right, right. So it was really nice. The wife came out too, we had a lovely week. We went hiking, we went camping and then I got back home and they offered me a job. It was less money than I was getting paid in banking. I mean, everything's less money than you get paid in banking. And so I kind of thought, "No, I don't know. I don't know if I can be bothered up rooting everybody to do this." And so I said no. And then the wife got a bit fed up with her job, she was actually also working for the same company, but doing other stuff. Our team was in charge of making risky bets with people's money, and her team was the credit risk control department. And so she and I were actually not really supposed to ever talk to each other. Chris Rae (00:17:04): But anyways, so she got slightly fed up of her job, I got a bit fed up of my job and I emailed Dave Gaynor and said, "Look, you know that job you offered me six months ago?" And he's like, "Oh yeah, okay." And I'm like, "Could I have it in eight months or something like that? And then I'll just take some time off for a while." I mean, I think at this point he's like, "Whatever. I mean, this guy's never going to turn up, so sure. Okay, why don't you do that?" And then so we did that. We both quit our jobs. We drove around Europe a bit, bought a travel trailer, I'd call it a caravan but I think you'd call it a travel trailer. Rob Collie (00:17:31): I've seen the movie Snatch where they call it a caravan. Chris Rae (00:17:34): Okay. Yeah, so that. Rob Collie (00:17:35): I learn everything about your country of origin from Guy Ritchie movies. Chris Rae (00:17:39): That's both surprising and alarming. Thomas LaRock (00:17:42): What about Outlander? Rob Collie (00:17:43): Nope, never saw that. Just Guy Ritchie movies. Thomas LaRock (00:17:45): Just Guy Ritchie movies? Rob Collie (00:17:46): Yeah. Thomas LaRock (00:17:46): Oh, all right. Rob Collie (00:17:47): Yeah, that's it. Chris Rae (00:17:49): Yeah, so then I moved to Seattle in October and then it started raining for six months, I want to say. Rob Collie (00:17:56): Oh, at least, yeah. Chris Rae (00:17:57): And my wife is Greek, and so that went really well. Rob Collie (00:18:00): She's used to that, that oppressive blanket of clouds that rolls in and stacks up against the mountains and never leaves. Chris Rae (00:18:06): In my defense, she lived in London for 10 years as well, so it wasn't like she wasn't going into this with her eyes open. But Seattle is a bit grayer than London, and I discovered the UK you'll have sunny, wintry days a fair bit, but not so true in Seattle, it's really overcast all the time. So at that point I discovered the Greek people were solar-powered and I'd made a terrible error. Rob Collie (00:18:24): It's an adjustment. I really struggled having grown up in Florida. I struggled the first couple of winters in Seattle. It was rough, I feel it. Chris Rae (00:18:31): Yeah. I'm from Scotland, so I didn't really mind it, actually. Rob Collie (00:18:34): It's not that the Scottish weather prepares you for it, it's just the Scottish temperament that prepares you for- Chris Rae (00:18:39): Well, I think it's all kind of tied in a bit, isn't it? We were camping last weekend, the last camping trip of the year. And it started raining and everyone else there is like, "Oh, great, it started raining." And I'm like, "Don't you think the countryside smells so nice and stuff when it rains?" And everyone's like, "No." Rob Collie (00:18:59): So that was fun, that transition. At what point in this timeline, did you take the lengthy sabbatical to become a best-selling author of the septic companion? Chris Rae (00:19:10): It was a bit odd going from the business of financial products to the business of software. Because when I worked at UBS it'd be like, "Oh, this seems kind of fine, let's ship it and we'll just pull it in a couple of hours of it's not working properly." Because we weren't making money off that software, we were making money off stuff we made for that software. So at Microsoft, you can't just ship this bug and hope to fix it and service [inaudible 00:19:30] one, that's not how it works. And so it's a bit weird, we were doing different things that pay the money. That was after a couple of years, once I was working at Microsoft, I was commuting on, do you remember the Connector, the bus that Microsoft provided? Rob Collie (00:19:45): Yep, yep. Chris Rae (00:19:45): Yeah, and so I live in Seattle, but Microsoft's in Redmond. And so I was taking this bus, and it's like 40 minutes or something each way, and you're just sitting on a bus. And I had, for a long time had this, for like 20 years at that point, I'd had this website called englishnumbertwoamerican.com, which I set up, God it'd be mid '90s or something like that. I mean, I wrote HTML in notepad, and it was just a list of British words with American translation. So there's a whole bunch of these things, so I think that might be in the first website. But I wrote this list, and I had sort of been thinking for a while, it got longer and longer and longer, and I'd sort of been thinking for a while I might do something bookish with it, and I'd just used those Connector bus rides to turn it into a book, basically. This is a self-published book, this is as successful a book as anyone could just pay some money and have a book that appears on Amazon. Chris Rae (00:20:37): But weirdly, actually, I did send it to two publishers before I went down the self-publishing. And they wrote back and they were like, "Oh, here's a bunch of things you could change, and we don't want it right now but if you completely redo it in this way, then fine." I mean, I knew perfectly well... You know what books are like, I'm going to make 15 bucks a year out of this and I'm not going to sit there rejigging it for this publisher guy to have it the way he wants it, I'm just going to do whatever I want. And so I did the self-publishing thing. Chris Rae (00:21:02): And then there's a little community of people who write British-American crossover things. And I was friends with this guy, he was an American living in the UK, he had written a blog that he turned into a book and got actually published published. But anyway, Mike Harling, I'll plug your book, Postcards from Across the Pond. He wrote this book, so he published it. I think I maybe wrote the back cover blurb for his book or something like that. Chris Rae (00:21:23): And he sends it to his publisher, and they actually sent me a message, this is after I'd self-published and was selling it linked from my website. They sent me a message saying, "Oh, we'd actually like to publish your book, to really publish it." And I was like, "Oh, that's amazing." And they said, "Pretty much we will publish it as it is. I mean, we quite like it." I said, "That's great. Yeah, I'm happy to switch." And they said, "How many are you selling a year?" And I was like, "I don't know 0 or 100 or something." And they're like, "Yeah, I don't know if we'd be able to sell that many." And so at that point I'm stuck with the position that either I can continue making $200 a year and having a self-published book, or I can make 100 and have a truly published by a publisher book. I just followed the money, too long in banking, I think. Rob Collie (00:22:10): Mm-hmm (affirmative). [inaudible 00:22:12]. Thomas LaRock (00:22:12): Would this book have the phrase Bob's your uncle and the translation, or is it just words? Chris Rae (00:22:18): It's mostly words. Thomas LaRock (00:22:19): Mostly words. So it didn't do phrases or idioms, just words? Chris Rae (00:22:23): Well, but I did... There's some super common phrases, I'm just looking it up in my own book here. Thomas LaRock (00:22:28): Okay. So like lift is elevator? Rob Collie (00:22:31): Can we have like an audio book moment here where, as the author of the Septic Companion, will you read us a passage? Thomas LaRock (00:22:37): Yes. Rob Collie (00:22:38): We're here for story time with Chris Rae. Thomas LaRock (00:22:40): The sleeping policemen. Chris Rae (00:22:41): The sleeping policemen is in here, this fine book available on Amazon, The Septic's Companion by Chris Rae. Rob Collie (00:22:54): We'll link it, and we'll of course make it an affiliate link so that we, P3, can make an extra $5 a year. Chris Rae (00:23:00): You guys could probably give up working at this point, I think. Rob Collie (00:23:02): Yeah. I mean, once we link it, yeah. That's what it's all about. Chris Rae (00:23:05): "The sleeping policemen, noun, speed bump. The name probably derives from a time when narcoleptic policemen were employed to slow down traffic." It says here. Some of this book is not factually correct, and I did very little research, because, well these days you could just look it up on Wikipedia and kind of transliterate it, I think that's what you do if you're writing a book like this. But back then I was just kind of making it up all myself really. I occasionally get emails from people saying some of this book is not factually correct. Rob Collie (00:23:35): So I've always wanted to know why is it called The Septic's Companion? What is that? Chris Rae (00:23:40): You know this Cockney rhyming slang business, where it's very much a London thing, but it's recognized across the UK, where you have a two word couplet and you say the first word, like butcher's hook is a phrase. And you say the first word, which is butchers, but what you're meaning is a word that rhymes with the second word, which is hook. So the Cockney rhyming slang for a look is butcher's hook, but people just say butchers. So you'd say, "Give me a butchers of that thing." And there's a lot of Cockney rhyming slang something that is actually in quite common use in the UK. Rob Collie (00:24:10): I have never been more confused than I am at this moment right now. I have no idea what was just explained. I'm more confused after the explanation than I was before. Chris Rae (00:24:21): How about this one? Do you know the phrase lose your bottle, is that a British thing? It means to give up on something or chicken out or something. Rob Collie (00:24:27): Well I do now, I didn't know it before. Chris Rae (00:24:29): Okay. Yeah, I wasn't sure if it was I translated, but that's the thing that most British people will recognize that phrase. That's Cockney rhyming slang because it was bottle and glass and glass rhymes with arse. And so losing your arse, posterior is like losing your ability to do something. Anyway so septic is British slang for American, because septic tank, yank, basically. Rob Collie (00:24:53): Oh my God damn. Chris Rae (00:24:54): And it also exists in New Zealand and Australia, Australians call American's seppos, which is like an abbreviation of septic. Rob Collie (00:25:02): Wow. Chris Rae (00:25:03): Just in case you thought it wasn't complicated enough. Rob Collie (00:25:04): We're going to have to just move past this, that this is just a game that is played over in that corner of the world, that makes no sense at all to anyone else. Chris Rae (00:25:12): I mean, America has its share of weird language things. I think the reason that you don't have more is just that this country is only kind of five minutes old, you've not been conquered by anybody. Rob Collie (00:25:21): We are five minutes old, and really hoping that we get to be six minutes old. Chris Rae (00:25:24): I assume I just get to talk as much as anyone in this thing, is that right? Rob Collie (00:25:27): Oh yeah, totally. Yeah. Chris Rae (00:25:27): Yeah, excellent. Okay, you know this crazy story that I've told a lot of people about, Kissinger in China, one of the actual good things that Nixon did was improve relationships with China. And when Nixon was visiting [Deng Xiaoping 00:25:38], Kissinger got chatting with Deng through a translator obviously, and they were just kind of making small talk and they ended up chatting about the French Revolution. And Kissinger asked Deng Xiaoping through the translator, what he thought the impact of the French revolution was on Chinese politics or on the politics of Asia, the interpreter translated it. And Deng Xiaoping sat for a little while and then said through the interpreter, "It's too soon to tell." It's this is classic, it's only been a few hundred years, difficult to tell whether that's really changed anything. Thomas LaRock (00:26:14): That just underscores the long game that China is willing to play. Chris Rae (00:26:19): Exactly, yeah, right. Rob Collie (00:26:21): Wow, it's too soon. Chris Rae (00:26:23): So the Septic's Companion is from the word septic, which is an offensive term about Americans. And somebody pointed out that you shouldn't really write offensive things about the people you're intending selling the book to. Thomas LaRock (00:26:32): It's not offensive, it's short for yank. Chris Rae (00:26:34): Yeah, but it is septic tank, which you probably... They could have picked something nicer for a rhyming slang for Americans. Rob Collie (00:26:41): I would say that in order for us to be insulted, we first of all have to understand that it was referring to us. Thomas LaRock (00:26:47): Yeah, we're too dumb to know we're being insulted, so that's the first thing. Rob Collie (00:26:50): No, no, no, no, it's not about being dumb in this case. I mean, this is an inscrutable reference, this is one case where the Americans don't have to apologize for not understanding it, this is just stupid. Chris Rae (00:27:03): Have you ever heard of James May, he was a guy on this TV program called Top Gear? Rob Collie (00:27:06): Yeah. Thomas LaRock (00:27:07): Yes. Chris Rae (00:27:07): So when I was posing this book, you have to have a blurb from, a kind of endorsement from somebody. And so I got a couple of endorsements from other people in this community of people who wrote things about Britain and America. But I sent a copy of that book to James May and said, "If you want to write an endorsement, that would be great." And he sent me one back and it said, "This would be an invaluable use to any Americans visiting the United Kingdom, assuming they could read." Thomas LaRock (00:27:32): Wow. He's not wrong. Rob Collie (00:27:35): Is that on the cover? Chris Rae (00:27:36): No. Well, I thought that might be pushing it a bit much. Rob Collie (00:27:44): Do you think more British people have purchased this book than Americans? Chris Rae (00:27:47): No, I think it's mostly Americans, although it is a bit of a mixture. It's probably like 50 books a year or something I sell now, but of those, I sell like two thirds of them at Christmas time. It's the kind of thing that you buy as a gift for people you don't really like, you haven't got any idea what to buy them and you just buy them like some crap off the internet. But interestingly, I'd like to bring this back to data, actually, Rob. Rob Collie (00:28:07): Please, let's do that. Chris Rae (00:28:08): The way it works is that I self-publish it, it goes through this kind of print-on-demand company. And so any bookseller can theoretically just asked for a copy of my book, and select my book from these people and I don't get involved at all. But the only real booksellers that show it are the online ones, because they just get all the books and as soon as somebody orders one, they actually go and get it. Chris Rae (00:28:27): Now Amazon obviously will have an algorithm for how many of these they need to give in stock, and generally this book will show up as available on Prime and it can be shipped to you. But at Christmas they run that every single year a third of the way through December. I mean, I think I might sell 70 books if Amazon didn't run out of stock of it every single Christmas. And how does your algorithm not work for this? The first year I was thinking, "Oh, BML or something, they're going to catch up." But no, no, every single year they just run out of stock of this book. And of course, because it's a crappy Christmas present you're buying for someone, you can't get it after Christmas... I'll have you know, this is like the 16,000th most popular book in travel/United Kingdom. Rob Collie (00:29:05): Well, I think they have a filter in their algorithm which is, does this SKU matter, right? Chris Rae (00:29:13): I'll have you know, this is the 16,000th most popular book in travel/United Kingdom. Rob Collie (00:29:19): Well, we should introduce you to Bill Jelen. Chris Rae (00:29:21): I've met him actually on one of these Excel trips, because he was an MVP, right? He was one of the vocal MVPs? Thomas LaRock (00:29:27): Yeah. Rob Collie (00:29:28): That's right, and he runs Holy Macro! Books. Chris Rae (00:29:31): That is quite a good name actually. Rob Collie (00:29:32): Yeah it is, isn't it? It's something else. It's the kind of name that you hear it and I can tell, "Why didn't I think of that?" Right? Chris Rae (00:29:37): Yeah, yeah, right. Rob Collie (00:29:39): But yeah, our book is 100% printed and published through his company, which then supplies the books to exactly the same sort of distributor that you're talking about, who then sells it to Amazon and Barnes & Noble and all of those people. But we don't print on demand, we maintain an inventory. I mean, this could really boost your profits, at 50 to 70 units a year, maybe you could get it up to 90 or 100 units a year by staying in stock during the holidays- Chris Rae (00:30:04): I'm all ears. Rob Collie (00:30:05): And you get a larger percentage of the revenue. Chris Rae (00:30:08): We should get my people to your people about this, I think. Rob Collie (00:30:10): Yeah, the publishing department, yeah. They're currently on a supplier trip to Brazil where they're evaluating the paper crop down there, but they'll be back. Chris Rae (00:30:19): I don't know, maybe your book isn't as eco as mine, but my book is available on Kindle. Rob Collie (00:30:25): Probably. Was yours written before mine? Chris Rae (00:30:27): Well yeah, probably. But I mean at the end of the day it's, if you're second you're last. Rob Collie (00:30:31): Second place is the first loser. A friend of mine at Microsoft used to work for a development manager who'd been an ex-football player at Florida State, and no lie, the front of this guy's house he had a sign over the front door that said second place is the first loser. And whenever the kids came into the house, they had to jump up and touch that sign. Chris Rae (00:30:49): Wow, my kids are quite young so I'm only starting this kids in sport thing. And I've discovering that what I have to keep my fingers crossed about is that neither of them are any good at any of these sport things. Rob Collie (00:30:59): Or at least not interested. Chris Rae (00:31:00): Because then you end up just driving around. Rob Collie (00:31:02): Let me straighten you out on this. What you're describing is the worst case scenario in my experience. Yes, interested, but not any good means that you're still being dragged around to all these events. Chris Rae (00:31:13): But you're not going to ones in other states or wherever. Rob Collie (00:31:16): Oh no, no, no, it turns out that the whole travel sports thing, it's been realized that that's a really, really good business and the more inclusive, the better, right? You can't just have the really, really talented kids because that's a small market, you need all of the kids, right? What you don't want is a kid that's interested in all this, and so you're driving them around and everything, but you're just watching and going, "This is never going anywhere." It deprives you of the enthusiasm that would sustain you while at the same time dragging you out into the same crappy weather over and over again that the future semi-pro kids are also experiencing. Chris Rae (00:31:48): But then if I got them disinterested and useless, then I have to drag them out the door every Wednesday to this stupid soccer practice that I didn't even want to go to anyway. Rob Collie (00:31:57): Right, it's a very delicate balance. Chris Rae (00:31:58): You got to have them good and interested in it. Rob Collie (00:32:01): You might want to get one of those signs, second place is the first loser. Chris Rae (00:32:03): I think I like the opposite sign and then try and put them off this whole thing. Rob Collie (00:32:08): So despite all of the riches brought to you by The Septic's Companion, you weren't one of those people that let success get to your head and sort of get lazy? Chris Rae (00:32:18): No, I kept working at Microsoft just for the love of the craft. Rob Collie (00:32:21): Mm-hmm (affirmative). Chris Rae (00:32:22): Yeah. Rob Collie (00:32:22): But now we do this other thing, the app and the boat theft. Chris Rae (00:32:26): Yeah, and it's much better. I mean, honestly, I mean, as I said to a few people, I've only ever worked for companies that have more than a hundred thousand employees or fewer than three, and so I'm missing that kind of middle segment. Well, as you know yourself, you've done the same kind of path, but your business is more successful than mine, but you've got that thing where a lot of what you do in running a business is not really the thing that you thought you were going to do. I mean, ostensibly, I'm kind of doing computer programming in this business, but we're selling to the US government. Chris Rae (00:32:55): So I spend half my time filling in forms that say, "I don't do any business with Iran." And then going and getting them notarized. And then realizing it was me that was supposed to invoice these people. And I've no idea how much it is for, and I think we only said it in a call and didn't write it down anywhere. Yeah, that's all the stuff about running your own business. Like if you run out of pens, then you have to go and get some pens, it's not like you go to the cupboard at the end of the hall. Rob Collie (00:33:18): That continues. As you scale your boat confiscation empire, you will discover that that only gets worse. Chris Rae (00:33:25): There'd be less paperwork in a criminal enterprise, I think. Rob Collie (00:33:27): Probably. Yeah, that's true. But I mean what you're looking for is one of those state sponsored criminal enterprises. Those are very profitable. Chris Rae (00:33:35): Oh, so I go and get these boots and then split it with like the State of South Carolina. Rob Collie (00:33:39): Yeah, that's where the money is. So I encourage you to maybe go watch some Guy Ritchie films or something, get up to speed on what the underworld's really like. You've already taken the first steps. Chris Rae (00:33:52): Yeah, no, there definitely would be something ramping up if I was to go down this path. Rob Collie (00:33:55): Yeah, and you've already got some of those relationships with the local officials. Chris Rae (00:33:59): I already know what my current ledger price is going to be, and that's the hard part. Rob Collie (00:34:01): Oh, well see, and you say you're not a crook. Chris Rae (00:34:04): Yeah, that's true. Yeah, I've actually been for drinks with them. Thomas LaRock (00:34:07): No, I just think the rules have changed over the years and there's more scrutiny on certain things. Chris Rae (00:34:12): I know. Man, I've got to say, at least all of the government people I've worked with have been squeaky clean about letting you pay for even drinks or food or anything. As I say, the field that I'm in, which is basically sort of coastal hazards, everyone is super careful about not inadvertently taking gifts from people, even when we've tried to give them drinks or food or whatever, they will not do it. So maybe these are the wrong local government people I'm in with, to be honest. Rob Collie (00:34:35): Or maybe they're just incredibly savvy evaluators of [inaudible 00:34:39], right? Chris Rae (00:34:39): Yeah, maybe. Rob Collie (00:34:40): Like, this drink could cost me my job, and all I would have gotten is a drink. If you really want to get serious about this you're going to have to start offering to buy them a seaside home. Thomas LaRock (00:34:50): No a boat, you should offer to buy them a boat. Chris Rae (00:34:53): Oh, interesting. Yeah, get them a boat. That's true. I mean the other thing, I suppose they might be thinking, "If I let Chris Rae buy me a drink then he's going to think he can take another six months with this 10 minute project he's doing for us, and I'd rather he didn't feel like he has that satisfaction." Rob Collie (00:35:07): So without giving away any trade secrets, what's the architecture of your solution? Chris Rae (00:35:11): It's all on WordPress. The pictures are stored on, there's a DigitalOcean data storage thing, it's basically S3, and we ended up using that after a complicated and painstaking evaluation which revealed to us that we already have logins for DigitalOcean, and so we went with that solution. Thomas LaRock (00:35:26): That was your analyses, we already have a login, we couldn't possibly create new logins with a different service. Chris Rae (00:35:33): Well, honestly, I think the amount of our time we would expect tracing new logins would have negated the $5 a month it costs us to store all this with DigitalOcean. Because data-wise, there's not actually a huge amount of data on this, we have maybe 20 or 30,000 reports of anything in the system. And that's three data fields and one JPEG, so there's not really that much data on this. Rob Collie (00:35:54): Let me help you, put your marketing hat on. Chris Rae (00:35:56): It's big data actually, sorry. It's a big data project and it's in the cloud. Rob Collie (00:36:00): Another phrase that you need to get really effortless at dropping, which is, best of breed. Oh, you've got a WordPress S3 DigitalOcean, you went and selected a best of breed infrastructure. Chris Rae (00:36:12): No, these are premier products. Thomas LaRock (00:36:14): A single vendor solution. Chris Rae (00:36:16): Yeah. Rob Collie (00:36:17): But I think you also need to highlight how you work with unstructured data. Chris Rae (00:36:21): It's pretty structured actually, to be honest. I mean, theoretically in the comment field, someone could paste [inaudible 00:36:28]. Yeah, and then not that we'd do anything with it, but that would be unstructured data living within our... Yeah, okay, it's unstructured data, it's in the cloud. Thomas LaRock (00:36:34): No, the image, the image is unstructured too. Chris Rae (00:36:36): Oh, well, it does adhere to kind of JPEG, but the content of the image there might be a log. Rob Collie (00:36:43): This is actually one of the ongoing jokes on our podcast that you haven't listened to. Chris Rae (00:36:47): No, I listen to it loads of times, I know all the ongoing regular jokes. Rob Collie (00:36:50): Sure you do, sure you do. It's the systems that our business acquires and decides to invest in over time, the various line of business systems. Those definitely aren't the random hodgepodge of systems that you would accidentally just sort of like stumble backwards into adopting. No, those are your best of breed infrastructure. Chris Rae (00:37:06): This is just my chance to ramble on about anything. When I joined UBS, I was given two computers. So I was on the trading floors, you get your six screens. And I had five screens with this Windows PC, and I have this other screen, it was kind of a big screen. And I got a NeXTstep machine, and they said, "Yeah, so here's your Windows machine, you'll use that for pretty much everything. And here's your NeXTstep, you're going to use that for our risk management platform and a couple of other apps that we haven't managed to port off NeXTstep yet." And it turned out that UBS had double-down on NeXTstep as it's the next big thing, several years beforehand. And for anyone who doesn't know, NeXTstep was the Unix operating system that Steve Jobs did kind of in between Apple the first time and starting Apple the second time. But it wasn't hardware, it was just DOS. Chris Rae (00:37:50): And so UBS double-downed on NeXTstep, it was completely out of support. Steve jobs was off doing something else at this point, NeXTstep was kind of sunsetted and UBS were desperately trying to move everything they'd written on NeXTstep off NeXTstep, but you still had it for some apps. And what happened was everyone had been given an NeXTstep machine, and then you get a nice big monitor for the NeXTstep machine. But none of the drivers on the NeXTstep machine had been updated whatsoever, and the mouse cursor moved at the same speed pixels per second as it had done on tiny little monitors. Chris Rae (00:38:18): So on my first day at work, I sit down and I've got a switcher between these machines, and I could switch Windows between the NeXTstep machine. And somebody brought me four mouse mats and I'm like, "Oh, no, it's okay, I've already got a mouse mat." And he goes, "No, no, you're going to tape them together." And I'm like, "I don't know if that's going to happen. And so I crank up the NeXTstep machine, and I'm just trying to get across the screen and I realized, "Oh yeah, I'm going to need these four taped together mouse mats." But the funny thing was that NeXTstep, the development language was Objective-C, which was used on NeXTstep. Rob Collie (00:38:53): That was the platform. Chris Rae (00:38:53): That was the only platform that Objective-C team was used on that. So if you were an Objective-C developer, you had got to Objective-C because you were a NeXTstep guy. And so we ended up, at UBS, we had to hire a bunch of these Objective-C developers because there weren't any. And so it's that usual thing where the thing you're working on dies and you're like, "Well, that's the end of that career." And to translate, no, no, they actually the start at the well-paid part of that career. But we had all these NeXTstep developers, and they were thinking, "The end of UBS, that's the end of Objective-C. Once UBS finally migrates off NeXTstep, that's Objective-C." And then of course that happened, so there was three or four years where nobody was using Objective-C, and now it's the primary development language for the iPhone, so these guys have got another entire career that's kicked off. Rob Collie (00:39:33): Oh, because Steve brought the Objective-C sort of like mentality with him to Apple. Well, for a while there, I was thinking that we would make the joke that if it was only able to be used on NeXTstep machines, that it was probably a better termed subjective seat. But if it's now used on Apple, on iOS development then, well damn. Chris Rae (00:39:53): Yeah, and it's a weird choice, it's an old language. If somebody had said, "What language is Steve Jobs going to do for the iPhone?" That would not have been in the top 50 ideas that anyone might have had. But it's what ended up happening. Rob Collie (00:40:04): It's hard to imagine Steve having an opinion about a programming language. Chris Rae (00:40:08): I don't know if it was him, but at least some part of what he'd been at NeXTstep, I assume he took some people between these companies. Rob Collie (00:40:14): Yeah, it's probably the people that went with him, yeah. That's probably true. Thomas LaRock (00:40:18): You're joking. Right? Because Steve had an opinion about everything. Rob Collie (00:40:21): I think an opinion about a programming language would have been in some way beneath him. Thomas LaRock (00:40:24): No, it seems to be right in line with his control of everything. Remember how he was like, "We're done with Flash." He wasn't really removed from a lot of these decisions. Rob Collie (00:40:34): So say you're done with Flash, and to say that we're going to use Objective-C- Chris Rae (00:40:38): To Tom's point, it is a language you can control, right? Because I mean, if it's only your platform it gets used on, nobody's going to update it without you knowing. Thomas LaRock (00:40:47): Exactly, that's really what it is. Yeah, just like controlling the app store- Rob Collie (00:40:51): Proprietary. Thomas LaRock (00:40:52): Proprietary, absolutely. Rob Collie (00:40:53): Okay, I could see him getting behind proprietary for sure, no two ways about it. I get that, right? But it's not like he had an opinion about the way that Objective-C did their data typing. Thomas LaRock (00:41:02): No. Rob Collie (00:41:03): That would be a little off-brand for him. Chris Rae (00:41:04): No, if he thought about that he probably wouldn't have used it. Thomas LaRock (00:41:10): Probably not, right? Rob Collie (00:41:11): So any other sort of funny data or Excel related stories? Chris Rae (00:41:16): Oh, I lost the company a fair amount of money on it's spreadsheet, if you'd be interested in that? Do you have Excel people listening to this, because this is a bit Excel-ish. Rob Collie (00:41:24): We have plenty of Excel people, yeah. Chris Rae (00:41:26): Yeah, so basically dividends are an important part of pricing anything like stocks or stock options or whatever, because whenever a dividend comes out, the person who owns the stock is paid a dividend and then the stock price goes down by the amount that the dividend was because you could have bought it the day before and got the dividend, so that all gets kind of factored into the stock price. And when you're doing derivatives, dividends make a big difference because they'll jump you over some threshold that you now have to actually fulfill this contract. I'd say dividends are really important in stock pricing, and what you sometimes sell are derivatives based on indexes, so you would sell S&P 500. So people will buy an index, which is basically like buying every single stock in an index. So when you say we want our portfolio to represent the S&P 500, what you have to really do is buy every stock in the S&P 500. And so you want to buy derivatives based on that, then you need to work out what they're worth with all of the dividends and all these stocks. Chris Rae (00:42:17): And so in a normal stock you'll get a dividend every year or maybe more often than that, but that's probably it. And so we had an array formula in the spreadsheet, which was, we have a function built in out of this database we had of dividends, and we had an array function that pulled in far too many of them, like 20 rows of future dividends, when in reality you probably only got three or four. But the problem with array formulas is you can't have them kind of extend indefinitely. So you have to make it too long and then refer to too big an array. Chris Rae (00:42:45): And what happens when you trade an index is all of the dividends turn into the appropriate index proportion of that dividend. So if Apple is 10% of your index, then your index dividend is Apple dividend times 10%, and then all of the other stocks at 10%. Well, there's a lot of dividends in that list. And so if you do, say for example, a 20 row array formula, you may only get a couple of weeks of dividends on the thing you were pricing that's five years out. Rob Collie (00:43:13): Okay, yeah. Chris Rae (00:43:14): And then your sales person might trade that thing based on that price, which was surprisingly competitive. And then once you've traded it, you might realize, "Wow, we sold this awful cheap." But then some people might start hunting around as to why we sold it off cheap. And then they might turn up at my desk and say, "I'd like to talk to you about this array formula that we just lost $200,000 from." A sorry, a sad afternoon, because it was pretty rapidly obvious... We're basically typing an index saying hit refresh dividends, you see it fill the whole thing and you're like... Oddly enough, thank God I'd written at the top of the spreadsheet, "This is not for pricing indexes." And so it wasn't my fault. Chris Rae (00:43:59): And honestly, one of the things I wanted to do in Excel was to have this concept of a structure inside a cell. So what I would really like to have is have cell one be the dividend array. And then I can say, "Equals R XLL DLL wrapper function [this cell]." Which is in fact like a whole structured array inside of that. And inside that cell I'd have our array formula return to all the dividends, but you never have this problem of having to extend arrays. And it's a constant problem you get in Excel with all sorts of stuff. It's just when you have pieces of data like length and, it's just a shit show. I mean, I really like Excel, I've worked in Excel, but it's derived from a product that was made in the '70s and it doesn't work well in determinate length of data. Thomas LaRock (00:44:41): So did you ever just consider to use the correct tool for what you were trying to get done? Chris Rae (00:44:46): What was the correct tool for what we were trying to do? Thomas LaRock (00:44:49): Something that would work with arrays. I don't know, Python. Chris Rae (00:44:52): Well, but these are guys who came out of university with MBAs and they're going to be derivative traders, they don't give a flying fuck about arraying. Rob Collie (00:44:58): That's fine. That was one of those Cockney rhymes, right? That wasn't actually the F word, it was meant to rhyme with buck, like pass the buck. Chris Rae (00:45:04): It's from fuckwit and then bit is the word, so they don't care a bit, is what I'm saying, yeah. So everyone says this in the UK, it will be fine on the BBC, I think. I mean Tom, the truth is, these people are not interested in learning a programming language. Excel, has this glory of kind of casual obscenity to it that it looks like anyone can use it. Everyone can get to grips within 10 seconds with what Excel basically does, and you can get someone going, and then as things get more and more complicated, you get to a point that man, you really should have written this in something else. But you started off in Excel, because it's so easy to start off in Excel. And so it's just so easy to say, "I'm going to do one of these and one of these and I'm going to have those together. Now I've got a derivative product." Chris Rae (00:45:45): And there was always a pressure in the bank, because we have lots of these kinds of errors. Not only things that we've done wrong in spreadsheets, but also just we didn't lock the spreadsheets down enough so people would just change one formula in a whole segment, but it turns out all the other ones were the same. And we would constantly have whole arrays of data where one of the formulas was different because some of the thought that was going to change them all. And we actually wrote a bunch of tools to check... You highlight an array and it's like, "Tell me what I've screwed up in this array." Chris Rae (00:46:10): And so everyone wanted to use Excel, and so they did use Excel. And then the truth is that we ended up, you can price things pretty quickly because people are like, "Oh, this thing you sold us last week..." Which is a thing we had a whole risk management system for.And this other thing, we wanted to get a rough idea of, what if we combine those two things, what would that cost? And so you obviously do that in spreadsheet, it's the right place to do it. Chris Rae (00:46:28): But then they say, "Oh, actually we might trade that." And then at that point we tried to never actually risk manage our spreadsheets. So when you'd sold a product, and somebody has to find out every day how much of IMB stock do you buy or sell in order to fulfill this derivative in two years, that would all happen in this backend system. But there were a number of cases where people had done things in Excel that weren't... They'd used add-ins for getting data from somewhere that nobody had written into this backend system yet. And so we ended up having some things that got risk managed off Excel, and I hope nobody from UBS is going to come and have me assassinated now. Chris Rae (00:47:04): But we also had some things that got risk managed of Mathematica. So we would take Excel models and turn them into Mathematica, which is kind of a bit more of that, like what I was talking about, that kind of structured thing, you're not going to fall off the end of an array. And one of the other guys on our team was building quite a lot models in Mathematica for things that couldn't quite be built into the risk management thing. I mean, you're right, that that is a very chunky way of doing these kinds of things, but I do think it's kind of the right way. And I wish there was a product that was a bit like Excel but a bit more well suited to this kind of thing, I think. Rob Collie (00:47:32): Excel is working on becoming that product, as you're aware. Chris Rae (00:47:35): They're trying, it's just, as we all know there's a lot of history there, and you've developed this new feature and it turns out it doesn't work in Excel for macro language. People always used to, when I was going around customers, people were always worried about when we were going to get rid of VBA, even then it was 10 years old, and I was 20 years old. And it was never a good programming language for anything, it was just a thing that got kind of shoehorned into office before our time. And I always said to people, "You're right, and we would love to get rid of VBA, But what you need to look at is Excel for macro language." And so we've taken out the UI to create Excel for macro language, we did that ages ago. But you could still run Excel for macros in Excel when I left in 2014 or something like that. And I said, "If you're worried about VBA being got rid of, you just keep an eye on Excel for macro language, because when that goes, you've got another 20 years." Rob Collie (00:48:21): I don't see VBA ever going anywhere. Let's circle back to Tom's point for a moment. Tom, you know the answer to this, right? By the time you get into something like array formulas, you have reached a point where there is a far more elegant tool for that part of it. Array formulas are, I think the single most difficult and abstract thing in Excel. Chris Rae (00:48:43): And useful to be honest, but yes. Rob Collie (00:48:45): Super, super useful. The array formula crowd really, really, really needs to meet DAX's X functions, because, oh my God are those great. Chris Rae (00:48:54): Not that you're biased at all. Rob Collie (00:48:55): Well, I'm only biased of having used them both. Chris Rae (00:48:58): No, I thought you've worked on them. Rob Collie (00:49:00): No, I didn't have anything to do with the DAX language. I wish that I could say that I did, but I didn't. And honestly, even the X functions in DAX, those trace their lineage to the X functions or the similar iterator functions that were already in MDX that I never was able to learn. So the vast majority of array formula problems are just thousands of times more elegant, higher performance, everything if you implement them in X functions, there's almost like an entire business model of going and helping the array formula crowd at investment banks adopt the DAX X functions. Chris Rae (00:49:33): The DAX functions, you have to run them on PivotTable or structured data of some sort, right? Rob Collie (00:49:37): They run across tables of inputs. What really matters is the inputs, the flavor of output can be in a cube formula, for instance. If you wanted to tie it back into a regular Excel model, the cube formula can be the way to get them back into the grid on a cell by cell basis. Chris Rae (00:49:53): Can I put like a bunch of numbers in column A and then in column B use a DAX function to do something with them? Because I don't think I can. Rob Collie (00:50:00): No, you really can't. There's a refresh problem there. You need to be refreshing the data model from that range, and then it would work, right? The biggest problem is that is not integrated with the calc chain, the Excel calc chain and the DAX calc chain don't listen to each other for notifications. Chris Rae (00:50:18): I see, I see. Yeah. Rob Collie (00:50:19): That would be amazing if we ever got to that point, but a lot of these things could be done potentially end to end in the Power BI environment without ever having to run into... But now you're running into the same sort of thing, which is, "I've got to leave Excel." And there was a reason why I was in Excel in the first place. And we're talking about in some ways, not the last mile, but a critical sub component of a model, it's not the whole thing. The net benefits of using Excel for these problems, including array formulas and all of their gotchas and complexity, and also, frankly, inscrutability, you really can't watch an array formula evaluate. Chris Rae (00:50:53): That's true. I wished they'd done some more tools. I mean, I wrote tools at UBS to try and help people with some of this. Like, why is my spreadsheet going slow was a good one, and that kind of stuff, Rob Collie (00:51:01): Because you used array formulas, is the answer. Chris Rae (00:51:03): Well, yeah. And there's a bunch of ways that array formulas work that really are horribly ineffective, just because you need to keep things working from versions ages ago. All of us who've worked on this, there would be a new glorious fast way of doing this, but unfortunately it's going to break these spreadsheets that somebody saved from 15 years ago that are business critical and they're going to have to patch as soon as we sent it out, et cetera. Chris Rae (00:51:24): One of the reasons that people are all using Excel for this is not only because of Excel itself, but because everything has an add-in for Excel, like you've always got external data coming from somewhere and everyone makes an add-in for Excel. You get your FX rates from an add-in that the company matrix sell. You get your Reuters data. You get data from other indexes. Because each stock exchange will often have its own data retrieval system to get stock prices, and so you would need all of those plugins. And somebody needs to make those in whatever system you're doing your pricing in, and that, really the only thing that they all make for is Excel. Rob Collie (00:51:57): Chris, do you still have friends working in that environment, people that you know? Are you getting any signals? I imagine that there are now Python libraries and our libraries for integration with these same sorts of services. In the same way that Excel became a center of gravity for add-on writing, are your friends seeing R and Python and things like that kind of leaking in around the edges? I mean, Mathematica, if you were using Mathematica back then... Chris Rae (00:52:21): So the truth is that I don't talk to anyone about that kind of stuff, but my friends were slightly in an odd situation because we were the quant department rather than the trading department, so they were all kind of programmy type people anyway. So they were all doing what you think would be a good idea, but it's the traders that are tougher. And I bet you there's a whole new generation of people going into derivatives training that actually can write programming languages, at least enough to get their work done, much more than... This was 20 years ago when I started doing this, but I think you'd be surprised as to how much of it still happens in Excel. Chris Rae (00:52:49): Because often you'll be like, "Oh, okay. I got something vaguely working." You're a trader, you've got something vaguely working but you have to send it to the sales guy. You can't send the sales guy your GitHub. He needs to just be able to double click on it in his email and see a number and then change this other number and see what that number does. There's a lot of reasons that this still happens. Yeah. Rob Collie (00:53:10): So I just want to take a moment here and marvel at the contrast in something in particular. So again, we're coming back to this topic, did not enjoy Math 101. Chris Rae (00:53:19): Right. Rob Collie (00:53:19): That was Chris Rae's experience, did not enjoy math 101, okay. And yet, just for replay of bits and pieces of this conversation so far, Objective-C, you seem to have some opinions about that language. VBA, you had some opinions about that language as well. This idea of storing an array, an entire array in a cell and passing that around as like a first-class object, these are very technical opinions. Chris Rae (00:53:47): No, these all have more to do with the English degree that I got than they do with math. Rob Collie (00:53:50): Well, explain. Chris Rae (00:53:51): What's an array and a sale got to do with mathematics? Rob Collie (00:53:54): Okay, but what does it have to do with English? I mean, so here's the thing- Thomas LaRock (00:53:57): It might have words in it. Rob Collie (00:53:59): You would agree that the default belief, the preponderance of belief in the world is that these things have a lot more to do with math than they do with English? Chris Rae (00:54:10): Right. And I think, in my experience, I would say it's true that to be a computer programmer's computer programmer, like the guy who writes algorithms for stuff... I've got this friend who used to work for Pixar rendering stuff, and now he works for Google Maps on routing stuff. These are programmer's programmers who are inventing algorithms and things, and doing things with matrixes and log space. Those guys will benefit from having both sort of skills in mathematics, but also the kind of brain that is good at mathematics. Chris Rae (00:54:37): But I think these days, and really consistently all the way through, there's lots and lots of room for programmers who are not quite as sort of methodical and mathy focused as those people. I mean, you need those people to create the algorithm. I mean, we all know these days programming is mostly about finding things that other people have done and sticking them together. And frankly, by the time you are writing a sort of algorithm, you've screwed something up because you shouldn't be making anything where you have to do that. I think there's lots of different types of computer programs. And I mean, I've basically been a computer programmer for 25 years, but I've just not been that mathy type of computer programmer. And most computer programmers are not really, you're making databases and sticking things together and writing PHP and other non-glamorous languages. I know you want this to be true, Rob, but I just don't- Rob Collie (00:55:24): I don't, quite the opposite. But by playing devil's advocate, I think we tease it out more, right? It's like, there is a myth, and it's a very, very, very durable myth. And I think it traces back to high school and it traces back to university, as you would say, not to the real world, but it traces back to school. When I would stick my head into the computer lab in high school, the faces I would see sitting in there were largely the same faces that I saw in the physics class or in the calculus class. I didn't see the people who were in the humanities class in there, but then the real world happens and it's different. Chris Rae (00:56:03): Well sort of. Although I think it's also time. I mean, I think what computer programming meant twenty years ago is not the same as what it means now. I feel like the people who were writing computer programs, say 40 years ago, were optimizing memory users, they were minimizing the number of clock cycles they we're using for a particular algorithm. They were messing around with these real kind of low level type things. And computer programmers now, they tend to be much more people who know about the business that they're actually trying to work in and are just trying to use the computer to get something done for their business, because computing has got so much easier. The base level that you get when you start programming computers, it's just so much harder. Chris Rae (00:56:39): When you and I started programming computers, the base level you got was assembler and you were writing things that were compiled into assembler. Now the base level you get is libraries for recognizing glyphs on your camera screen. And that's the case of like import this, take a picture of your glyph, and now you've done that. I mean, I don't have any idea how that works, but people are just sticking things together more. And that's great because now this means that everybody can more and more use computers for their kind of stuff. But I do think that skill set is a bit different as well as the kind of perception of what was needed. Rob Collie (00:57:05): Yeah, I completely agree, and that's been a theme on this show that you've listened to every episode of. Chris Rae (00:57:09): Yeah, yeah, episode eight was really good. Rob Collie (00:57:12): That was a real zinger. So this notion of a hybrid person who knows the business but also has the technical skill to execute, is the most valuable thing going. That's the present, that's the future, that's kind of everything. Chris Rae (00:57:23): And it's great because it means that people who are subject matter experts will do computer stuff. I'm fed of people in Silicon Valley revolutionizing some industry they've never heard of just because they can make a website and there's other people who can't. I mean, you don't know anything about that, and now you've decided you're going to revolutionize it. I mean, it's complicated. And you see all these startups who start with, "Oh, we're going to take out the middleman and blah-di-blah." And then it turns out, oh yeah, that actually was more complicated than you thought it was and there was a reason that middleman existed, and then you go bankrupt, but by that point you've sold your company to some other company. But then hopefully we get to the point that subject matter experts are going to revolutionize their industry by using computers. Thomas LaRock (00:57:54): These types of functional analysts have existed forever, somebody who can do tech and somebody that can know the business, look at the character in Office Space, he has people skills. I mean, they've always been around. But here's the best example I have for what you just said about Silicon Valley. It was a couple of years ago, it was a ride sharing. And the person, this executive, I think, they were dreaming of the perfect ride sharing journey where one person would get on and then in the middle of that journey, they would stop and pick up another person, and then the first person might get off. And so this journey of never ending, this ride share journey of people getting on and off, and somebody just goes, "That's a bus." You've just invented the bus, and you think it's revolutionary. Chris Rae (00:58:43): But they've got an app. They've got an app. [crosstalk 00:58:47]. Thomas LaRock (00:58:44): They've got an app. Chris Rae (00:58:48): They've got $6 million in venture capital funding. They've got these amazing tables that go up and down. Rob Collie (00:58:55): They've never seen a bus. Thomas LaRock (00:58:56): And they're panting for the likes, I need four stars. Chris Rae (00:59:00): I mean, and also, we're all computer people, and we've all sat in bars with people who are trying to find an industry to revolutionize, busting off app ideas about things you don't know anything about. That's not how this is supposed to work. I'm not supposed to say I've bought a taxi, what could I do with it? You're supposed to have decided what you're going to do before you go off. Rob Collie (00:59:24): Plus the dirty reality is that even if you have a really good idea, a really good idea is worth next to nothing, it's just so worthless. Chris Rae (00:59:34): We've all had those. Rob Collie (00:59:35): How do you get from that idea to a reality that the market accepts it? And even if ultimately there is a real product in this space, how do you make sure, or service, how do you make sure that it's you and not some second mover that out executes you? You mentioned that you were potentially the first website in that burgeoning industry of English to English translation, and now you've been out hustled, out competed by the new breed- Chris Rae (00:59:58): But I'm still going. I get hundreds of hits. Rob Collie (01:00:01): Well, yeah hundreds. But they didn't go brick and mortar like you, you went into physical goods with the book. Chris Rae (01:00:06): Yeah, I was going to revolutionize that web industry. Rob Collie (01:00:09): Yeah. I mean, you had a really good idea, and you even pursued it and reduced it to practice and still got left behind. It takes more than that. Chris Rae (01:00:20): Yeah, that's true. I mean, I think there's still an awful lot of just startup nonsense, but you do see more startups that are being made by people who are addressing an actual need that they knew about. They bought this thing on the internet and then decided to revolutionize the industry that that thing was in, because something about their buying experience was unsatisfactory or something. Rob Collie (01:00:37): Our company essentially comes from that. I had been on all three sides of the business intelligence triangle. I had been building the software. I had, believe it or not, also hired a consulting firm to execute on that platform for us, that's something I got to do when I worked briefly for that five minutes I was over on MSN. And then now the third side is, running a consulting company, right? Rob Collie (01:01:02): And so very much filling that practical need as a subject matter expert, like seeing how the industry needed to change and all of that. But still the startup crowd, when we start talking, if I talked to Silicon Valley types, right? So often without even realizing it, they'll just make some comment, "We'd never go into any sort of labor intensive, people intensive business. We don't want that, that's a terrible business." Without realizing that they're talking about me. There's something really satisfying about running a business like this and starting a business like this, so many people, including the people who work for us, lives are so greatly improved and we get to make a living at the same time, it's awesome. Chris Rae (01:01:47): It's nice making stuff that people use, this is why you and I were working at Microsoft too. Rob Collie (01:01:50): No one's ever going to be a billionaire in this business. Chris Rae (01:01:53): That's just such a weird ephemeral moment in this kind of whole industry, where people are being made billionaires for things that were [inaudible 01:02:01] ideas, that they had an exit strategy for and that kind of stuff. 50 years from now, that's not going to be happening, it's going to be some other thing that's going nuts. It was oil a hundred years ago. We all happen to work in this field, but it's turned out to be the thing where people could make lots of money, and it was just super lucky, but you don't have to do the whole Silicon Valley nonsense. Chris Rae (01:02:18): I was asked by my wife's boss's boss, actually at some social event... My wife is now working at Microsoft, I think might get her on here at some point, because she actually does do data things. And her bosses both asked me what the exit strategy for my company was, and I mean, this is a two person business where you take pictures of logs and report them to the local environment agency, we're not about to be bought by Oracle. And I don't have an exit strategy. To me, if I had an exit strategy that would mean this business wasn't doing very well. Who on earth rates the success of what you're doing by how quickly you're going to sell it to someone else and run away? Rob Collie (01:02:49): It's echoes of the financial crisis, right? The late 2000s financial crisis. It's like, we will initiate these financial deals, but immediately sell them off as security so that someone else is holding the risk, right? Chris Rae (01:03:04): Yeah, right, right. Rob Collie (01:03:05): And, it's like, "Oh God, no, we would never hold these things for more than a millisecond." Chris Rae (01:03:09): Yeah, well I suppose I was involved in that industry a bit. Rob Collie (01:03:14): Maybe, maybe. Well, you voted with your feet, I'm sure it was an ethical and moral decision to leave that all behind. And fundamentally, the vast majority of those business models are like, "How do we get the people out of the equation?" Chris Rae (01:03:26): Right. But people are really how you made the money. I mean, there's a very short period where you can make money by churning through compute time. Soon everyone's going to do that and that's not going to make money anymore. It's people that make you money. Rob Collie (01:03:37): I guess the goal is trying to become the new infrastructure, in the same way that like the power companies and utilities and all of that, the software world offers the opportunity to be a private infrastructure that is essentially of public need, right? Chris Rae (01:03:56): And it's true. And actually, honestly my current business is sort of that. I mean, we sell it to states on an annual subscription basis, and so the goal with this is, you do a bunch of work up front and then in year 10, you will hopefully be doing not as much work, but still making the same money. I did think that about year three, but now I'm in year eight and I'm thinking it about year 20. Rob Collie (01:04:15): Okay, I'm sure you remember the podcast we did with John Hancock? Chris Rae (01:04:19): Oh, that was a good one, yeah. Rob Collie (01:04:20): He founded a SaaS company that sells to government agencies, and he's based in your neck of the woods. They're software, first and foremost, they have a freemium model where they give away a version of their product, but the freemium version of their product catches child predators. Chris Rae (01:04:38): Whoa, that's a bit of a dark free version. Rob Collie (01:04:40): Right, it's not the typical loss leader. But again, I'm just recapping the show that you listened to, so you know all of this. Chris Rae (01:04:46): Yeah, I actually made notes, I've got them somewhere. Thomas LaRock (01:04:48): Somewhere. Rob Collie (01:04:49): But yeah, so honestly, I think you're the third person we've had on the show who's built a SaaS product that is sold to government agencies. Derek Ricard also, emergency reporting is targeted at emergency service departments, like ambulance, paramedic, all that, all over the world. Chris Rae (01:05:07): Are these people all similarly old? Because I do think there's an age element to this. And I think younger programming people are going for more glamorous, exciting things that might make lots of money. And older people, I'm mid-40s, I'm not that old, but I'm old for a computer programmer. And the older people they're looking for just steady, easy-going, it brings in money every month. And conversely, a thing that our customers value a lot is that we've been doing this for eight years and we're going to keep doing it. You see this all the time that everyone wants an app for stuff. People will hire a consultant, they'll make an app, the consultant will go away because the apps finished. And then six months later there's an IRS update where the app doesn't work anymore, and they can't hire the consultant because the consultant's disappeared or isn't interested in doing it or et cetera, and then your stuff kind of falls apart. Rob Collie (01:05:48): Well, no, they executed their exit strategy. Chris Rae (01:05:50): Right. But I think it takes a while to build up relationships in government like this. And actually my business partner has worked in this field for 20 years now, so he knew these people well before I did. And we don't make anything super exciting, but we make something predictable and we're find-able on the phone and we'll support things after we've finished it, and the government customers like that stuff. One of the things with government customers is that they often can't find funding very easily. They'll fund something via some grant that they got from some agency or some other part of their own company or something, but then once that's done, you can't just pull money out of the kitty to do some more work on it. You need to go and apply for another grant, and that's not going to be for 500 bucks. So they slow machinations of government mean that they do want people who are going to kind of stick with them in the longer term, and accept that, yeah, I'm not technically being paid for this, but they're going to subscribe next year. Rob Collie (01:06:39): Integrity and commitment, again, that's not where you get the most multiple on your exit strategy. Chris Rae (01:06:43): No, but it's a job. Rob Collie (01:06:45): It's a job and it feels good. It scratches a lot of other itches that I think are just as important, more important. It turns out not everyone can be billionaires, there's not enough room for that. Chris Rae (01:06:54): Yeah, that's true. I mean, I have got two boats, as I mentioned. Rob Collie (01:06:55): So we can't let this recording end. I know that you're a busy man and you've got wall to wall meetings stacked up behind this, but there are a couple of things we absolutely have to talk about, otherwise I will feel like we've really failed. So first of all, I'm glad that we were able to have this friendly conversation because it's one of the last times that we will ever be friendly with one another, given that we're about to enter into competition in the hotly contested startup market of what you call Breadpacitor, but what we know to be Doughpacitor on our side. Can you tell the story of how you came around to this first mover, but going to ultimately fail to succeed in the face of competition, this Breadpacitor? Chris Rae (01:07:42): Did you sign the NDA, I did send it through. Rob Collie (01:07:45): I don't remember seeing that. Thomas LaRock (01:07:46): I don't sign NDAs, sorry. Rob Collie (01:07:48): But he knows that, he's listened to all of our podcasts, he's heard us talk about how we don't sign NDAs. Chris Rae (01:07:53): Yeah, well I'm going to assume you use some discretion here, but just between- Rob Collie (01:07:57): FriendDA. No, we won't tell anyone. Thomas LaRock (01:07:58): A friendDA, I love that. Rob Collie (01:08:03): I didn't make that up. I heard it recently, and I'm like, "I'm using that from now on." Chris Rae (01:08:06): I don't have an electronics background, but I do have a radio-controlled submarine, a little one. It was on Kickstarter, it's this company called Chasing that make submarines. You know DJI, the drone company? They make some really fancy ones that you can use for actually making films, but they also make ones you can afford to use in your backyard. And so this Chasing company is a bit like that, but for submarines. So they make ones that you can use to inspect an oil rig, but they also make a couple of cheaper ones, it's one that's only cheap if you're actually a person who owns a yacht. But then there was one that they put on Kickstarter, it was like 300 and something bucks, and I backed it on Kickstarter. And the way it works is you've got the actual submarine device, there's a cable that goes to this floating buoy, I call it buoy, I think you'd call it buoy, and that's not in my books. It's this cable that goes to this buoy on the surface, because you can't transmit radio waves through water, so- Rob Collie (01:08:54): Just stop, we should make very clear for our American listeners, there is not a male human child swimming around on the surface- Chris Rae (01:08:59): Oh no, it's a male human child, actually. Yeah. Rob Collie (01:09:04): ... with a cable. Chris Rae (01:09:05): There's a cable that sticks into the male human child. You can put Water Wings on them. Rob Collie (01:09:08): How much do you get on a charge of the buoy? Chris Rae (01:09:11): It's two bowls of cereal and you're pretty much good for an hour to two with the submarine. Yeah, so that bit floats on the surface and then you connect to that on your phone via Wi-Fi and you get live video from the submarine, it records onto an SD card, or onto something that's in the submarine, and you drive around the place. And so I've been messing around with this here in Seattle, and I have kids, because you don't want to be an adult is going around and playing with a toy submarine on a crowded beach, it makes you look like some sort of dweeb. And so I was hoping that I'd be able to take the kids and then it would look like the kids were playing with it, but my kids actually, it turns out, are not interested in any of the things that I'm interested in and don't want to play with this remote control submarine. And Seattle, there's not that many exciting things you can go and look at under the water, so I kind of did my fill of things around here. Chris Rae (01:09:55): We had this lucky trip in the middle of the pandemic where we went to Europe, obviously I'm Scottish, and I mentioned that my wife's Greek, so we were going to Scotland and Greece. And of course I was going to take this submarine to Greece because her parents live kind of near the sea and you can kind of mosey down to the ocean, also them being Greek they kind of spend a lot of the evenings arguing with each other, and so I could sneak off with my submarine. The first night I sneaked off with my submarine down to the sea and it died. Rob Collie (01:10:22): You flew internationally with a buoy in your luggage? Chris Rae (01:10:26): Yeah, with a buoy in my luggage, I risked a lot. And this is actually a little bit complicated, because my wife decided that this was a trip we were going to pack lighter on, and caught me packing this submarine. So I used up some birdie points to get the submarine to Greece, certainly, but then it died. And so I thought, "Well, I can't get any spare parts, it's not a big company. So I'm not going to get any spare parts in Greece, and so it might just try and fix it." And I was pretty sure that the part that died was the buoy part because the connector was wiggly, I thought maybe water had got into it. And it's not openable, it's been glued together. I hacked it open with a saw, and I thought, you could still use the submarine if you don't put the buoy in the water, you've just got the length of the cable. Chris Rae (01:11:03): I hacked the top off it, and I realized it had corrosion on the motherboard. And so I did a bit of looking on the internet and they said, "Oh, you can just use distilled water and a toothbrush." So I bought some distilled water, and I stole my sister-in-law's toothbrushes. I said, "Have you got a toothbrush I could use to clean this thing?" She's like, "Oh yeah, sure." So I started cleaning this thing only to discover that this toothbrush was covered in toothpaste. If that had been my family, they'd have assumed that, "Oh yeah, you need a toothbrush to clean some mechanical thing." First of all, I covered the motherboard in toothpaste and then worked out how I was going to get rid of that. Chris Rae (01:11:35): And then I cleaned all the corrosion off and I plugged it in and it still wasn't working, but it was not working in a kind of different way. And because I had taken the top of this buoy thing off, I was messing around with it, and I had discovered that there was a capacitor in there... When you plugged it in, there was obviously a power light on this PCB and it would flicker. And I thought, I bet that's not supposed to flicker. And there was a capacitor, it was the only component on there apart from chips, it was this capacitor. And I was messing around, I thought it must be a power regulator or something. So I was trying to wiggle it, see if it was not connected properly or something. And as I was touching it, the light was going on more solidly on the board. Rob Collie (01:12:08): Just when you grabbed the capacitor? Chris Rae (01:12:10): When I was wiggling the capacitor in my fingers, and I thought, "Well, it must be a loose connection or something like that." Well I got to the point where I could lift it up on this capacitor, so it [inaudible 01:12:18] loose connection, it was kind of flickering less. So I thought, I wonder if it works. So I kind of cranked the thing up, starting my phone, held onto this capacitor, if you hold onto it quite tight, the light would actually go pretty solid and you'd get a green light on the submarine, which meant it was ready to go. And sure enough, I connect to Wi-Fi, sat in the room we were sitting in with it running for half an hour taking video just in the room, but with me holding this capacitor. Chris Rae (01:12:39): And I thought, well, I've got a proof of concept now for her to fix this thing, whatever on earth is wrong with it. And I just need to work out... Because you need two hands to control the thing, and the children are even less interested in just sitting, holding a capacitor. And the wife, I did actually ask the wife, but she said some words that I'm not going to use on your broadcast. And so I thought, "Well, I need something that's going to replace my finger and it's better." And I thought the things that I might be doing are, I know that I'm not fixing this loose connection, but I'm wondering if I'm earthing it or something like that. So I got a spoon out of my mother-in-law's kitchen, and I put that on it, it didn't seem to make any difference, it was as if I was doing nothing. And I thought, "Well, it's something to do... Maybe it's the heat." Right? So I kind of left it in the sun for a bit, and then that didn't help. Chris Rae (01:13:22): And I actually put it on Facebook, you probably saw this Facebook thread. I put it on Facebook saying, "I've recently ended what I can do with this buoy, does anyone have any ideas about how I should really fixed it?" And I was like, "If I hold it up to my hand, it works and if I tried all these other things, it doesn't work." So I thought, "I've got some smart friends." So it turns out they were all completely useless. They're like, "Ah, capacitors are complicated things." And nobody really gave me any advice at all. Chris Rae (01:13:42): So I closed Facebook, and I just went looking for other things that might be like my hand. So I actually got a piece of raw chicken out of the fridge, but I realized it's quite wet, raw chicken, right? And so I was going to flop this onto this motherboard and I think it would cause other things. So I just put it in a plastic bag and I put it on the motherboard over this capacitor., It didn't do anything at all, so it really had to touch it. So the raw chicken was out, and I got a banana and I tried sort of squishing the banana onto the capacitor, well that didn't do anything. And then I got a piece of bread, there's loads of bread in Greece. So I go a bit of bread, just a chunk of a loaf of bread, I got a picture of it somewhere. Well, actually I made a video of this whole saga that I posted on some ask electronics thing on Reddit. And then I get the adoration of my kids, because I might get another 10 views. So it's over a hundred views now. Chris Rae (01:14:27): Yeah, so I've got this piece of bread, stuck it on it and it actually worked for a while. 10 minutes I managed to run it again in the bedroom, took video, but then it stopped and I couldn't get that piece of bread to work anymore with it. And I noticed it was a bit warm and I thought, "Okay, well I've got the solution here, so I just need to productionalize it." And so I got a bottle cap and I stuffed a whole bunch of bread in that. And then I wrapped it kind of with tape such that there was still a hole for the capacitor. I found if the bread was wet it made a big difference. Chris Rae (01:14:51): So I think that was why the piece of bread didn't work. So I drink some water, distilled water because I'm a professional here, I put some distilled water into the bottle cap thing and then I plop the bottle cap upside down on top of this capacitor and the submariner worked for like half an hour. And so I could actually go out and take little videos and things, and then it would die again and I'd have to take the bread capacitor off and put a little bit more water in it and then I'd plop it on again and I'd get another hour out of it. Chris Rae (01:15:15): And so this all works pretty good. I mean, it was a bit of a hassle because sometimes you have to put more water in the bread and sometimes it'd be a bit windy and it would knock over the buoy and the bread would fall off and I'd have to go and find it again and put it on. But it worked pretty well until actually the last day I was in Greece when it died completely, and there was quite a lot of burn marks on the bread. And there were also some marks on the PCB that I think were from bits of wet bread that had fallen out of the bread capacitor and maybe damaged the PCB, so it's probably not returnable at this point. Thomas LaRock (01:15:45): You invented toast. Chris Rae (01:15:47): It's a not effective way of making toast, I got to say. Thomas LaRock (01:15:51): It's not efficient. Chris Rae (01:15:51): No, it's like Timbits toast kind of thing, I could do. Thomas LaRock (01:15:56): I'm just thinking that there might be a way for us to take this and get some VC money. Chris Rae (01:16:01): No you're right. And I mean, as they say in start-up land, pivot. So I could pivot onto toast. Rob Collie (01:16:08): Just a chunk of bread fixes your remote control submarine? Chris Rae (01:16:13): I think with this particular project, that strategy is definitely to exit. Rob Collie (01:16:19): The exit strategy here is just to exit. It's... Chris Rae (01:16:23): Man, yeah, but I've already done this video. Rob Collie (01:16:25): Now that you have shared all of this, and a copycat startup with a much more compelling name. Chris Rae (01:16:31): What's your story in this game? Are you also involved in submarine repair with bread? Rob Collie (01:16:36): Doughpacitor is already a thing, I incorporated Doughpacitor last month. Chris Rae (01:16:40): So you didn't come up with any idea at all, you're just stealing mine and using a different- Rob Collie (01:16:43): Oh yeah, I'm just going to out-compete you because I have a better name. Chris Rae (01:16:46): No, I appreciate you're candor. Rob Collie (01:16:48): Totally just stealing your public domain Facebook idea, I'm funded by Zuckerberg. Chris Rae (01:16:52): Yeah, no, I'm sure you are. Yeah, yeah. Have you got nice tables that go up and down? Thomas LaRock (01:16:57): See, it'd be better if he said you were funded by Wonder Bread. Chris Rae (01:17:00): Oh yeah, you can revolutionize their whole industry. Rob Collie (01:17:02): The whole gluten-free movement is really threatening the Wonder Bread company, they need to pivot, they need to pivot into submarine repair, yeah Chris Rae (01:17:10): This might work with gluten-free. Rob Collie (01:17:12): All right, so here's my other question for you, in the last 10 years, what's your turnover rate on automobile? Chris Rae (01:17:17): I see where you're heading with this, and I have actually not owned that many cars. How many cars have you owned in your life? Rob Collie (01:17:24): I don't know, it's probably around the order of six. Chris Rae (01:17:27): Okay, well, it's more than that. Yeah. Maybe I've been hanging around with too many car racing people. Rob Collie (01:17:31): The thing that always blows me away is that when you're acquiring a car or thinking about acquiring a car, it's always at a price point that I didn't even know you could get a car. Chris Rae (01:17:42): Yeah, I've been into cars for a long time. My mate Jan was into cars and built... My dad's just actually recently built a car from scratch, that you can buy plans and you will go off and build this car. It was actually just after he retired, and I think my mother agreed to this concept because she thought my dad was going to get these plans, he was going to work on this in the garage until he died. And it was going to be a reasonably good investment, and she would not have to put up with him fumbling around the house getting under her feet. But my dad being somewhat good at this kind of thing and being retired, knocked out this car in six months. My mother thought it was the biggest waste of money ever. But actually it turns out that he finishes it, he did a really good job of it actually. Chris Rae (01:18:17): And then he of course got on the internet, there's forums for these build your own car things, and most of them are being built by people like us who are middle-aged with jobs and kids and things. And they've got halfway through this project and stalled, they got stuck on something. And so my dad ended up being one of those guys on the forums who's just like, "Oh, I could just come and help you with that for a week." And so I can send you a picture of these too, but these are three wheeled cars, so they're a bit odd. It's like a motorcycle engine in the front, no roof, that kind of thing. They look a bit like an old Morgan, if you know what that looks like. And the people who make these are usually a little bit eccentric, and so he's been going around Europe, staying in castles and fancy chalets, [inaudible 01:18:54] eccentric old men who are building these cars that have got stuck on it. Chris Rae (01:18:57): So he's off doing that kind of thing all the time. Most of my family have had this long interest in cars, and I've been racing cars for, well, 10 years since we came here, and other kinds of racing in the UK. And I've also been buying weird cars. I have of course a 911, which I bought when it was eight years old, eight years ago. And I paid $39,000, so that car was twice as much money as I'd ever spent on any other car at all. And all of our friends now, they're like, "Oh well, you've got that fancy Porsche." And I'm like. "That SUV you're driving around in was like $48,000. You don't even know what color it is." So I just have not spent large amounts of money of cars, but have gone through some kind of weirdish ones. Rob Collie (01:19:34): You're a very entertaining Facebook read. It doesn't matter what you're up to, I don't have to be into it to appreciate the eccentricity of it, right? I'm like, "Oh, I can relate to that. That I can relate to." Chris Rae (01:19:48): 10 years ago, Facebook was all... You go onto Facebook to see what people are doing. Like, they went camping with their kids or they've done something at work or something like that. And now Facebook's all just, here's something you want to read for your betterment. And, here's an argument I'm going to start with people. And also the things people talk about themselves on Facebook are always unambiguously positive things. It's like, "Oh, I'm so honored to have taken part in this thing, and years of hard work." But I'm not sure that people posted, "I actually fell down a flight of stairs this morning while trying to put a sticking plaster on my knee." That's much more interesting, that makes me feel better reading [inaudible 01:20:23]. So I feel like you should just put on things that you're doing, whether they went well or badly, and don't put on things that you think other people want to read. If I was in charge of Facebook I would change it so you cannot post links on it anymore. Why is that particularly good? Rob Collie (01:20:36): I think I share that belief. I try... First of all, if I don't post very much on Facebook anymore. Chris Rae (01:20:41): I like your output too. I mean, I'll do some fluffing here. Rob Collie (01:20:43): I appreciate that. I mean, it's like, if I'm not contributing- Chris Rae (01:20:46): Especially your [inaudible 01:20:48]. Rob Collie (01:20:48): ... if I'm not contributing to the mirth level of the greater picture when I'm posting on there, that I don't really want to be on there. Chris Rae (01:20:57): Yeah, then it's not something people are going to want to read. Rob Collie (01:21:03): The pictures of my ridiculous looking helmet that I got to ride our scooters, the before and after. Like, "This is the helmet before the 30 minute first ride on the scooter, and this is me in the hospital after the 30 minute first ride." Chris Rae (01:21:15): Yeah, that's A1 Facebook content, in my opinion. Rob Collie (01:21:18): Well, Chris, I know this has been a real honor for you, longtime listener, first time guest, that kind of thing. Chris Rae (01:21:25): It's been my dream ever since I started listening to this to really take part in it. Rob Collie (01:21:29): Yeah. I really have appreciated it. I learned a lot about your background I didn't know. You might be the clearest example yet of the contrast between, not into the math, not into that stuff, but really into this other stuff. Chris Rae (01:21:41): Right, and I have hope for people who are not into math but like computers. Rob Collie (01:21:44): That's right. Chris Rae (01:21:45): You too could be as successful as I am, have an inflatable boat and a book that makes you a hundred bucks a year. Rob Collie (01:21:50): Two inflatable boats. Chris Rae (01:21:50): Two inflatable boats, sorry. You're right, as of this morning, yeah, yeah. Rob Collie (01:21:55): I really do appreciate you taking time to spend with us, it was great catching up with you. Chris Rae (01:21:59): Yeah, you too. This was just a lot of fun. Announcer (01:22:01): Thanks for listening to the Raw Data By P3 Adaptive Podcast. Let the experts at P3 Adaptive help your business, just go to p3adaptive.com. Have a data day!

There's a place where sports and data meet, and it's as powerful a collision as on any football field!  Jeff Sagarin has been a figurehead in the sports analytics realm for decades, and we're thrilled to have had the chance to have him on to talk about his data journey!  There's a fair mix of math AND sports geek out time in this episode.  And, did we mention that Dr. Wayne Winston is sitting in on this episode as well? References in this Episode: 2 Frictionless Colliding Boxes Video Scorigami Episode Transcript: Rob Collie (00:00:00): Hello, friends. Today's guest is Jeff Sagarin. Is that name familiar to you? It's very familiar to me. In my life, Jeff's work might very well be my first brush with the concept of using data for any sort of advantage. His Power Ranking Columns, first appeared in USA Today in 1985, when I was 11 years old. And what a fascinating concept that was. Rob Collie (00:00:29): It probably won't surprise you if I confess that 11-year-old me was not particularly good at sports, but I was still fascinated and captivated by them. 11-year-old kids in my neighborhood were especially prone to associating sports with their tribal identity. Everyone had their favorite teams, their favorite sports stars. And invariably, this led to arguments about which sports star was better than the other sports star, who was going to win this game coming up and who would win a tournament amongst all of these teams and things of that sort. Rob Collie (00:01:01): Now that I've explained it that way though, I guess being an adult sports fan isn't too terribly different, is it? Those arguments, of course, aren't the sorts of arguments where there's anything resembling a clear winner. But in practice, the person who won was usually the one with the loudest voice or the sickest burn that they could deliver to their friends. And then in 1985, the idea was planted in my head by Jeff Sagarin's column in USA Today, that there actually was a relatively objective way to evaluate teams that had never played against one another and likely never would. Rob Collie (00:01:33): I wasn't into computers at the time. I certainly wasn't into the concept of data. I didn't know what a database was. I didn't know what a spreadsheet was. And yet, this was still an incredibly captivating and powerful idea. So in my life, Jeff Sagarin is the first public figure that I encountered in the sports analytics industry long before it was cool. And because it was sports, a topic that was relevant to 11-year-old me, he's really also my first brush with analytics at all. Rob Collie (00:02:07): It's not surprising then, that to me, Jeff is absolutely a celebrity. As a guest, in insider podcasting lingo, Jeff is what we call a good get. We owe that pleasure, of course, to him being close friends with Wayne Winston, a former guest on the show, who also joined us today as co-guest. Rob Collie (00:02:28): Now, if none of that speaks to you, let's try this alternate description. He's probably also the world's most famous active FORTRAN programmer. I admit that I was so starstruck by this that I didn't even really push as hard as I normally would, in terms of getting into the techniques that he uses. I didn't want to run afoul of asking him for trade secrets. At times, this conversation did devolve into four dudes sitting around talking about sports. Rob Collie (00:02:59): But setting that aside, there are some really, really interesting and heartwarming things happening in this conversation as well. Again, the accidental path to where he is today, the intersection of persistence and good fortune that's required really for success in anything. Bottom line, this is the story of a national and highly influential figure at the intersection of the sports industry and the analytics industry for more than three decades. It's not every day you get to hear that story. So let's get into it. Announcer (00:03:34): Ladies and gentlemen, may I have your attention, please? Announcer (00:03:39): This is the Raw Data by P3 Adaptive podcast with your host, Rob Colley and your co-host, Thomas LaRock. Find out what the experts at P3 Adaptive can do for your business. Just go to p3adaptive.com. Raw Data by P3 Adaptive is data with the human element. Rob Collie (00:04:02): Welcome to the show, Jeff Sagarin. And welcome back to the show. Wayne Winston. So thrilled to have the two of you with us today. This is awesome. We've been looking forward to this for a long time. So thank you very much gentlemen, for being here. Jeff Sagarin (00:04:16): You're welcome. Rob Collie (00:04:18): Jeff, usually we kick these things off with, "Hey, tell us a little about yourself, your background, blah, blah, blah." Let's start off with me telling you about you. It's a story about you that you wouldn't know. I remember for a very long time being aware of you. Rob Collie (00:04:35): So I'm 47 years old, born in 1974. My father had participated for many years in this shady off-the-books college football pick'em pool that was run out of the high school in a small town in Florida. Like the sheets with everybody's entries would show up. They were run on ditto paper, like that blue ink. It was done in the school ditto room and he did this every year. This was like the most fascinating thing that happened in the entire year to me. Like these things showing up at our house, this packet of all these picks, believe it or not, they were handwritten. These grids were handwritten with everyone's picks. It was ridiculous. Rob Collie (00:05:17): He got eliminated every year. There were a couple of hundred entries every year and he just got his butt kicked every year. But then one year, he did his homework. He researched common opponents and things like that or that kind of stuff. I seem to recall this having something to do timing wise with you. So I looked it up. Your column first appeared in USA Today in 1985. Is that correct? Jeff Sagarin (00:05:40): Yeah. Tuesday, January 8th 1985. Rob Collie (00:05:44): I remember my dad winning this pool that year and using the funds to buy a telescope to look at Halley's Comet when it showed up. And so I looked up Halley's Comet. What do you know? '86. So it would have been like the January ballgames of 1986, where he won this pool. And in '85, were you power ranking college football teams or was that other sports? Jeff Sagarin (00:06:11): Yes. Rob Collie (00:06:12): Okay. So when my dad said that he did his research that year, what he really did was read your stuff. You bought my dad a telescope in 1986 so that we could go have one of the worst family vacations of all time. It was just awful. Thank you. Jeff Sagarin (00:06:31): You're very welcome. Rob Collie (00:06:39): I kind of think of you as the first publicly known figure in sports analytics. You probably weren't the first person to apply math and computers to sports analytics, but you're the first person I heard of. Jeff Sagarin (00:06:51): There is a guy that people don't even talk about very much. Now a guy named Earnshaw Cook, who first inspired me when I was a sophomore in high school in the '63-'64 school year, there was an article by Frank Deford in Sports Illustrated about Earnshaw Cook publishing a book called Percentage Baseball. So I convinced my mom to let me have $10 to order it by mail and I got it. I started playing around with his various ideas in it. He was the first guy I ever heard of and that was in March of 1964. Rob Collie (00:07:28): All right, so everyone's got an origin story. Jeff Sagarin (00:07:31): The Dunkel family started doing the Dunkel ratings back I believe in 1929. Then there was a professor, I think he was at Vanderbilt, named [Lipkin House 00:07:41], he was I think at Vanderbilt. And for years, he did the high school ratings in states like maybe Tennessee and Kentucky. I think he gave Kentucky that Louisville courier his methodology before he died. But I don't know if they continue his work or not. But there were people way before me. Rob Collie (00:08:03): But they weren't in USA Today. Jeff Sagarin (00:08:04): That's true. Rob Collie (00:08:06): They weren't nationally distributed, like on a very regular basis. I've been hearing your name longer than I've even been working with computers. That's pretty crazy. How did you even get hooked up with USA Today? Jeff Sagarin (00:08:23): People might say, "You got lucky." My answer, as you'll see as well, I'd worked for 12 years to be in a position to get lucky. I started getting paid for doing this in September of 1972 with an in-house publication of pro football weekly called Insider's Pro Football Newsletter. Jeff Sagarin (00:08:45): In the Spring of '72, I'd written letters to like 100 newspapers saying because I had started by hand doing my own rating system for pro football in the fall of 1971. Just by hand, every Sunday night, I'd get the scores and add in the Monday night. I did it as a hobby. I wasn't doing it for a living. I did it week by week and charted the teams. It was all done with some charts I'd made up with a normal distribution and a slide rule. So I sent out letters in the spring of '72 to about 100 papers saying, "Hey, would you be interested in running my stuff?" Jeff Sagarin (00:09:19): They either didn't answer me or all said, "No, not interested." But I got a call right before I left to go to California when an old college friend that spring. It was from William Wallace, who was a big time football correspondent for The New York Times. That anecdote may be in that article by Andy Glockner. He called me up, he was at the New York Times, but he said also, "I write articles for extra money for pro football weekly. I wanted to just kind of talk to you." Jeff Sagarin (00:09:49): He wrote an article that appeared in Pro Quarterback magazine in September of '72. But during the middle of that summer, I got a phone call from Pro Football weekly, the publisher, a guy named [inaudible 00:10:04] said, "Hey Jeff. Have you seen our ad in street and Smith's?" It didn't matter. It could have been their pro magazine or college. I said, "Yeah, I did." And he said, "Do you notice it said we've got a world famous handicapper to do our predictions for us?" I said, "Yeah, I did see that." He said, "How would you like to be that world famous handicapper? We don't have anybody." Jeff Sagarin (00:10:25): We just said that because he said William Wallace told us to call you. So I said, "Okay, I'll be your world famous handicapper." I didn't start off that well and they had this customer, it was a paid newsletter and there was a customer from Hawaii. He had a great name, Charles Fujiwara. He'd send letters every week saying, "Sagarin's terrible, but he's winning a fortune for me. I just reverse his picks every week." So finally, finally, my numbers turn the tide and I had this one great week, where I went 8-0. He sent another letter saying, "I'm bankrupt. The kid destroyed me." Because he was reversing all my picks. That's a true story. Rob Collie (00:11:07): At least he had a sense of humor. It sounds like a pretty interesting fellow on the other end of that letter. Jeff Sagarin (00:11:13): He sounds like he could have been like the guy, if you've ever seen reruns of the old show, '77 Sunset Strip. In it, there this guy who's kind of a racetrack trout gambler named Roscoe. He sounds like he could have been Roscoe. Rob Collie (00:11:26): We have to look that one up. Dr. Wayne Winston (00:11:27): It's before your time. Rob Collie (00:11:28): I don't think I saw that show. Jeff Sagarin (00:11:29): Yeah. Wayne's seen it though. Rob Collie (00:11:31): Yes. I love that. There are things that are both before my time and I have like old man knees. So I've heard this kind of thing before, by the way. It's called the 10-year overnight success. Jeff Sagarin (00:11:47): I forgot. How did I get with USA Today? I started with Pro Football weekly and continued with them. I was with them until actually why don't we say sometime in the fall of '82. I ended up in other newspapers, little by little: The Boston Globe, Louisville Courier Journal. And then in the spring of '81, I got into a conversation over the phone with Jim van Valkenburg, who is the stat guy at the NCAA. I happened to mention that going into the tournament, I had Indiana to win the tournament. They were rated like 10th in the conventional polls. Jeff Sagarin (00:12:23): And so he remembered that and he kept talking behind the scenes to people in the NCAA about that. And so years later, in 1988, they called me out to talk to them. But anyhow, I had developed a good reputation and I gave him as a reference. Wayne called me up excitedly in let's say, early September of 1984. He said, "Hey, Jeff. You've got to buy a copy of today's USA Today and turn to the end of the sports section. You're going to be sick." Jeff Sagarin (00:12:53): I said, "Really? Okay." So I opened to where he said and I was sick. They had computer ratings by some guy. He was a good guy named Thomas Jech, J-E-C-H. And I said, "Damn, that should be me. I've been doing this for all these years and I didn't even know they were looking for this." So I call up on the phone. Sometimes there's a lot of luck involved. I got to talk to a guy named Bob Barbara who I believe is retired now there. He had on the phone this gruff sounding voice out of like a Grade B movie from the film, The War. "What's going on Kitty?" It sounds like he had a cigar in his mouth. Jeff Sagarin (00:13:30): I said, "Well, I do these computer ratings." [inaudible 00:13:33] Said "Well, really? That's interesting. We've already got somebody." He said, "But how would you even send it to us?" I said, "Well, I dictate over the phone." He said, "Dictate? We don't take dictation at USA Today, kid. Have you ever heard of personal computers and a modem?" I said, "Well, I have but I just do it on a mainframe at IU and I dictate over the phone to the Louisville Courier and the local..." Jeff Sagarin (00:13:58): Well, the local paper here, I gave them a printout. He said, "Kid, you need to buy yourself a PC and learn how to use a modem." So I kind of was embarrassed. I said, "Well, I'll see." So about 10 days later, I called him up and said, "Hey, what's the phone number for your modem?" He said, "Crap. You again, kid? I thought I got rid of you." He says, "All right. I'll give you the phone number." So I sent him a sample printout. He says, "Yeah, yeah, we got it. Keep in touch. We're not going to change for football. But this other guy, he may not want to do basketball. So keep in touch. Who knows what will happen for basketball?" Jeff Sagarin (00:14:31): So every month I'd call up saying, "It's me again, keeping touch." He said, "I can't get rid of you. You're like a bad penny that keeps turning up." So finally he says look, after about five of these calls, spreading out until maybe late November, "Look kid, why don't you wait... Call me up the first Sunday of the new year," which would have been like Sunday, January 6 of 1985 I believe. So I waited. I called him up. Sure enough, he said, "You again?" I said, "You told me you wanted to do college basketball." Jeff Sagarin (00:15:04): He said, "Yeah, you're kind of right. The other guy doesn't want to do it." So he said, "Well, do you mind if we call it the USA Today computer ratings? We kind of like to put our own name on everything." I said, "Well, wait a minute. During the World Series, you had Pete Rose as your guest columnist, you want not only gave his name, but you had a picture of him." He said, "God damn it." He said, "I can't..." He said, "You win again kid. Give us a bio." Jeff Sagarin (00:15:32): An old friend of both me and Wayne was on a business trip. He lived in California, but one of the companies he did work for was Magnavox, which at the time had a presence in Fort Wayne. So he had stopped off in Bloomington so we could say hi. We hadn't seen each other for many years. So he wrote my bio for me, which is still used in the agate in the USA Today. So it's the same bio all these years. Jeff Sagarin (00:15:56): So they started printing me on Tuesday, January 8 of 1985. On the front page that day and I got my editor of a couple years ago, he found an old physical copy of that paper and sent it to me and I thought that's pretty cool. And on the front page, they said, "Well, this would be the 50th birthday of Elvis Presley." I get, they did not have a banner headline at the top, "Turn to the sports and see Jeff Sagarin's debut." That was not what they did. It was all about Elvis Presley. And so people will tell me, "Wow! You got really lucky." Jeff Sagarin (00:16:30): Yeah, but I was in a position. I'd worked for 12 years since the fall of '72 to get in position to then get lucky. They told me I had some good recommendations from people. Rob Collie (00:16:42): Well, even that persistence to keep calling in the face of relatively discouraging feedback. So that conversation took place, and then two days later, you're in the paper. Jeff Sagarin (00:16:54): Well, yeah. He said, "Send us the ratings." They might have needed a time lag. So if I sent the ratings in on a Sunday night or Monday morning, they'd print them on Tuesday. They're not as instant. Now, I update every day on their website. For the paper, they take whatever the most recent ones they can access off their website, depending on I've sent it in, which is I always send them in early in the morning like when I get up. So they print on a Tuesday there'll be taking the ratings that they would have had in their hands Monday, which would be through Sunday's games. Rob Collie (00:17:26): That Tuesday, was that just college basketball? Jeff Sagarin (00:17:28): Then it was. Then in the fall of 85. They began using me for college football, not that they thought I was better or worse one way or the other than Thomas Jech who was a smart guy, he was a math professor at the time at Penn State. He just got tired of doing it. He had more important things to do. Serious, I don't mean that sarcastically. That was just like a fun hobby for him from what I understand. Rob Collie (00:17:50): I was going to ask you if you hadn't already gone and answered the question ahead of time. I was going to ask you well, what happened to the other guy? Did you go like all Tonya Harding on him or whatever? Did you take out your rival? No, sounds like Nancy Kerrigan just went ahead and retired. Although I hate to make you Tonya Harding in this analogy and I just realized I just Hardinged you. Jeff Sagarin (00:18:10): He was just evidently a really good math professor. It was just something he did for fun to do the ratings. Rob Collie (00:18:17): Opportunity and preparation right where they intersect. That's "luck". Jeff Sagarin (00:18:22): It would be as if Wally Pipp had retired and Lou Gehrig got to replace him in the analogy, Lou Gehrig gets the first base job but actually Wally Pipp in real life did not retire. He had the bad luck to get a cold or something or an injury and he never got back in the starting lineup after that. Rob Collie (00:18:38): What about Drew Bledsoe? I think he did get hurt. Did we ever see him again? Thomas LaRock (00:18:43): The very next season, he was in Buffalo and then he went to Dallas. Rob Collie (00:18:46): I don't remember this at all. Thomas LaRock (00:18:47): And not only that, but when he went to Dallas, he got hurt again and Tony Romo came on to take over. Rob Collie (00:18:53): Oh my god! So Drew Bledsoe is Wally Pipp X2. Thomas LaRock (00:18:58): Yeah, X2. Rob Collie (00:19:02): I just need to go find wherever Drew Bledsoe is right now and go get in line behind him. Thomas LaRock (00:19:08): He's making wine in Walla Walla, Washington. I know exactly where he is. Rob Collie (00:19:12): I'm about to inherit a vineyard gentlemen. Okay, so Wayne's already factored into this story. Dr. Wayne Winston (00:19:23): A little bit. Rob Collie (00:19:23): A bit part but an important one. We would call you Mr. Narrative Hook in the movie. Like you'd be the guy that's like, "Jeff, you've got to get a copy of USA Today and turn to page 10. You're going to be sick." Jeff Sagarin (00:19:37): Well, I was I'm glad Wayne told me to do it. If I'd never known that, who knows what I'd be doing right now? Rob Collie (00:19:44): Yeah. So you guys are longtime friends, right? Dr. Wayne Winston (00:19:47): Yeah. Jeff, should take this. Jeff Sagarin (00:19:49): September 1967 in the TV room at Ashdown Graduate's House across from the dorm we lived, because the graduate students there had rigged up, we call it a full screen TV that was actually quite huge. It's simply projected from a regular TV onto a maybe a 10 foot by 10 foot old fashioned movie projector screen. We'd go there to watch ballgames. Okay, because better than watching on a 10 inch diagonal black and white TV in the dorm. And it turned out we both had a love for baseball and football games. Thomas LaRock (00:20:26): So just to be clear, though, this was no ordinary school. This is MIT. Because this is what people at MIT would do is take some weird tech thing and go, "We can make this even better, make a big screen TV." Jeff Sagarin (00:20:38): We didn't know how to do it, which leads into Wayne's favorite story about our joint science escapades at MIT. If Wayne wants to start it off, you might like this. I was a junior and Wayne was a sophomore at the time. I'll set Wayne up for it, there was a requirement that MIT no matter what your major, one of the sort of distribution courses you had to take was a laboratory class. Why don't we let Wayne take the ball for a while on this? Dr. Wayne Winston (00:21:05): I'm not very mechanically inclined. I got a D in wood shop and a D in metal shop. Jeff's not very mechanically inclined either. We took this lab class and we were trying to figure out identifying a coin based on the sound waves it would produce under the Scylla scope. And so the first week, we couldn't get the machine to work. And the professor said, "Turn it on." And so we figured that step out and the next week, the machine didn't work. He said, "Plug it in." Jeff can take it from there. Jeff Sagarin (00:21:46): It didn't really fit the mathematical narrative exactly of what metals we knew were in the coin. But then I noticed, nowadays we'd probably figure out this a reason. If we multiplied our answers by something like 100 pi, we got the right numbers. So they were correctly proportional. So we just multiplied our answers by 100 pi and said, "As you can see, it's perfectly deducible." Rob Collie (00:22:14): There's a YouTube video that we should probably link that is crazy. It shows that two boxes on a frictionless surface a simulation and the number of times that they collide, when you slide them towards a wall together, when they're like at 10X ratio of mass, the number of times that they impact each other starts to become the digits of pi. Jeff Sagarin (00:22:34): Wow. Rob Collie (00:22:35): Before they separate. Jeff Sagarin (00:22:36): That's interesting. Rob Collie (00:22:36): It's just bizarre. And then they go through explaining like why it is pi and you understand it while the video is playing. And then the video ends and you've completely lost it. Jeff Sagarin (00:22:49): I'm just asking now, are they saying if you do that experiment an infinite amount of times, the average number of times they collide will be pi? Rob Collie (00:22:57): That's a really good question. I think it's like the number of collisions as you increase the ratios of the weight or something like that start to become. It's like you'll get 314 collisions, for instance, in a certain weight ratio, because that's the only three digits of pi that I remember. It's 3.14. It's a fascinating little watch. So the 100 pi thing, you said that, I'm like, "Yeah, that just... Of course it's 100 pi." Even boxes colliding on a frictionless surface do pi things apparently. Jeff Sagarin (00:23:29): Maybe it's a universal constant in everything we do. Rob Collie (00:23:29): You just don't expect pi to surface itself. It has nothing to do with waves, no wavelength, no arcs of circles, nothing like that. But that sneaky video, they do show you that it actually has something to do with circles and angles and stuff. Jeff Sagarin (00:23:44): Mutual friend of me and Wayne, this guy named Robin. He loves Fibonacci. And so every time I see a particular game end by a certain score, I'll just say, "Hey, Robin. Research the score of..." I think it was blooming to North against some other team. And he did. It turned out Bloomington North had won 155-34, which are the two adjacent Fibonacci, the two particular adjacent Fibonacci. Robin loves that stuff. You'll find a lot of that actually. It's hard to double Fibonacci a team though. That would be like 89-34. Rob Collie (00:24:18): I know about the Fibonacci sequence. But I can't pick Fibonacci sequence numbers out of the wild. Are you familiar with Scorigami? Jeff Sagarin (00:24:26): Who? I'd never heard of it obviously. Rob Collie (00:24:29): I think a Scorigami is a score in the NFL that's never happened. Jeff Sagarin (00:24:32): There was one like that about 10 years ago, 11-10, I believe. Pittsburgh was involved in the game or 12-11, something like that. Rob Collie (00:24:40): I think there was a Scorigami in last season. With scoring going up, the chances of Scorigami is increasing. There's just more variance at the higher end of the spectrum of numbers, right? Jeff Sagarin (00:24:50): I've always thought about this. In Canada, Canadian football, they have this extra rule that I think is kind of cool because it would probably make more scores happen. If a punter kicks the ball into the end zone, it can't roll there. Like if he kicks it on the fly into the end zone and the other team can't run it out, it's called a rouge and the kicking team gets one point for it. That's kind of cool. Because once you add the concept of scoring one point, you make a lot more scores more probable of happening. Rob Collie (00:25:21): Oh, yeah, yeah, yeah, totally. You can win 1-0. Thomas LaRock (00:25:25): So the end zone is also... It's 20 yards deep. So the field's longer, it's 110 yards. But the end zone's deeper and part of it is that it's too far to kick for a field goal. But you know what? If I can punt it into the end zone and if I get a cover team down there, we can get one point out. I'm in favor of it. I think that'd be great. Jeff Sagarin (00:25:43): I think you have to kick out on the fly into the end zone. It's not like if it rolls into it. Thomas LaRock (00:25:47): No, no, no. It's like a pop flop. Jeff Sagarin (00:25:50): Yeah. Okay. Rob Collie (00:25:50): If you punt it out of the end zone, is it also a point? Thomas LaRock (00:25:52): It's a touch back. No, touch back. Jeff Sagarin (00:25:54): That'd be too easy of a way to get a point. Rob Collie (00:25:57): You've had a 20 yard deep target to land in. In Canadian fantasy football, if there was such a thing, maybe there is, punters, you actually could have punters as a position because they can score points. That would be a really sad and un-fun way to play. Rob Collie (00:26:14): But so we're amateur sports analytics people here on the show. We're not professionals. We're probably not even very good at it. But that doesn't mean that we aren't fascinated by it. We're business analytics people here for sure. Business and sports, they might share some techniques, but it's just very, very, very different, the things that are valuable in the two spaces. I mean, they're sort of spiritually linked but they're not really tools or methods that provide value. Rob Collie (00:26:39): Not that you would give them. But we're not looking for any of your secrets here today. But you're not just writing for USA Today, there's a number of places where your skills are used these days, right? Jeff Sagarin (00:26:51): Well, not as much as that. But I want to make a favorable analogy for Wayne. In the world of sports analytics, whatever the phrases are, I consider myself to be maybe an experimental applied physicist. Wayne is an advanced theoretical physicist. I do the grunt work of collecting data and doing stuff with it. But Wayne has a large over-viewing of things. He's like a theoretical physicist. Dr. Wayne Winston (00:27:17): Jeff is too modest because he's experimented for years on the best parameters for his models. Rob Collie (00:27:27): It's again that 10-year, 20-year overnight success type of thing. You've just got to keep grinding at it. Do the two of you collaborate at all? Jeff Sagarin (00:27:35): Well, we did on two things, the Hoops computer game and Win Val. I forgot. How could I forget? It was actually my favorite thing that we did even though we've made no money doing the randomization using Game Theory of play calling for football. And we based it actually and it turned out that I got great numerical results that jive with empirical stuff that Virgil Carter had gotten and our economist, named Romer, had gotten and we had more detailed results than them. Jeff Sagarin (00:28:06): But in the areas that we intersected, we had the same as them. We used a game called Pro Quarterback and we modeled it. We had actually, a fellow, I wasn't a professor but a fellow professor of Wayne's, a great guy, just a great guy named Vic Cabot, who wrote a particular routine to insert the FORTRAN program that solved that particular linear programming problem that would constantly reoccur or else we couldn't do it. That was the favorite thing and we got to show it once to Sam White, who we really liked. And White said, "I like this guy. I may have played this particular game," we told him what we based it on, "when I was a teenager." Jeff Sagarin (00:28:46): He said, "I know exactly what you want to do." You don't make the same call in the same situation all the time. You have a random, but there's an optimal mix Game Theory, as you probably know for both offense and defense. White said, "The problem is this is my first year here. It was the summer of '83." And he said, "I don't really have the security." Said, "Imagine it's third and one, we're on our own 15 yard line. And it's third and one. And the random number generator says, 'Throw the bomb on this play with a 10% chance of calling up but it'll still be in the mix. And it happens to come up.'" Jeff Sagarin (00:29:23): He said, "It was my eight year here. I used to play these games myself. I know exactly." But then he patted his hip. He said, "It's mine on the line this first year." He said, "It's kind of nerve wracking to do that when you're a rookie coach somewhere, to call the bomb when it's third and one on your own 15. If it's incomplete, you'll be booed out of the stadium." Rob Collie (00:29:46): Yeah, I mean, it's similar to there's the general reluctance in coaches for so long to go for it on fourth and one. When the analytics were very, very, very clear that this was a plus expected value, +EV, move to go for it on fourth and one. But the thing is, you've got to consider the bigger picture. Right? The incentives, the coaches number one goal is actually don't get fired. Jeff Sagarin (00:30:14): You were right. That's what White was telling us. Rob Collie (00:30:14): Yeah. Winning a Super Bowl is a great thing to do. Because it helps you not get fired. It's actually weird. Like, if your goal is to win as many games as possible, yes, go for it on fourth and one. But if your goal is to not get fired, maybe. So it takes a bit more courage even to follow the numbers. And for good reason, because the incentives aren't really aligned the way that we think they are when you first glance at a situation. Jeff Sagarin (00:30:41): Well, there's a human factor that there's no way unless you're making a guess how to take it into account. It may be demoralizing to your defense if you go for it on fourth and one and you're on your own 15. I've seen the numbers, we used to do this. It's a good mathematical move to go for it. Because you could say, "Well, if you're forced to punt, the other team is going to start on the 50. So what's so good about that? But psychologically, your defense may be kind of pissed off and demoralized when they have to come out on the field and defend from their own 15 after you've not made it and the numbers don't take that into account. Rob Collie (00:31:19): Again, it's that judgment thing. Like the coach hung out to dry. Dr. Wayne Winston (00:31:22): Can I say a word about Vic Cabot, that Jeff mentioned? Jeff Sagarin (00:31:26): Yeah, He's great. Dr. Wayne Winston (00:31:27): Yeah. So Vic was the greatest guy any of us in the business school ever knew. He was a fantastic person. He died of throat cancer in 1994, actually 27 years ago this week or last week. Jeff Sagarin (00:31:43): Last week. It was right around Labor Day. Dr. Wayne Winston (00:31:46): Right. But I want to mention, basically, when he died, his daughter was working in the NYU housing office. After he died, she wrote a little book called The Princess Diaries. She's worth how many millions of dollars now? But he never got to see it. Jeff Sagarin (00:32:06): He had a son, a big kid named Matt Cabot, who played at Bloomington South High School. I got a nice story about Matthew. I believe the last time I know of him, he was a state trooper in the state of Colorado. I used to tell him when I was still young enough and Spry enough, we'd play a little pickup or something. I'd say, "Matthew, forget about points. The most important thing, a real man gets rebounds." Jeff Sagarin (00:32:32): They played in the semi state is when it was just one class. In '88, me and Wayne and a couple of Wayne's professor buddies, we all... Of course, Vic would have been there but we didn't go in the same car. It was me, Wayne and maybe [inaudible 00:32:48] and somebody else, Wayne? Jeff Sagarin (00:32:49): They played against Chandler Thompson's great team from Muncie Central. In the first three minutes, Chris Lawson, who was the star of the team went up for his patented turn around jumper from six feet away in the lane and Chandler Thompson spiked it like a volleyball and on the run of Muncie Central player took it with no one near him and laid it in and the game essentially ended but Matt Cabot had the game of his life. Jeff Sagarin (00:33:21): I think he may have led the game of anyone, the most rebounds in the game. I compliment him. He was proud of that. And he's played, he said many a pickup game with Chandler Thompson, he said the greatest jumper he's ever been on the court within his entire life. You guys look up because I don't know if you know who Chandler Thompson. Is he played at Ball State. Look up on YouTube his put back dunk against UNLV in the 90 tournaments, the year UNLV won it at all. Look up Chandler Thompson's put back dunk. Rob Collie (00:33:52): Yeah, I was just getting into basketball then, I think. Like in the Loyola Marymount days. Yeah, Jerry Tarkanian. Does college basketball have the same amount of personalities it used to like in the coaching figures. I kind of doubt that it does. Rob Collie (00:34:06): With Tark gone, and of course, Bob Knight, it'll be hard to replace personalities like that. I don't know. I don't really watch college basketball anymore, so I wouldn't really know. But I get invited into those pick'em pools for the tournament March Madness every year and I never had the stamina to fill them out. And they offer those sheets where they'll fill it out for you. But why would I do that? Jeff Sagarin (00:34:28): I've got to tell you a story involving Wayne and I. Rob Collie (00:34:31): Okay. Jeff Sagarin (00:34:31): In the 80 tournament, I had gotten a program running that would to simulate the tournament if you fed in the power ratings. It understood who'd play who and you simulate it a zillion times, come up with the odds. So going into the tournament, we had Purdue maybe the true odds against him should have been let's say, I'll make it up seven to one. Purdue and Iowa, they had Ronnie Lester, I remember. Jeff Sagarin (00:34:57): The true odds against them should have been about 7-1. The bookmakers were giving odds of 40-1. So Wayne and I looked at each other and said, "That seems like a big edge." In theory, well, odds are still against them. Let's bet $25 apiece on both Purdue and Iowa. The two of them made the final four. Jeff Sagarin (00:35:20): In Indianapolis, I'll put it this way, their consolation game gave us no consolation. Rob Collie (00:35:30): Man. Jeff Sagarin (00:35:31): And then one of the games, Joe Barry Carroll of Purdue, they're down by one they UCLA. I'm sure he was being contested. I don't mean he was all by himself. It's always easy for the fan who can't play to mock the player. I don't mean... He was being fiercely contested by UCLA. The net result was he missed with fierce contesting one foot layup that would have won the game for Purdue, that would have put them into the championship game and Iowa could have beaten Louisville, except their best player, Ronnie Lester had to leave the game because he had aggravated a bad knee injury that he just couldn't play well on. Jeff Sagarin (00:36:11): But as I said, no consolation, right Wayne? Dr. Wayne Winston (00:36:14): Right. Jeff Sagarin (00:36:15): That was the next to the last year they ever had a consolation game. The last one was in '81 between LSU and Virginia. Rob Collie (00:36:23): Was it the '81 tournament that you said that you liked Indiana to win it? Jeff Sagarin (00:36:28): Wait, I'm going to show you how you get punished for hubris. I learned my lesson. The next year in '82, I had gotten a lot of notoriety, good kind of notoriety for having them to win in '81. People thought, "Wow! This is like the Oracle." So now as the tournament's about to begin in '82, I started getting a lot of calls, which I never used to do like from the media, "Who do you got Jeff?" I said confidently, "Oregon State." I had them number one, I think they'd only lost one game the whole year and they had a guy named Charlie Sitting, a 6'8 guy who was there all American forward. Jeff Sagarin (00:37:06): He was the star and I was pretty confident and to be honest, probably obnoxious when I'd be talking to the press. So they make the regional final against Georgetown and it was being held out west. I'm sort of confidently waiting for the game to be played and I'm sure there'll be advancing to the final four. And they were playing against freshmen, Patrick Ewing. Jeff Sagarin (00:37:29): In the first 10 seconds of the game, maybe you can find the video, there was a lob pass into Ewing, his back was to the basket, he's like three feet from the basket without even looking, he dunks backwards over his head over Charlie Sitton. And you should see the expression on Charlie Sitton's face. I said, "Oh my god! This game is over." The final score was 68-43 in Georgetown's favor. It was a massacre. It taught me the lesson, never be cocky, at least in public because you get slapped down, you get slapped down when you do that. Rob Collie (00:38:05): I don't want to get into this yet again on this show. But you should call up Nate Silver and maybe talk to him a little bit about the same sort of thing. Makes very big public calls that haven't been necessarily so great lately. Just for everyone's benefit, because even though I'd live in the state of Indiana, I didn't grow up here. Let's just be clear. Who won the NCAA tournament in 1981? Jeff Sagarin (00:38:29): Indiana. Rob Collie (00:38:30): Okay. All right, so there you go. Right. Jeff Sagarin (00:38:33): But who didn't win it in 1982? Oregon State. Rob Collie (00:38:38): Yeah. Did you see The Hunt for Red October where Jack Ryan's character, there's a point where he guesses. He says, "Ramy, as always, goes to port in the bottom half of the hour with his crazy Ivan maneuvers and he turns out to be right." And that's how he ends up getting the captain of the American sub to trust him as Jack Ryan knew this Captain so well, even knew which direction he would turn in the crazy Ivan. But it turns out he was just bluffing. He knew he needed a break and it was 50/50. Rob Collie (00:39:08): So it's a good thing that they were talking to you in the Indiana year, originally. Not the Oregon State year. That wouldn't be a good first impression. If you had to have it go one way or the other in those two years, the order in which it happened was the right order. Jeff Sagarin (00:39:22): Yeah, nobody would have listened to me. They would have said, "You got lucky." They said, "You still were terrible in the Oregon State year." Rob Collie (00:39:28): But you just pick the 10th rated team and be right. The chances of that being just luck are pretty low. I like it. That's a good story. So the two of you have never collaborated like on the Mark Cuban stuff? On the Mavs or any of that? Jeff Sagarin (00:39:43): We've done three things together. The Hoops computer game, which we did from '86-'95. And then we did the Game Theory thing for football, but we never got a client. But we did get White to kind of follow it. There's an interesting anecdote, I won't I mentioned the guy who kind of screwed it up. But he assigned a particular grad assistant to fill and we needed a matrix filled in each week with a bunch of numbers with regarding various things like turnovers. Jeff Sagarin (00:40:13): If play A is called against defense B, what would happen type of thing? The grad assistant hated doing it. And one week, he gave us numbers such that the computer came back with when Indiana had the ball, it should quick kick on first down every time it got the ball. We figured it out what was going on, the guy had given Indiana a 15% chance of a turnover, no matter what play they called in any situation against any defense. Jeff Sagarin (00:40:44): So the computer correctly surmised it were better to punt the ball. This is like playing Russian roulette with the ball. Let's just kick it away. So we ended up losing the game in real life 10-0. White told us then when we next saw him, we used to see him on Monday or Tuesday mornings, real early in the day, like seven o'clock, but that's when you could catch him. And he kind of looked at us and said, "You know what? We couldn't have done any worse said had we kicked [inaudible 00:41:14]." Rob Collie (00:41:13): That's nice. Jeff Sagarin (00:41:14): And then we did Mark Cuban. That was the last thing. We did that with Cuban from basically 2000-2011 with a couple of random projects in the summer for him, but really on a day to day basis during a season from 2000-2011. Rob Collie (00:41:30): And during that era is when I met Wayne at Microsoft. That was very much an active, ongoing project when Wayne was there in Redmond a couple of times that we crossed paths. Dr. Wayne Winston (00:41:43): And we worked for the Knicks one year, and they won 54 games. Jeff Sagarin (00:41:47): Here with Glen Grunwald. So they won more games than they'd ever won in a whole bunch of years. And like three weeks before the season starts or so in mid September, the next fire, Glen Grunwald. Let's put it this way, it didn't bother us that the Knicks never made the playoffs again until this past season. Rob Collie (00:42:10): That's great. You were doing, was it lineup optimization for those teams? Jeff Sagarin (00:42:15): Wayne knows more about this than I do. Because I would create the raw data, well, I call it output, but it needed refinement. That was Wayne's department. So you do all the talking now, Wayne. Dr. Wayne Winston (00:42:26): Yeah. Jeff wrote an amazing FORTRAN program. So basically, Jeff rated teams and we figured out we could rate players based on how the score of the game moved during the game. We could evaluate lineups and figure out head to head how certain players did against each other. Now, every team does this stuff and ESPN has Real Plus-Minus and Nate Silver has Raptor. But we started this. Jeff Sagarin (00:42:58): I mean, everybody years ago knew about Plus-Minus. Well, intuitively, let's say you're a gym rat, you first come to a gym, you don't know anyone there and you start getting in the crowd of guys that show up every afternoon to play pickup. You start sensing, you don't even have to know their names. Hey, when that guy is on the court, no matter who his teammates are, they seem to win. Jeff Sagarin (00:43:20): Or when this guy's on the court, they always seem to lose. Intuitively since it matters, who's on the court with you and who your opponents are. Like to make an example for Rob, let's say you happen to be in a pickup game. You've snuck into Pauley Pavilion during the summer and you end up with like four NBA current playing professionals on your team and let's say an aging Michael Jordan now shows up. He ends up with four guys who are graduate students in philosophy because they have to exercise. You're going to have a better plus-minus than Michael Jordan. But when you take into account who your teammates were and who's his were, if you knew enough about the players, he'd have a better rating than you, new Michael Jordan would. Jeff Sagarin (00:44:08): But you'd have a better raw plus-minus than he would. You have to know who the people on the court were. That was Wayne's insight. Tell them how it all started, how you met ran into Mark Cuban, Wayne, when you were in Dallas? Dr. Wayne Winston (00:44:20): Well, Mark was in my class in 1981, statistics class and I guess the year 1999, we went to a Pacers Maverick game in Dallas. Jeff Sagarin (00:44:31): March of 2000. Dr. Wayne Winston (00:44:33): March of 2000, because our son really liked the Pacers. Mark saw me in the stands. He said, "I remember you from class and I remember you for being on Jeopardy." He had just bought the team. And he said, "If you can do anything to help the Mavericks, let me know." And then I was swimming in the pool one day and I said, "If Jeff rates teams, we should rate players." And so we worked on this and Jeff wrote this amazing FORTRAN program, which I'm sure he could not rewrite today. Jeff Sagarin (00:45:04): Oh, God. Well, I was motivated then. Willingness to work hard for many hours at a time, for days at a time to get something to work when you could use the money that would result from it. I don't have that in me anymore. I'm amazed when I look at the source code. I say, "Man, I couldn't do that now." I like to think I could. Necessity is the mother of invention. Rob Collie (00:45:28): I've many, many, many times said and this is still true to this day, like a previous version of me that made something amazing like built a model or something like that, I look back and go, "Whoo, I was really smart back then." Well, at the same time I know I'm improving. I know that I'm more capable today than I was a year ago. Even just accrued wisdom makes a big difference. When you really get lasered in on something and are very, very focused on it, you're suddenly able to execute at just a higher level than what you're typically used to. Jeff Sagarin (00:46:01): As time went on, we realized what Cuban wanted and other teams like the next would want. Nobody really wanted to wade through the monster set of files that the FORTRAN would create. I call that the raw output that nobody wanted to read, but it was needed. Wayne wrote these amazing routines in Excel that became understandable and usable by the clients. Jeff Sagarin (00:46:26): The way Wayne wrote the Excel, they could basically say, "Tell us what happens when these three guys are in the lineup, but these two guys are not in the lineup." It was amazing the stuff that he wrote. Wayne doesn't give himself the credit that otherwise after a while, nobody would have wanted what we were doing because what I did was this sort of monstrous and to some extent boring. Dr. Wayne Winston (00:46:48): This is what Rob's company does basically. They try and distill data into understandable form that basically helps the company make decisions. Rob Collie (00:46:58): It is a heck of a discipline, right? Because if you have the technical and sort of mental skills to execute on something that's that complex, and it starts down in the weeds and just raw inputs, it's actually really, really, really easy to hand it off in a form that isn't yet quite actionable for the intended audience. It's really fascinating to you, the person that created it. Rob Collie (00:47:23): It's not digestible or actionable yet for the consumer crowd, whoever the target consumer is. I've been there. I've handed off a lot of things back in the day and said, "The professional equivalent of..." And it turned out to not be... It turned out to be, "Go back and actually make it useful, Rob." So I'm familiar with that. For sure. I think I've gotten better at that over the years. As a journey, you're never really complete with. Something I wanted to throw in here before I forget, which is, Jeff, you have an amazing command of certain dates. Dr. Wayne Winston (00:47:56): Oh, yeah. Jeff Sagarin (00:47:57): Give me some date that you know the answer about what day of the week it was, and I'll tell you, but I'll tell you how I did it. Rob Collie (00:48:04): Okay, how about June 6, 1974? Jeff Sagarin (00:48:08): That'd be a Thursday. Rob Collie (00:48:10): Holy cow. Okay. How do you do that? Jeff Sagarin (00:48:11): June 11th of 1974 would be a Tuesday, so five days earlier would be a Thursday. Rob Collie (00:48:19): How do you know June 11? Jeff Sagarin (00:48:19): I just do. Dr. Wayne Winston (00:48:23): It's his birthday. Rob Collie (00:48:24): No, it's not. He wasn't born in '74. Dr. Wayne Winston (00:48:27): No, but June 11th. Jeff Sagarin (00:48:29): I happen to know that June 11 was a Tuesday in 1974, that's all. Rob Collie (00:48:34): I'm still sitting here waiting what passes for an explanation. Is one coming? Jeff Sagarin (00:48:39): I'll tell you another way I could have done it, but I didn't. In 1963, John Kennedy gave his famous speech in Berlin, Ich bin ein Berliner, on Wednesday, June 26th. That means that three weeks earlier was June 5, the Wednesday. So Thursday would have been June 6th. You're going to say, "Well, why is that relevant?" Well, 1963 is congruent to 1974 days of the week was. Rob Collie (00:49:07): Okay. This is really, really impressive. Jeff, you seem so normal up until now. Thomas LaRock (00:49:16): You want throw him off? Just ask for any date before 1759? Jeff Sagarin (00:49:20): No, I can do that. It'll take me a little longer though. Thomas LaRock (00:49:22): Because once they switch from Gregorian- Jeff Sagarin (00:49:25): No, well, I'll give it a Gregorian style, all right. I'm assuming that it's a Gregorian date. The calendar totally, totally repeats every possible cycle every 400 years. For example, if you happen to say, "What was September 10, of 1621?" I would quickly say, "It's a Friday." Because 1621 is exactly the same as 2021 says. Rob Collie (00:49:52): Does this translate into other domains as well? Do you have sort of other things that you can sort of get this quick, intuitive mastery over or is it very, very specific to this date arithmetic? Jeff Sagarin (00:50:02): Probably specific. In other words, I think Wayne's a bit quicker than me. I'm certain does mental arithmetic stuff, but to put everybody in their place, I don't think you ever met him, Wayne. Remember the soccer player, John Swan? Dr. Wayne Winston (00:50:14): Yeah. Jeff Sagarin (00:50:15): He had a friend from high school, they went to Brownsburg High School. I forgot the kid's name. He was like a regular student at IU. He was not a well scholar, but he was a smart kid. I'd say he was slightly faster than me at most mental arithmetic things. So you should never get cocky and think that other people, "Oh, they don't have the pedigree." Some people are really good at stuff you don't expect them to be good at, really good. This kid was really good. Rob Collie (00:50:45): As humans, we need to hyper simplify things in order to have a mental model we can use to navigate a very, very complicated world. That's a bit of a strength. But it's also a weakness in many ways. We tend to try to reduce intelligence down to this single linear number line, when it's really like a vast multi dimensional coordinate space. There are so many dimensions of intelligence. Rob Collie (00:51:11): I grew up with the trope in my head that athletes weren't very bright. Until the first time that I had to try to run a pick and roll versus pick and pop. I discovered that my brain has a clock speed that's too slow to run the pick and roll versus pick and pop. It's not that I'm not smart enough to know if this, than that. I can't process it fast enough to react. You look at like an NFL receiver or an NFL linebacker or whatever, has to process on every single snap. Rob Collie (00:51:45): It's amazing how much information they have the processor. Set aside the physical skill that they have, which I also don't have and never did. On top of that, I don't have the brain at all to do these sorts of things. It's crazy. Jeff Sagarin (00:52:00): With the first few years, I was in Bloomington from, let's say, '77 to '81, I needed the money, so I tutored for the athletic department. They tutored math. And I remember once I was given an assignment, it was a defensive end, real nice kid. He was having trouble with the kind of math we would find really easy. But you could tell he had a mental block. These guys had had bad experiences and they just, "I can't do this. I can't do this." Jeff Sagarin (00:52:25): I asked this defensive end, "Tell me what happens when the ball snap, what do you have to do?" I said, "In real time, you're being physically pulverized, the other guy's putting a forearm or more right into your face. And your brain has to be checking about five different things going on in the backfield, other linemen." I said, "What you're doing with somebody else trying to hurt you physically is much more intellectually difficult, at least to my mind than this problem in the book in front of you and the book is not punching you in the face." Jeff Sagarin (00:52:57): He relaxed and he can do the problems in the room. I'd make sure. I picked not a problem that I had solved. I'd give him another one that I hadn't solved and he could do it. I realized, my God, what these guys they're doing takes actually very quick reacting brainpower and my own personal experience in elementary school, let's say in sixth grade after school, we'd be playing street football, just touch football. When I'd be quarterback, I'd start running towards the line of scrimmage. Jeff Sagarin (00:53:26): If the other team came after me, they'd leave a receiver wide open. I said, "This is easy." So I throw for touchdown. Well, in seventh grade, we go to junior high. We have squads in gym class, and on a particular day, I got to be quarterback. Now, instead of guys sort of leisurely counting one Mississippi, two Mississippi, they are pouring in. It's not that you're going to get hurt, but you're going to get tagged and the play would be over. It says touch football, and I'd be frantically looking for receivers to get open. Let's just say it was not a good experience. I realized there's a lot more to be in quarterback than playing in the street. It's so simple. Jeff Sagarin (00:54:08): They come after you and they leave the receivers wide open. That's what evidently sets apart. Let's say the Tom Brady's from the guys who don't even make it after one year in the NFL. If you gave them a contest throwing the ball, seeing who could throw it through a tire at 50 yards, maybe the young kid is better than Tom Brady but his brain can't process what's happening on the field fast enough. Thomas LaRock (00:54:32): As someone who likes to you know, test things thoroughly, that student of yours who was having trouble on the test, you said the book wasn't hitting him physically. Did you try possibly? Jeff Sagarin (00:54:45): I should have shoved it in his face. Thomas LaRock (00:54:49): Physically, just [crosstalk 00:54:50]. Rob Collie (00:54:50): Just throw things at him. Yeah. Thomas LaRock (00:54:52): Throw an eraser, a piece of chalk. Just something. Jeff Sagarin (00:54:56): I'll tell you now, I don't want to name him. He's a real nice guy. I'll tell you a funny anecdote about him. I had hurt my knuckle in a pickup basketball game. I had a cast on it and I was talking to my friend. And he had just missed making a pro football team the previous summer and he was on the last cut. He'd made it to the final four guys. Jeff Sagarin (00:55:18): He was trying to become a linebacker I think. They told him, "You're just not mean enough." That was in my mind. I thought, "Well, I don't know about that." He said, "Yeah, I had the same kind of fractured knuckle you got." I said, "How'd you get it?" "Pick up [inaudible 00:55:32]. Punching a guy in the face." But he wasn't mean enough for the NFL. And I heard a story from a friend of mine who I witnessed it, this guy was at one point working security at a local holiday inn that would have these dances. Jeff Sagarin (00:55:47): There was some guy who was like from the Hells Angels who was causing trouble. He's a big guy, 6'5, 300 whatever. And he actually got into an argument with my friend who was the security guy. Angel guy throws a punch at this guy who's not mean enough for the NFL. With one punch the Jeff Sagarin tutoree knocked the Hell's Angels guy flat unconscious. He was a comatose on the floor. But he wasn't mean enough for the NFL. Rob Collie (00:56:17): Tom if I told my plus minus story about my 1992 dream team on this show, I think maybe I have. I don't remember. Thomas LaRock (00:56:24): You might have but this seems like a perfect episode for that. Rob Collie (00:56:27): I think Jeff and Wayne, if I have told it before, it was probably with Wayne. Dr. Wayne Winston (00:56:31): I don't remember. Rob Collie (00:56:32): Perfect. It'll be new to everyone that matters. Tom remembers. So, in 1992, the Orlando Magic were a recent expansion team in the NBA. Sometime in that summer, the same summer where the 1992 Dream Team Olympic team went and dominated, there was a friend of our family who ran a like a luxury automotive accessories store downtown and he basically hit the jackpot. He'd been there forever. There was like right next to like the magic practice facility. Rob Collie (00:57:09): And so all the magic players started frequenting his shop. That was where they tricked out all their cars and added all the... So his business was just booming as a result of magic coming to town. I don't know this guy ever had ever been necessarily terribly athletic at any point in his life. He had this bright idea to assemble a YMCA team that would play in the local YMCA league in Orlando, the city league. Rob Collie (00:57:35): He had secured the commitment of multiple magic players to be on our team as well as like Jack Givens, who was the radio commentator for The Magic and had been a longtime NBA star with his loaded team. And then it was like, this guy, we'll call this guy Bill. It's not his real name. So it was Bill and the NBA players and me and my dad, a couple of younger guys that actually I didn't know, but were pretty good but they weren't even like college level players. Rob Collie (00:58:07): And so we signed up for the A league, the most competitive league that Orlando had to offer. And then none of the NBA players ever showed up. I said never, but they did show up one time. But we were getting blown out. Some of the people who were playing against us were clearly ex college players. We couldn't even get the ball across half court. Jeff Sagarin (00:58:33): Wayne, does this sound familiar to you? Dr. Wayne Winston (00:58:35): Yes, tell this story. Jeff Sagarin (00:58:38): Wayne, when he was a grad student at Yale, and I'm living in the White Irish neighborhood called Dorchester in Boston, I was young and spry. At that time, I would think I could play. Wayne as a grad student at Yale had entered a team with a really intimidating name of administration science in the New Haven City League, which was played I believe at Hill House high school at night. So Wayne said, "Hey Jeff, why don't you take a Greyhound bus down. We're going to play against this team called the New Haven All Stars. It ought to be interesting." Rob Collie (00:59:14): Wayne's voice in that story sound a little bit like the guy at USA Today for a moment. It was the same voice, the cigar chomping. Anyway, continue. Jeff Sagarin (00:59:25): They edged this out 75-31. I thought I was lined up against the guy... I thought it was Paul Silas who was may be sort of having a bus man's holiday playing for the New Haven all-stars. So a couple weeks later, Paul Silas was my favorite player on the Celtics. He could rebound, that's all I could do. I was pitiful at anything else. But I worked at that and I was pretty strong and I worked at jumping, etc. Jeff Sagarin (00:59:53): So a few weeks later, Wayne calls me up and says, "Hey Jeff, we're playing the New Haven All-Stars again. Why don't you come down again and we'll get revenge against them this time?" Let's just say it didn't work out that way. And I remember one time I had Paul Silas completely boxed out. It was perfect textbook and I could jump. If my hands were maybe at rim level and I could see a pair of pants a foot over mine from behind, he didn't tell me and he got the rebound and I'm at rim level. Jeff Sagarin (01:00:24): We were edged out by a score so monstrous, I won't repeat it here. I'm not a guard at all. But I ended up with the ball... They full court pressed the whole game. Rob Collie (01:00:34): Of course, once they figure out- Jeff Sagarin (01:00:36): That we can't play and I'm not even a guard. It was ludicrous. My four teammates left me in terror. They just said, "We're going down court." So I'm all alone, they have four guys on me and my computer like my thought, "Well, they've got four guys on me. That must mean my four teammates are being guarded by one guy down court. This should be easy." I look, I look. They didn't steal the ball out of my hands or nothing. I'm still holding on to it. They're pecking away but they didn't foul me. I give them credit for that. I was like, "Where the hell are my teammates?" Jeff Sagarin (01:01:08): They were in terror hiding in single file behind the one guy and I basically... I don't care if you bleeping or not, I said, "Fuck it." And I just threw the ball. Good two overhand pass, long pass. I had my four teammates down there and they had one guy and you can guess who got the ball. After the game I asked them, I said, "You guys seem fairly good. Are you anybody?" The guy said, "Yeah, we're the former Fairfield varsity we were in the NIT about two years ago." Jeff Sagarin (01:01:39): I looked it up once. Fairfield did make the NIT, I think in '72. And this took place in like February of '74. It taught me a lesson because I looked up what my computer rating for Fairfield would have been compared that to, let's say, UCLA and NC State and figured at a minimum, we'd be at least a 100-200 point underdog against them in a real game, but it would have been worse because we would never get the ball pass mid-court. Rob Collie (01:02:10): Yeah, I mean, those games that I'm talking about in that YMCA League, I mean, the scores were far worse. We were losing like 130-11. Jeff Sagarin (01:02:19): Hey, good that's worse than New Haven all-stars beat us but not quite that bad. Rob Collie (01:02:24): I remember one time actually managing to get the ball across half court and pulling up for a three-point shot off of the break. And then having the guy that had assembled the team, take me aside at the next time out and tell me that I needed to pass that. I'm just like, "No. You got us into this embarrassment. If I get to the point where like, there's actually a shot we can take like a shot, we could take a shot. I'm not going to dump it off to you." Thomas LaRock (01:02:57): Not just a shot, but the shot of gold. Rob Collie (01:03:00): The one time we did get those guys to show up, we were still kind of losing because those guys didn't want to get hurt. It didn't make any sense for them to be there. There was no upside for them to be in this game. I'm sure that they just sort of been guilted into showing up. But then this Christian Laettner lookalike on the other team. He was as big as Laettner. Rob Collie (01:03:25): This is the kind of teams we were playing against. There was a long rebound and that Laettner lookalike got that long rebound and basically launched from the free throw line and dunked over Terry Catledge, the power forward for the Magic at the time. And at that moment, Terry Catledge scored the next 45 points in the game himself. That was all it was. Rob Collie (01:03:50): He'd just be standing there waiting for me to inbound the ball to him, he would take it coast to coast and score. He'd backpedal on defense and he would somehow steal the ball and he'd go down and score again. He just sent a message. And if that guy hadn't dunked over Catledge, we would have never seen what Catledge was capable of. So remember, this is a team that's blowing us out 132-11 on an average basis. And then Catledge, there's more distance between Catledge who wasn't even like an NBA star, he was just like a replacement level player in the NBA. The distance between him and those guys we were playing against was even larger than the distance between these YMCA stars and us it was unreal. A force of nature. Jeff Sagarin (01:04:35): I got a Wilt Chamberlain story I've read about like that. I've read that... You've heard of the [inaudible 01:04:40] tournament. This is way before it became, how would I put it? Public and sheek and everything this was back in the late '50s, maybe early '60s. Connie Hawkins was a high school legend who graduated high school at the time in '60. Jeff Sagarin (01:04:58): It was on the playground in that '59 maybe the '61 Chamberlain was already Wilt Chamberlain. There was a guy on the other team named Jumpin Jackie Jackson who was a legend on the playgrounds who could really jump. He happened at the time of Chamberlain fadeaway jump shot and blocked it in midair. Chamberlain was so infuriated that the next 20 possessions, he just had the ball thrown into him and he would dunk very viciously and nobody even wanted to try to block it. He was sending the same message Catledge sent on that. Jeff Sagarin (01:05:35): I've heard the same analogy. Wilt Chamberlain when he was about 50, he would play in pickup games. I think Magic Johnson told this story. At age 50 Chamberlain would be playing at quality pavilion is where all the guys would hang out, Magic Johnson, a bunch of the Lakers and Chamberlain. Chamberlain at age 50, he was pissed off somebody made a bad call. He said, "You fouled me on that Will or something?" Wilt would get enraged and suddenly start dunking where nobody wanted to even get in his way. Rob Collie (01:06:07): Jeff, again, we're not asking for any secret sauce. But I was going to ask you about in a vague sense, sort of like methodologies that you found to be useful. Thomas LaRock (01:06:16): So I was just going to say, Rob, I had an idea along what you were just saying about the myths and things like that. I was wondering if there's some type of common statistic that a lot of people just take for granted, but it's something that actually should be just ignored. Along the lines of Jeff and Wayne's experience, is there something out there that people kind of rely on but it's actually basis? Dr. Wayne Winston (01:06:40): Well, I'll take that. In baseball, batting average is worthless and games won is worthless. But those are pretty well known. Rob Collie (01:06:51): What about in the methodology department itself? Things that might be applicable in general outside of sports? Jeff Sagarin (01:06:58): Well, I guess this actually... There's a lot of analogies as to real life. A famous example in sports is the Pittsburgh Pirates beating the Yankees in 1960, four games to three. You look up the scores, the Yankees out-scored them 55 runs to 27. Yet Pittsburgh won four of the seven games, the famous Mazeroski home run. Jeff Sagarin (01:07:20): Well, in college sports, often it's more like college to pros, nobody cares about ratings because it's decided by wins and losses, which is kind of like what the Pirates did. But I once asked is that there was a restaurant in town where one of the guys who ran it was kind of connected in sports. And a lot of times there'd be some assistant coaches, coaches would meet in the back office and we'd all talk over stuff. I once asked them, and I got a 50/50 distribution on the answers. Jeff Sagarin (01:07:49): I created a hypothetical league like a 10 team League, where one team won all nine of its games against the other nine teams in it by one point the game. The other team in this discussion, won eight of its games by 50 points and lost of course by one point to the team that won every game they had played by one point. I said, "Who's the better team?" Half of the guys in the room who were actually professionals, I mean, they were coaches, half of them said the team that was undefeated and the other half said, that was a fluke that that other team lost. The team that was winning by 50 a game against the same opponents the other team played, they're clearly the better team and it's just a fluke that they lost. Jeff Sagarin (01:08:35): So when you're rating teams, you run into that problem, sometimes in real life. Some teams, as a cliche phrases, a friend of mine made this phrase up about this one team that was undefeated. It was Nebraska, I remember in 1972, when they were trying to repeat as national champions, where they've been undefeated in '71. They managed to lose two games and tie one. And every game they lost was by like this, really close. And then they had a streak in the middle of the season where they outscored their opponents 201-0. You could look that up. Jeff Sagarin (01:09:12): The '72 Nebraska team. And so this friend of mine said, "Nebraska, they lost any game that they had an opportunity to lose." Most games, they had no opportunity to lose it. They were the snack good and they never had experience till they lost. They didn't know what to do when the game was close against somebody that could play with them. Rob Collie (01:09:32): Okay, what's the right answer to the hypothetical 10 team league experiment? Jeff Sagarin (01:09:37): It's dependent on the parameters you use. If you emphasize winning to the computer, you'll get the team that's undefeated. If you just say scores are all the matter, you'll get the team that's winning by 50 points a game and loses the one game by one. Rob Collie (01:09:51): Okay, so you can definitely tilt the model toward one conclusion? Jeff Sagarin (01:09:55): Yeah, you try to find out which works empirically the best predicting the most accuracy over a large sample of games. You'll test different parameter values to see which works, but it may not work and obviously in any one particular game better than the other method. And then when you're wrong, people say, "Oh, you're stupid. You should have been doing it this way." Jeff Sagarin (01:10:16): And little did they know when you've tested it over a sample of 10,000 games, that the methods there, "stupid", for using is the more accurate method. Rob Collie (01:10:25): But which one's the right answer? Jeff Sagarin (01:10:28): I lean towards scores by the way. Rob Collie (01:10:29): Yeah, I figured. I'm on that side, the team that won by 50 and then lost by one. I'm going to pick that team. Thomas LaRock (01:10:36): I think you need more features. I think you need to know like was somebody hurt, where the game is being played. Jeff Sagarin (01:10:40): Well, of course you know where the games are played. That's always built in, I know that. But you start saying, "Well, who had the flu? Who had a sprained ankle?" You can at that point, start putting in factors that make whatever team you want be the favorite team. Rob Collie (01:10:55): Bring more and more of the human bias into it, yeah. I can see that. Do you still do any work with the NCAA or is that behind you at this point? Jeff Sagarin (01:11:03): They can get my stuff online. But as a courtesy, just to keep my streak alive with them, I've been giving them my numbers for free since the spring of '84, technically winter of '84. I started with a real nice guy named Vic Bulbous. He used to be the coach at Duke back in the '60s and then he eventually ended as... With the NCAA, he was head of the tournament committee. In the early mid '80s, I've done that every, say March, commencing in 1984 but that's because he called me up and said Jim van Valkenburg. Jeff Sagarin (01:11:39): Remember, I mentioned way at the beginning at the NCAA that I had talked to in 1981. Van Valkenburg told him to call me saying, "Hey, he had Indiana to win in '81." That's kind of all connected. Rob Collie (01:11:52): One of the problems that basically all data professionals run into is when they try to get buy-in for their methodology and the value that they can bring to the table. They try to get non-data people to understand and/or believe in it. Are there any lessons learned from working with the NCAA over the years that might be relevant? Jeff Sagarin (01:12:14): They wanted to talk to me directly in October of 1988 during the Dodger Oakland A's World Series and then I said, "We want to talk to you regarding our RPI and the difference with your system, etc." So I said, "Okay, we can talk." And they said, "We want to talk to you in person." I said, "Man, look, I'll talk to you for free over the phone. But I don't like to travel. And if you're going to make me travel to Kansas City, I want to get paid. But I'm not asking to be paid. Why can't we just do this on a speakerphone? I'm not interested in making money out of this. I just don't like traveling." Jeff Sagarin (01:12:52): So they said basically, in a nice way, "We insist. We want you here in person to meet with the NCAA Tournament committee." So I said, "What's the most you pay anybody? I want to get paid that." So they paid me, I think it was $500. And I said, "And I need another $100 to pay for a friend of mine to drive me up to the airport early like at 5:00 am in Indianapolis, and then pick me up later that night at 11 o'clock at night." Jeff Sagarin (01:13:18): So they paid me two separate checks, 500 and 100. So I paid my friend 100 bucks. I think we stopped at a bar on the way home to watch the Dodgers beat Oakland that night. Okay. That's how I remember that. What they told me was they said, "Look, we're the NCAA. RRPI does not use scores, because we can't, in good conscience, tell the teams, 'We're going to use scores,' because then we're sort of looking like we're encouraging teams to run up the score. But we don't want you to change your system to do that. Because we all know as ex coaches and athletic directors that scores tell you who the better team is." Jeff Sagarin (01:13:58): They know that. And these are true professionals, former coaches and athletic directors. They're not some philosopher kings, they know that 50 point the game team, it's a fluke that they lost by one. That was their personal opinion. And so they said, "Don't change what you do. We don't want to have to officially acknowledge that though." Jeff Sagarin (01:14:18): So ironically, their new system that they talked about in January 20th of 2017, a Friday, they said, "Yeah, we're going to start using scores." But they put a cap on it. I think that's so any victory greater than 10 points the computer collapses to be 10, it produces some inaccuracies, but it's okay and at least it differentiates teams better. I don't know how to duplicate that. And they have some other fancy things in it. But that's the basic underpinning is all victories are by no more than 10. I don't know what they do with the home edge. But that's not that important to me, but they acknowledged you need to use scores. Rob Collie (01:15:00): Were you at all involved or did you have a ringside seat at all for the BCS era of college football? Jeff Sagarin (01:15:07): I had more than a ring. I was in one of the guys in the ring. Rob Collie (01:15:10): They kept talking about the computer rankings going into the BCS. Was that you? Jeff Sagarin (01:15:14): Yeah, I was one of the... Actually, I was with them every year they did it. I think it was 16 years. I was with it all the years. The way I'd put it would be it really wasn't good for me to be in it. It would have been worse for me given that they had it, to not be in it. People say, "He must not be good enough to be in it." But all it did was bring you brief from fans. Jeff Sagarin (01:15:36): I have a fun story. The very, very, very first year at Tennessee, I didn't have them number one. People in Tennessee were real unhappy with me and I was young and kind of stronger then. There was a friend of mine who was a bouncer at this local bar who was actually a native of Knoxville, but he was here. He's a real big guy, one of those 6'4, 350 pound guys. We had a fun, I call it rivalry. Whenever we see each other, we have some laughs today. Jeff Sagarin (01:16:04): We were in fact in a mutual one of these pick'em contests for the tournament this past March. At the time I asked him, I said, "How would I do if I were in a sports bar right now in Knoxville, Tennessee?" And he laughed and said, "You'd need me with you." You get some weirdo fans out there. Very weirdo fans. Rob Collie (01:16:26): The NCAA Tournament, it's a lot more, I don't know, almost like continuous of a problem. It's so many teams, the best teams in the country are going to be near the top. If you give someone a two seed when they think they deserved a one, it's not that big of a slight. Compared to you're in the BCS title game or you're not. Jeff Sagarin (01:16:49): Correct. Rob Collie (01:16:49): To add to that, I think that the football fan base is a bit more vicious than the basketball fan base. Jeff Sagarin (01:16:57): Yeah, I'd say you're right. Rob Collie (01:16:59): We had Michael Salfino, who's a sports writer for The Athletic and FiveThirtyEight and Wall Street Journal. He was a previous guest. And one of the things he was talking about was how, this is what Tom brought up is that analytics are changing sports too. They're not just predicting and helping people manage the game, but they're just completely changing the way the game is played. Like the NBA is famously now got a whole desert. The mid range jumper is gone from the NBA. Baseball has become all about nothing but home runs and strikeouts. Jeff Sagarin (01:17:32): It's sort of like saying the three-point shot is the home run and if you miss it, it's kind of like a strikeout. Rob Collie (01:17:38): Yeah, yeah. And so in those two sports in particular, there is a rising, especially baseball, I think. Really, baseball wasn't a sport that could afford to become more boring. It didn't have room for that. But is it making those sports less interesting to watch? I don't think it's making football less interesting to watch, at least not so far. Football, to me is still a peak, entertaining product. Jeff Sagarin (01:18:05): In what ways? I don't know the answer, because I'm not involved in any of this stuff. I understand about baseball and basketball, the three-point shot and "home runs", how is it changing football? Maybe teams go for it more on fourth down. But that's not really changing the physical play in baseball. It changes how batters swing, their upper cutting, trying to hit a home run and then basketball, they're taking more three point shots. What are they doing different in football? Thomas LaRock (01:18:31): I'll tell you. So here's a great example of running quarterback. I grew up your quarterback, he took five or seven steps back and that was it and he was in the pocket. And these days, you got guys who were just mobile. I think once you have one quarterback or two, have some success and you had some data to show, "Hey, this changes the nature of the game a little bit. Now, teams start to say, "Well, I'm going to imitate that success." My question was more about have you seen or felt that some of the statistics or things that you've uncovered has eventually helped shape the way sports get played? Jeff Sagarin (01:19:06): I've never been involved in doing statistics from football. Wayne and I did the game theory thing, trying to say how you should randomly intelligently optimize your play calling mix. But it's not like I analyzed running averages and passing yards per pass attempt. Jeff Sagarin (01:19:24): One thing I will say is when people compare they have the quarterback rating that the NFL has and it's just a simple algebra formula. I've forgotten it. But I have it. It's not that complicated. They'll say, "My God, guys like Johnny Unitas are a fraud. He only had a rating of let's say 80. And now the worst passer in the league has a rating of 80. And that makes them the worst guy in the league." Jeff Sagarin (01:19:49): And I'll say why don't you compare guys to the league average when they played, when Unitas played league average in the way they do that NFL rating was like in the high 50s or early 60s. So being in 80 meant you were good. Now, league average I believe is in the 90s. Yeah, being an 80 now is below average, but they've changed the rules. Imagine let's say in basketball, let's say the basket, like the physical dimensions of a game define who the better players are going to be. Let's say in basketball, the basket were like oil barrels that came to about waist tie. Jeff Sagarin (01:20:28): Well, maybe the best place would be these short, stocky, muscular construction workers who would dominate, play around the oil barrel. Being 7'2 wouldn't really be an advantage. Or what if the basket were like 20 feet high? Who would be the good shooters? Well, when Unitas was quarterback, offensive linemen were not allowed to hold. Defensive linemen were not penalized unless it was amazingly egregious for roughing the quarterback. And defensive backs could bump and run the entire route. Jeff Sagarin (01:21:02): Now, offensive lineman can hold. Defensive backs can only touch the receiver in the first five yards. The defensive linemen are severely penalized if they hurt the quarterback in any way. It's not really fair to judge guys like Unitas on today's numerical standards. Unitas would have been delighted to play if nobody was allowed to try to injure them. Rob Collie (01:21:27): You have to index to error, right? Jeff Sagarin (01:21:29): I'm a Johnny Unitas fan. That's why I kind of get upset. All these numbers today guys throwing for huge amounts of yardage. Let's see how they would do if their linemen weren't allowed to hold and if defensive backs bump and run the entire length of the pass route. Rob Collie (01:21:44): I had similar problems with like Titanic being constantly touted as the highest grossing film of all time forever. And I'm going, "What about inflation?" And I went through great lengths to prove that Star Wars was actually still number one. Dr. Wayne Winston (01:22:02): Gone with the Wind. Jeff Sagarin (01:22:02): Exactly, 1939. Rob Collie (01:22:02): Especially when you index for total population of the country, Gone with the Wind is a runaway winner. And I wasn't as pleased with that conclusion as I was with the original Star Wars conclusion, for obvious reasons. Jeff Sagarin (01:22:15): You were born in... So you were three years old when Star Wars came out? Rob Collie (01:22:18): That's right. That's right. Jeff Sagarin (01:22:20): And you were -35 when Gone with the Wind came out. Rob Collie (01:22:24): That's correct. Yes. Yeah. Jeff Sagarin (01:22:26): But Wayne will remember this. The theme music from Gone with the Wind went (singing). That was the theme music for Million Dollar Movie on Channel Nine in New York City. Dr. Wayne Winston (01:22:37): Yeah. Rob Collie (01:22:38): So back to the football thing for a moment. The rule changes have made a big difference. But even after the rule changes were made, it took a long time, I think for the style of play to react to the extent that it has today. The idea of having a strong running game in the NFL, the vast majority of its appeal has been lost. Jeff Sagarin (01:23:04): I think the reason for that is you can pass easily now because your quarterback is not allowed to be physically attacked and your offensive lineman can hold and you can't be touching the receiver his whole route. If you put those rules back in, teams would want to run the ball again. Rob Collie (01:23:19): Even then, it just took a long time for the NFL to come around to that conclusion because the rule changes, especially the way that you're allowed to interact with the receivers, those are now really old. And yet still, like in the '90s and 2000s, we had this obsession. It is very clearly the wrong move to ever pay a running back on his second contract. Take the rookie contract and as soon as they're off the rookie contract, go get another rookie. It's crazy. Jeff Sagarin (01:23:49): I had a comment to make on Tom's remark about running quarterbacks. He's correct, but there's a problem for each given running quarterback. Remember Michael Vic? Nobody could stop them. Some linemen eventually got to him. They wrecked his knee, he got a broken leg and when he came back, he was no longer Michael Vick. He suddenly could not elude people. Jeff Sagarin (01:24:11): Before he got hurt, I thought they'll never lose a game with this guy. He'll score every time they have the ball. But I guess NFL defensive players don't like a guy to be that good. They start when they get a chance to tackle him, after a while he doesn't stay that good. Thomas LaRock (01:24:26): The running quarterbacks of today are quite a bit more solidly constructed than Michael Vic was. I guess Kyler Murray is the exception. Kyler Murray is not a big guy. He's a Mike Vick type runner. He runs a lot. I think he's really good at avoiding the hits. Jeff Sagarin (01:24:44): Yeah, the guys that you're talking about, I'm going to guess the bigger running quarterbacks, they're not as elusive as Michael Vick. He's the most elusive NFL quarterback I've ever seen. But you don't stay elusive once you start being tackled a lot. I'm not trying to be funny. He was an amazing guy to watch. Rob Collie (01:25:03): The hits really took their toll on him. Jeff, really appreciate you taking the time both of you. Jeff Sagarin (01:25:09): I'm going to put in a commercial for Wayne and Wayne can correct a number. I believe Wayne was selected as Teacher of the Year in the business school a minimum of five times. Am I correct, Wayne? Dr. Wayne Winston (01:25:20): I won the NBA Teaching Award six times and went over 40 teaching awards. Jeff Sagarin (01:25:26): Yeah, he's the best teacher they ever had. Dr. Wayne Winston (01:25:28): Well, maybe. Jeff Sagarin (01:25:30): That's my vote. If there was a BCS, my ratings would have waned number one. Rob Collie (01:25:35): The margin of victory in these votes is important. Thomas LaRock (01:25:41): Yeah, it's not just the win. How much- Jeff Sagarin (01:25:44): Decisively. Rob Collie (01:25:46): He was a runaway first ballot. Hey, guys, thank you so much. I really super, super, super appreciate. Thomas LaRock (01:25:56): Thank you. Rob Collie (01:25:56): Thanks for being on the show guys. Dr. Wayne Winston (01:25:57): Bye. Jeff Sagarin (01:25:59): Thank you for calling. Announcer (01:25:59): Thanks for listening to the Raw Data by P3 Adaptive Podcast. Let the experts at P3 Adaptive help your business. Just go to p3adaptive.com. Have a data day!

Shelly Avery is a member of Microsoft's Healthcare Solution Acceleration Team, helping Healthcare customers digitally transform their businesses.  As you listen to this conversation you'll realize, as we did, that Shelly knows the tech AND the human side of the tech very well! References in this episode: FHIR Tom Scott - There is No Algorithm for Truth   Episode Timeline: 4:30 - The high value of customization and integrations in BI in the current era of Middleware, Microsoft Teams and how good it is at connecting humans, The speed of Innovation at MS (some of which is directly customer influenced) 32:10 -  Microsoft's FHIR (Fast Healthcare Interoperability) is revolutionizing the rather large problem of interoperability in the Healthcare space 49:30 - Microsoft Viva is born from My Analytics, Rob gets into Headspace, using data for nefarious purposes Episode Transcript: Rob Collie (00:00:00): Hello friends. Today's guest is Shelly Avery. We've had a lot of Microsoft employees on the show and Shelley continues that tradition. The reason we have that tradition is because there are so many interesting things going on at Microsoft these days. And Shelley brought some super fascinating topics and perspectives to our conversation. For instance, she has a deep background and history with the Teams product for Microsoft. And so we got into the question of what is it that makes Teams so special? I really, really, really appreciated and enjoyed her answer. Rob Collie (00:00:31): And given her current focus on the healthcare industry and on health solutions, we talked a lot about how Microsoft's business applications and Power Platform strategy is actually a perfect fit for what's going on in healthcare today. We did touch on some familiar themes there, such as the new era of middleware, how a 99% solution to a problem is often a 0% solution to a problem. How even 100% of a solution itself is a moving target. And my only slightly partisan opinion that may be Microsoft's competitors in all of these spaces should just save themselves the trouble and tap out now. We talk about the virtual teams that exist on the Teams team at Microsoft. Sorry, I just had to work that into the intro. Rob Collie (00:01:17): I learned a new acronym, FHIR, which is the new upcoming regulatory and technological standard for data interoperability in the healthcare space. We talk a little bit about Veeva. Have you heard of Veeva? I hadn't. It's one of those technologies with a tremendous amount of potential to be used in a positive way and maybe a little bit of potential to be misused if we're not careful. And that conversation was also the excuse for our first ever sound effects here on the Raw Data Podcast. We spared no expense. An iPhone was held very close to a microphone. All in all, just a delightful conversation. I smiled the whole time. We also had the ever upbeat and awesome Krissy in the co-pilot's chair for the duration of this conversation. And with that completely unintentional rhyme out of the way, let's get into it. Announcer (00:02:04): Ladies and gentlemen, may I have your attention please? Rob Collie (00:02:11): This is the Raw Data by P3 Adaptive Podcast, with your host, Rob Collie. Find out what the experts at P3 Adaptive can do for your business. Just go to p3adaptive.com. Raw Data by P3 Adaptive is data with the human element. Welcome to the show. Shelly Avery, how are you doing this morning? Shelly Avery (00:02:35): Hey guys, doing good today. Rob Collie (00:02:37): Well, thanks so much for being here. Another brave soul, first time meeting us. You're willing to have it recorded. That's into the breach. I like it. Shelly Avery (00:02:45): It's good to meet you guys. I'm happy to talk to you today. Rob Collie (00:02:48): We brought Krissy today. Krissy Dyess (00:02:49): How's everybody doing? Rob Collie (00:02:51): How are you Krissy? I mean, it's earlier your time. Krissy Dyess (00:02:53): It is early. Yeah. So normally we do these in the afternoon, but it's early. I'm enjoying the sunrise this morning. Rob Collie (00:03:00): Oh, fantastic. Krissy Dyess (00:03:00): Doing good. Rob Collie (00:03:01): Yeah. A cup of joe, maybe. Krissy Dyess (00:03:03): I don't drink coffee. Shelly Avery (00:03:04): I've had two today. Rob Collie (00:03:05): Shelly, I actually already noticed that. I had noticed before we started recording that the color of your coffee cup changed. That, yeah, she just hot swaps the coffee. Shelly Avery (00:03:16): Travel mug to drop off the kids this morning and then real mug once I got back to the home office. Rob Collie (00:03:22): So Shelly, why don't you tell us what you're doing these days. Give us your CV. Shelly Avery (00:03:25): I am at Microsoft now. I am in a new role that Microsoft has created. I am on a team that is called the Healthcare Solution Acceleration team. And our job is to really help our healthcare customers digitally transform their businesses, hopefully using Microsoft technology. But I've been here five years. I started as a technical specialist, helping customers migrate from on-premise server base infrastructure to Office 365, Exchange and SharePoint in OneDrive. And then Microsoft Teams came around because it wasn't around. It didn't exist when I started, and I became a Microsoft Teams technical specialist. I thoroughly enjoyed it. I loved it. Shelly Avery (00:04:12): Teams has really empowered the world to figure out how to do work different. It created lots of opportunities for people to create new ways of solving their business problems. And it was a lot of fun to be able to partner with our customers and really help them understand how technology can be an advocate for them and just help them do things faster and more efficiently and on their own terms. And so that was super fun, especially working with healthcare. I learned through that about some other features that Microsoft had, not that I didn't know, they didn't exist, but Power Platform, Power BI, Power Automate, Power Apps, and then later Power Virtual Agents. Shelly Avery (00:05:00): And using those inside of the UI of Microsoft Teams to even further enhance what Teams does, which is communication and collaboration, but then putting apps, low-code, no-code apps, and BI and data at the fingertips of these individuals to really, really step up their game and how they're solving their business problems. It's just been super fun and I thoroughly enjoy it. And so taking all of that into my new role, specifically working with healthcare and trying to help them accelerate solutions in their organization to solve their business problems. I thoroughly enjoy what I do every day. Rob Collie (00:05:41): Do you think that your recent background in Teams was a selection criteria for going into health? It would really seem to me like that strong basis in Teams is really quite an asset for you in the healthcare specific role. Shelly Avery (00:05:55): Well, I of course would love to say yes. And I think it is for me, that's how I learned. It's a background that I feel like I'm an asset to my customers, but my new team is comprised of people from all different backgrounds. And so what our new team hopes to be is people who are deep in various different technology areas so that we can lean on each other's expertise when a solution isn't bound by Microsoft Teams. So maybe we need to create a bot in Azure and build it off of a SQL database and put it in Teams. And so we're crossing the entire Microsoft stack. And so, yes, I'm deep in Office 365 and Teams and getting much better into the Power Platform, but as soon as I need to build a bot in Azure, I'm like, "What, how do I do that?" Shelly Avery (00:06:59): So I need that other person on my team who is deep in that area. We're here with you guys. I know y'all are deep in Power BI. We have data scientists on our team and experts in Power BI, which I am not that, but I leverage them because when I talk to my customers, they want to create dashboards and reports that they can have actionable insights on. And so I understand the use case or the problem they're trying to solve. And then I work with my data scientists on the team to help. We come together and bring our skills together to help the customer. So it's just a super fun team. We all geek out in our own area. Rob Collie (00:07:38): Yeah. I mean, it is really a perfect little microcosm of what Microsoft is trying to do with the Power Platform in general. Isn't it? Years ago when they renamed, they Microsoft renamed the Data Insights Summit to be the Business Applications Summit, it wasn't really clear what was going on. There just seemed like one of those funky Microsoft renames. You know how Microsoft changes the acronyms for all you folks in the field, every 18 months, just for yucks. It seemed like one of those, but no, that wasn't it at all, right? There actually was a really long-term grand plan that was already clear behind the scenes there, that just wasn't really clear on the outside. Rob Collie (00:08:18): And all of these technologies coming together, the low-code, no-code or rapid development, whatever you want to call it, right? All of these tools, they enable something to come to life that every single environment, every single customer is different and their needs are different. Their fundamental technological systems that they use, all their mind of business applications, all of those are different and unique. They're unique mix. Plus then you add in the unique challenges that are going on in their particular environment. Rob Collie (00:08:45): You want something off the shelf, but at the same time, if it's not incredibly flexible, if it's not incredibly customizable, it's never ever, ever going to meet the needs of that reality. And I think Microsoft has one of the strongest long-term bets I've ever seen Microsoft make. And it's been really interesting to see it come into focus over the years. Shelly Avery (00:09:06): I'm glad you see that and a lot of people do, but we have a lot of customers. I keep saying health because that's who I work with, that there are health care pointed solutions that are out there that have a single purpose and they are off the shelf. And they do usually do a great job at what they do, but they only do one thing. And we find that almost every application or SaaS that they subscribe to or purchase, has to be connected to data or systems or things like that. And then they have 50 different apps all connected to 50 different things, and it becomes complex. And you have service contracts and everything has to be managed. And so we are pushing that we have a turnkey solution. Shelly Avery (00:09:54): We're actually saying the opposite. We have a solution that gets you 80, 85% of the way there, but then that last bit is fully customizable to make it exactly like you want. And so sometimes that's hard to tell a customer that, "Hey, you're going to pay for something and then you have to build it," or, "You have to pay someone else to help you build it." And they have to be able to see the benefit of that to keep costs down and reduce complexity and app sprawl is something that we see a lot. And so being able to streamline that is something that we definitely try to do and help our customers understand the benefit of. Rob Collie (00:10:33): Sometimes 99% rounds to zero. You have a 99% solution to something, but you simply cannot do the last 1%. And a lot of cases, that's just a failure. I think a lot of off the shelf software, even if it got to 99% of what you need, which is a phenomenal number, it's still not doing it. Plus we also got to remember that the 100% target is also not static. Things change. Even if you're 100% today, your needs tomorrow are going to be different. The ability to customize, the ability to create new integrations and new applications, even if they're lightweight within your environment, is an ongoing must. Rob Collie (00:11:16): I think approaching this as a platform while at the same time making that platform very humane, it doesn't require me to sit down and write C-Sharp every single time I need something new, that's just amazing. I think if you zoom back on all of this, it's almost obvious once you know what to look for. All of the individual systems that we buy, and this is even true of our business here at P3. We're, "Best of breed," in terms of all the line of business software that we've adopted. Best of breed, AKA, whatever we stumbled into at that particular point in time. All those little silos, those line of business silos are very competent. Maybe not excellent all the time, but they're very competent at what that silo is supposed to do. Rob Collie (00:11:59): But an overall business environment, an overall team environment doesn't stop at those silos. It's like the whole thing. It's the whole picture. It's the whole organic total across all of those silos. That's where you live. You don't live in one of them. And so integration across them of various flavors. I think we're in this new second or third era of middleware right now. And Microsoft is just so, so, so well positioned in this game. I didn't see this coming. I just woke up one day and went, "Oh, oh my gosh. Look at what my old buddies are up to." Checkmate. It's been really cool to watch. Shelly Avery (00:12:40): Yeah. It's been really awesome to be here and live it. Sometimes when you're in it, you don't see it happening. And then you look back and you say, "Wow, we've come a long way in the last three years or five years." Rob Collie (00:12:52): Yeah. Let's talk about Teams a little bit more before we switch back into health. Shelly Avery (00:12:57): Yeah, sure. Rob Collie (00:12:57): I find the Teams phenomenon to be just fascinating, which is another way of saying that I missed it a little way, right? Back when I worked on the Excel team, every few years whenever office would finish a release, there'd be like this open season of recruiting. People could move around within office, like a passport free zone. You could just go wherever you wanted. I always struggled to get people who had never worked on Excel to come work on Excel. It was scary. Rob Collie (00:13:24): They've been working on things like Outlook or Word or something like that. It's easy to be, "An expert user of Outlook." It's easy to be an expert user of Word. In other words, the difference between the 80th percentile user of those apps and the 99th percentile user of those apps, it's hard to even distinguish. You can't even really tell the difference between them and practical usage. That's not true for Excel though, right? Shelly Avery (00:13:44): Right. Rob Collie (00:13:45): An Excel expert is like a magician compared to an amateur. And so that was really intimidating, I think. That was the fundamental reason why people struggled to take the leap to come to the Excel team. They felt more comfortable where they were, but a pitch I always gave, which were about a 20% success rate, was data fits through a computer really well. A CPU can improve data. It's built for that. Whereas Outlook and Word, even PowerPoint, I've revised my opinion on all of these since then, but this is me in my early 30s. Going, all those other things, those are about ideas, and communication, and collaboration. Rob Collie (00:14:25): And that's all human stuff. And human stuff doesn't really fit through a CPU all that well. It doesn't come out the other side, enriched in the same way that data does it. Hubris in hindsight, right? I said, "There's never an end to how the improvement that can happen in Excel." Whereas something like Outlook or Word, might be essentially nearing its end state. Then comes Teams, right? Teams is the kryptonite to that whole pitch. I hear myself back in the early 2000s, Teams is all about human interaction. I guess that's what it does. Rob Collie (00:15:02): I guess, to me, it's this alien form, Teams has just exploded. People love it. It's everywhere. I mean, this is an impossible question to answer, but I'm going to ask it anyway, because it's fun to do. What is it? Why are people so excited about Teams? For a while there, it's like SharePoint held a fraction of this excitement. It's in a similar spot, the hub for collaboration in the Microsoft ecosystem. It feels like Teams has said, "Here, let me show you what that really looks like." Shelly Avery (00:15:36): Yeah. I'll do my best to try, but this is my opinion. I don't know what anybody else thinks, but I think it takes the best of the consumer world and the best of the enterprise or commercial world and puts it together all in one app. It has things that when you chat with somebody, it's like you're using a text message. So it's no different than, if you're an Apple user, you open your phone and you go to the green text message app or you go to the Teams app and it looks exactly the same. It has gifts and it has reactions, and you can put stickers and memes in there. And so it's super fun. Shelly Avery (00:16:19): But then you take that enterprise and you can also share a OneDrive link or create a meeting or send someone an Outlook invite or whatever. So it takes that enterprise and mushes it with consumerism. And so it's like taking Facebook and LinkedIn and Office and SharePoint and smashing it all into one app. And so you can have fun with it. You can build relationships with your colleagues or even people external to your organization, but then you can also build presentations and dashboards and create, and even use the Power Platform from a low-code dev perspective, right inside of Teams. Shelly Avery (00:17:02): It spans the spectrum of fun to developing brand new stuff. And so everybody can get something out of it and they can use it the way they want to use it for the purpose that they need to work on, whatever they're doing for the day. And so it can be great for various different people in various different ways. Rob Collie (00:17:28): I love that answer. Krissy Dyess (00:17:29): I have a different perspective. I came from a background of data and technical and all of that type of thing, but this Teams, really with everything transitioning to remote in a hurry over the last year, I feel like it really helps with a level of organization and communication and assets that you talked about, Shelly, to centralize all that because in a difference of data coming at you from many places, now we have communications, now we have remote teams. Krissy Dyess (00:18:05): And I love, like you said, it is fun, it's interactive. Here's where I'm struggling a little bit with Teams. I love it, but what is proper Teams etiquette in terms of like meetings and conversations? For example, I'm having a meeting and I don't want to interrupt somebody, so I'm going to put it in the chat. But then sometimes people feel like, well, the chat is still a form of interruption. I see it as a form of participation. And so I think people are still learning how to embrace these tools. Shelly Avery (00:18:38): Yeah. Well, I think that it also comes to culture. Krissy Dyess (00:18:41): Sure. Shelly Avery (00:18:41): And Microsoft has an amazing culture. We have been on a journey through Satya, our CEO, on really changing the culture of inclusivity and a growth mindset. And it's interesting when we interact with customers who don't have a very friendly and open culture. But I think you use it the way it works for you and for the people that you're working with and your culture. So if you're in a small team setting and it's friendly people, you should feel comfortable to use it the way that it makes you feel comfortable. Shelly Avery (00:19:23): But if you're in a quarterly business review with executives, I mean, think about it. If you're going to lunch with your buddies, you're going to act different than if you're going to a formal dinner with executives, right? And so you use the technology in a way that you would use real life. And so if I'm going to lunch with my buddies, I'm going to be cutting up and giving them funny gifts and patting them on the back. And if I'm in a business meeting with executives, I'm going to have my best dress on and my polite manners. So I'm going to act that way in a meeting too. Krissy Dyess (00:19:51): I totally agree with you. I've had the opportunity recently to work with the Microsoft team and I agree there's a completely different culture than what we see, even from my background, even from our culture, I mean, we're all friendly and stuff. Every organization does have their own culture and exactly what you pointed out, even within that organization, there are different levels and cadences. Shelly Avery (00:20:13): Yeah, it's crazy. So I spent the last three years helping IT organizations deploy Microsoft Teams. And I did that in the midst of COVID, in healthcare. So when you say remote work overnight, literally help telecare organizations enable 35,000 individuals for Teams over a weekend. To the question about culture, it was very difficult for some of the IT organizations to say, "Well, what should we allow our users to do?" There's sensitivity that you can set on gifts in a team. You can say, do we want them to be explicit or PG-13 or PG or G? Shelly Avery (00:20:58): And I had one organization that if there was anything to do with a gift that had to do with politics, that was seen offensive, because what if I sent you a Trump gift and you were a Biden person. I mean, how dare you do that? And so that company was very, very sensitive and they would only allow gifts at a G rating. And a G rating were like cartoons and stickers, where other organizations are like, whatever. If you don't like it, don't use it. Shelly Avery (00:21:29): So there's definitely different cultures and different organizations across the country. And so luckily, there are the controls in the back end and the administrative section on those kinds of things. And then for data too, do you want data to be shared externally or do you want people to be able to chat externally or not? And who do they want to be able to chat with? So there's lots of governance and data protection controls in the background. Krissy Dyess (00:21:58): And being a data person, what is really cool about Teams and all these things that you just described is on the backend, all of that stuff is just data. That's why you can control. That's why you can help your organization with these. And I think that's really cool. I am super excited about Teams. I was excited about Power Pivot in Excel, and I was excited with Power BI Desktop, and what you explained too, how it starts to integrate the Power Apps, the bots, all of that into this changing ecosystem of how we work, the ability to bring that from the top level all the way down to the frontline workers, to impact and drive actions, I am super excited about Teams. I can't wait to see how organizations learn more, how that they can adopt these tools, because I think there's so much that people just don't know because it is so new and it's a different way, just like Power BI was. Shelly Avery (00:22:57): I'll give you an example about that. We have this one group inside of Microsoft, it's called the [SLATE 00:23:04] team. And you know how Microsoft is with making acronyms. I have no idea what SLATE actually stands for, but what they do is they work with customers who have a unique idea and they help them build low-code or apps inside a Teams. And they built this one app called the Company Communicator. Basically what it is, is it's like a mass texting app, where I can create a little message and push it out via chat or via a Team to everyone in the organization or to a subset of people. Shelly Avery (00:23:39): And it created a cute little adaptive card where you could put a headline and a picture, and then a little message. After that got so popular, Microsoft built it into the product, right? It started from a customer, it went through a program. It was customer purpose built. Then it got so much organic growth through all of our customers loving this idea of pushing notifications. So we turned it into code and now it's in the product. I think that, that is so cool, how Teams is democratizing the ability for customers to influence product and future releases that now everybody in the world gets to take advantage of. Shelly Avery (00:24:28): So that's another thing that I just, I love about it as a product, but also we call it the Teams team at Microsoft, is they're innovating so fast and I'm just a few months out of that role and I already feel behind. I just saw a blog with what's new in Teams in August. And I'm like, I need to go and read this to make sure I know everything that's new because they just come out with so much new stuff every month. And it closes the gap, Rob, you mentioned earlier, when a product's only 99%, it's really zero. Shelly Avery (00:25:03): I think the bet on Microsoft is, it might be 99% today, but it's probably going to be 100% in a couple of months because we're innovating so fast. And your 99% today, isn't going to be your 99% in six months. And so it's a moving target, not only for the customer, but for Microsoft too. And so we want to catch up with features that are on the backlog, but the backlog just keeps growing and growing. And so the faster we can innovate and build these into the product, we will. Rob Collie (00:25:33): I just feel like if you're watching a really high stakes chess match, which I never do, but imagine that you did, to the untrained eye, this is an even game. And all of a sudden, one of the chess masters just resigned, just tips the king over and says, "Yeah, I've lost." I just feel like as a software industry, we should just take a moment and say, "Hey, Salesforce, all your other, your SAP, do y'all just want to call it, you want to just tip your Kings over, save us all a lot of trouble." I don't even work for Microsoft and I'm looking at this going, "Oh, boy." Remember, I'm not paid to say this. I really think Microsoft has really, really, really dialed it in. Rob Collie (00:26:16): I'd like to also go back to your answer about why Teams is so special. I think it was a perfect answer. Rewind 10 years, 11 years, I'm struggling to explain to people why this whole DAX and tabular data modeling thing that was only present at that time, only in the Excel environment, and only as an add in, it was, in some ways the most primitive exposure possible of this new technology. I was trying to explain to people why this was so special. And it was particularly difficult to explain it to people who had intimately known it's [4Runner 00:26:49], which was the analysis services multi-dimensional. Rob Collie (00:26:52): And really, technologically speaking, there wasn't too much about this new thing that was superior. If you looked that gift horse in the mouth and examined its lines and everything, you'd be like, there's really not much different here or it's clearly better. Now it had one thing that was clearly better, which was the in-memory column oriented compression. And that was pretty sci-fi. That was pretty cool, but it wasn't the tech. It wasn't like one of these was able to make the CPU scream at 500% capacity or something like that. It wasn't that at all. It was that this new tech fit the way humans work so perfectly. It met the humans where they were, whereas the previous one forced the human world to bend to its will. The humans had to come to it and meet it where it was. And this is a very subtle and nuanced point. Rob Collie (00:27:49): But in practice, it is everything. In practice, it means that a company like ours, that operates completely differently than the data consulting firms and BI professional services firms of the past, and really honestly, today, I think most firms are still operating that old way. We're a completely different species of a company. And we exist because these tools work a different way for the humans. And over and over and over again, this is why the ROI from Power BI is so insane when you use it the right way, when you really lean into it strikes. Your explanation about Teams, it echoed that for me. It's professional tool that fits the humans really well. Rob Collie (00:28:36): And you don't typically talk about stuff like that. If you're a technology professional, those kinds of answers, you're always looking for some sort of more hardcore answer. It's capabilities. Look at the check boxes it's got on the box, right? This other description of it fits the humans really well, it's not a good sales pitch, but in reality, it's everything. It's a difficult thing to do, right? Rob Collie (00:28:59): One of its chief strengths is also just, doesn't make a good sound bite or like, oh, okay. So now you have to wait and see it for yourself. You have to experience it. And I think that's what we've seen. Is that the people who've really leaned into Teams, they all have this surprised reaction, or they say, six to 12 months after really getting into it, as they describe how much they like it, there's this undertone of, "Yeah, it's really turned out to be amazing." You can tell that they didn't quite expect it. And now they're a convert. Shelly Avery (00:29:31): Well, I think a lot of IT organizations, they push applications out and Teams to the masses is, oh, it's just another application that IT is forcing us to use. And they're resistant to change because the last app IT pushed out wasn't great. And then they finally get in there and they do what you and I are talking about. They chat in it, they text in it, they meet in it, they have fun in it. And then six months later, they're like, "How did I do my job without this?" They enjoy it. It's easy to use, it's very accommodating and friendly to different personalities and different work types. And it's so unique in the way that you and I and Krissy can all use it all day long, every day, and we use it completely differently, and yet we all have the same opinion of it, is it works great for me. Rob Collie (00:30:30): That's the whole mark of a successful product. And one that spreads itself, right? It develops impassioned evangelists. Again, just like everyone else, I would not have seen that coming. Shelly Avery (00:30:41): You were at Microsoft from an Excel Power Pivot perspective and you now are not, and have started your own business and they're successful in that. I know people that worked at Microsoft and literally quit Microsoft just to be a YouTube star on how awesome Teams is and all the cool stuff you can do with it, and they've made a living out of it because it's a product that does so much and it's never ending in the way that it can be used and how unique it is. It blows me away when I actually saw a gentleman who was at Microsoft as a product manager and I followed him on YouTube, and then one day he said, his YouTube post was, "I am retiring from Microsoft." And he was younger than I am. I'm like, "How are you mean you're retiring?" Krissy Dyess (00:31:32): I followed the same story that you did, Shelly. I know exactly who you're talking about. What I really love, what the appeal of it to me is, is it's always these little things that people don't know that make the biggest impact. And when you're in an environment where you're not exposed to people doing those neat tips and tricks, having the ability of finding somebody out onto YouTube sharing that, and then you can bring it into your organization and start to spread it, it's really impactful because a lot of times people think, "Oh, it needs to be this complicated technical solution." And honestly, it's always the little things that people are like, "Wow, I didn't know I could do that." Shelly Avery (00:32:12): Agreed. Rob Collie (00:32:13): So let's turn the corner. Let's talk about health, Shelly. Where should we start? Shelly Avery (00:32:16): Well, when you were talking earlier about how Microsoft Teams is this new thing, I think people had an aha moment and I think there is an aha moment that is about to come in health. And I'd love to talk about that for a minute. I think it plays into your audience well because it's about data. Rob Collie (00:32:41): Very important question. Are there people involved? Shelly Avery (00:32:43): There are people involved. Rob Collie (00:32:45): Oh, okay then. We're good. We're good. Shelly Avery (00:32:46): Yeah, yeah. Rob Collie (00:32:47): Okay. All right. Shelly Avery (00:32:48): Yeah. There is interoperability of data in health. So think about, from a human perspective, heaven forbid you get in a car accident and you go to an ER and they have to bandage you up. That ER is owned by some health organization and they now have data on you, but it's not the same health organization where you go to see your primary care physician. And so how does your primary care physician know about your ER visit and how do they know what medicines that you were given and whether those had adverse reactions to you or not? Shelly Avery (00:33:22): Well, without interoperability of data, that just doesn't happen. And there is an old version of healthcare interoperability called HL7. Again, another acronym, but the new interoperability standard is called FHIR, Fast Healthcare Interoperability. The idea of FHIR is supposed to be universal so that that ER can digitally transfer that information to your PCP, your primary care physician. And so your medical record and your information can stay up-to-date with all the people that are medically treating you or for even you, like if you move to another city and you want to say, "Hey, I need all my information. I'm going to take it to my new doctor." Shelly Avery (00:34:10): And so this idea of interoperability, it's not a Microsoft thing. It's a healthcare standard that is happening in the industry. But what Microsoft has done is we have gone full steam ahead on this FHIR interoperability and built a stack of technology solutions based on ingesting data through FHIR. And we have a bunch of healthcare APIs, FHIR API being one of them, to now take all those low-code, Power Platform, Microsoft Teams, bots, and hydrate those apps with all of this data from healthcare to now be able to really unleash this data. Shelly Avery (00:35:02): So you need an app to have a rounding solution bedside in a hospital. You now have the ability to suck that data in from Rob, that he's been to the ER and his primary care physician, and now you're in for knee surgery. And so I have all that information that's aggregated from all over, and now it's in this cute little rounding app that we built off of Power Platform, or same thing with Power BI, or a chat bot in Teams. We can chat this health data and say, "Hey, is Rob's labs ready yet?" And the chat bot goes and sucks that data in and says, "Yes, here's Rob's labs. Here's the link to it." Shelly Avery (00:35:44): And so just being able to unleash that and build these apps or bots or experiences for the human to be able to interact with that data is really what we are trying to do. And so I'm super excited about it. This is a new team that I'm on and this is really what we're trying to drive. So I think it's going to be game changer for the industry. Rob Collie (00:36:09): So this is my first time hearing of this new interoperability standard. First of all, FHIR, it sounds cool. I like it. It definitely sounds like it's useful for sharing healthcare and patient information across organizations. Do you also see it as something that's going to be useful even within an organization, like between the silos, between these different systems within a single entity? Shelly Avery (00:36:32): Yes. And it will do that first before it goes across organizations. And- Rob Collie (00:36:37): Okay. Shelly Avery (00:36:38): This is a challenge internally too, because there's software technology that these electronic medical records, that your medical record, my medical records sit in at each of these organizations. And most large healthcare providers have multiple instances of these electronic medical records. Sometimes they have multiple different types through mergers and acquisitions and growth over time, or this department got an upgrade, but the other department didn't. And even amongst themselves, they can't share information with each other. And so if a call center services 10,000 patients, but they have four different electronic medical records, when Rob calls into that call center, how the heck do I know which one you're in and who you are and all that? Shelly Avery (00:37:30): So if we can use this FHIR interoperability to aggregate all of that and have it in a single place, now we've built this great call center app that knows that Rob is calling in and who you are. And I immediately have your information. I could say, "Oh, Rob, are you calling about the meds that you got from your ER visit last week?" It's very personalized. So let's personalize care. Let's have better patient engagement. Let's round with our patients and have the right information where we need it, regardless of where the original data sits. Rob Collie (00:38:01): So it's a new standard, FHIR, right? Shelly Avery (00:38:04): Yes. Rob Collie (00:38:04): And so let's pretend I'm a healthcare organization and I have, again, these, "I've got a best of breed set up." I've got a jillion different siloed line of business systems. Some of them are new, some of them are not. These older systems that I have, they're not going to be playing nice with this new FHIR standard. They haven't even heard of it, that software. So- Shelly Avery (00:38:24): That's correct. Rob Collie (00:38:25): How do I, as an organization, connect those wires when some of my more long-ended two systems aren't going to be supporting the standard natively? Shelly Avery (00:38:36): And that's part of our challenge right now. A lot of the customers that we're talking to, they see the future, they like the vision that Microsoft is painting. They want these human interactions like we're discussing, but they'll say to us, "We aren't ready for FHIR," or, "We haven't made that transition yet." Our comment back to them is we can help you get there. And it is a requirement that they get there by a certain date in the future. So why not have a company like Microsoft help them? Shelly Avery (00:39:11): Now, it's not necessarily an easy task. There are data mappings that have to happen. And a lot of these electronic medical systems are in the old standard, which we can map from the old standard to the new standard. It takes a little bit of manual work, but you only have to do it once, because once you do it once, it's in the standard and now you've unleashed that data. There's also custom fields though. Some developers- Rob Collie (00:39:38): Of course. Shelly Avery (00:39:38): Have gone into these electronic medical records and they built some custom field that doesn't map to FHIR. So then you got to have somebody who knows that. And so there is hard work to do it in the beginning. I'm not trying to say that there isn't, but we do have healthcare interoperability partners, and system integrators, and Microsoft to help these organizations get into that standard. And the new marketed term for all of this is the Microsoft Cloud for Healthcare. Shelly Avery (00:40:10): And so it's all about ingesting that data and then unleashing that data to create these great, either apps or applets, or bots, or scenarios that empower the people who either work at these systems or even for patients to be able to interact with and have better experiences for themselves. And so, you only have to do the hard work once and then it's in there. And so you're right. It isn't a turnkey, there is work that has to be done, but they're going to have to do it eventually. So we'd love to be able to partner with them and help them get to meet those regulatory compliances that are coming in the near future. Rob Collie (00:40:52): Yeah. Another example of where it's good to have a platform, right? Shelly Avery (00:40:55): Right. Rob Collie (00:40:56): If that missing 1% is interoperability, that's a big 1% that a platform like Microsoft is very, very, very prepared to help you connect those dots. It also, it's really helpful that these older systems that we're talking about, if they already had to, as you pointed out, if they already had to play ball with an older interoperability format, that's end sharp contrast to your average line of business software that has no interest in interoperability at all. T Rob Collie (00:41:26): he average line of business system is like, no, no, no, no, no. We are a silo and we like being a silo. And why would we ... Mm-mm (negative), no. We are here to hoard the valuable data that is collected in here. Mm-mm (negative), no. Even though it sounds rightfully like labor intensive, one time investment, compared to the average interoperability game that happens across the world, across all industries, it sounds like there's already a really, really, really strong starting point. That's a big, big, big point in your favor. Plus if it's going to be a regulatory standard in the future, that is unheard either. Shelly Avery (00:42:00): Right. Krissy Dyess (00:42:01): I'm curious though, as to what changed, because honestly, it is one of the reasons why I'm appointment averse, is because every time you go into a different doctor and it's really common for people to move nowadays. And you're like, oh, I got to fill out all the same forms, over and over again. In my mind, I always thought it's somewhere. Why can't it be everywhere? I guess I thought maybe there was some privacy reason that was the blocker. Has something changed there? Shelly Avery (00:42:28): You're absolutely right. And no, there is still what's called the HIPAA regulations. And so the entire Microsoft Cloud for Healthcare is HIPAA compliant. It does meet all of the requirements for that. And so the FHIR standard, FHIR mandate is under that HIPAA compliance. And so that's a U.S. regulation. It's not in the EU or others. They have their alternative to HIPAA around keeping healthcare information protected. And it's important to be able to do that. And so the old HL7 standard of interoperability was highly customizable and the new FHIR standard is less customizable, and that is how it is able to have more liquid interoperability. Shelly Avery (00:43:27): I'll give you an example. Sex and gender are two completely different things. And we know that in this day and age, but in the FHIR standard, there is a born sex and it is one or another, and you can't really change it. But in the HL7, you could add seven or eight or nine or 10 different categories for that. So when you have the FHIR standards met, born sex is a one or a zero, basically. Right now they have the other category of gender that there's a bunch of options there. And then they even have another category. And so it's creating the standard that everyone in healthcare has to meet as opposed to going in and making it where I can make 37 customizations because in my hospital, I allow them to have 37 choices. Shelly Avery (00:44:28): Religion is another one. Religion is huge. I mean, there's endless amounts of religions. In the FHIR standard, there's a set amount and then in other. And so you have to fall into either the set amount or other, and that allows for that more liquid interoperability, or that is the goal. That's the goal of FHIR. Now, I'm getting a little deeper into more of the regulatory compliance and how the standards work. There's tons more deep technical experts on healthcare compliance than I am. I'm more of a technologist than a healthcare compliance expert, but knowing how it works a little bit helps you understand why the technology is empowering or we hope in the future has the potential to empower the industry to be able to do more with this data. Rob Collie (00:45:18): Even that little deep dive there, I mean, that really, for me and for the listeners, you really just certified your bonafides there. If anyone was wondering how deep you were into this stuff. You always got to be careful. You're not the expert on that. There are people who know it much better than you. The fact that you know that much while also being on top about all those other stuff, you're in the right role. Like Holy cow. Shelly Avery (00:45:41): For my role, they did require healthcare expertise. And we have another team that partners with us that actually are folks from the industry. So we have MDs, PhDs, ex-CIOs and nurses with their RNs, from industry that work at Microsoft as the healthcare industry team, that partner with us around more of these deep healthcare needs. And when we're talking to chief medical officers or chief nursing officers, who doesn't like their title to be matched. Shelly Avery (00:46:18): So when we have a chief medical officer like Dr. Rhew at Microsoft, or a chief nursing officer, or ex-CIOs of healthcare organizations to come in and talk to current CIOs, they feel like we're talking to them from their shoes. And so my team partners with that industry team. Not that they aren't technical and don't understand how the technology works, but we are supposed to understand healthcare enough and how the technology fits for those healthcare scenarios and use cases that they need help with. Rob Collie (00:46:52): To use a metaphor, if you're going to build re race cars, it helps to hire some people who drive race cars. Shelly Avery (00:47:00): Exactly. Rob Collie (00:47:00): Right. I've seen this evolution on the Excel team over the years too. There's more and more people on the Excel team who came up not originally as software engineers, but as people in finance and things like that. Whereas I was a computer science major that had to learn Excel in order to work on the Excel team. And so it was, if you populate a team with nothing but me, back then anyway, you end up with a team of mechanics who has no idea what it's like to go into turn three cars ride. I'm using a racing metaphor. I don't even watch racing. I find it incredibly dull, but I love a good metaphor though. Shelly Avery (00:47:40): Sure. Absolutely. I think Microsoft has done that and is continuing to expand that industry team, even our president of health and life sciences comes from the industry. A lot of our leaders from even a marketing perspective or from a product development perspective, they're starting to hire from the industry. Rob Collie (00:48:03): That's wisdom. That's humility. I think 20 years ago, we would've probably seen Microsoft put some up and coming computer science guard in that role. And you still need those people for sure. Someone who grew writing C++ isn't going to know everything that they need to know. It's again, there's this whole notion of collaboration. The thing we keep coming back to. It takes a lifetime to amass the expertise to be truly good at something. Rob Collie (00:48:29): And so, guess what, you're never going to find everything that you need in one person. You're going to need people with different histories in order to be successful. And so it's simple. And yet I don't take it for granted, when I see teams being assembled this way, I've learned to respect it, that it is a necessary and good thing. It's always worth praising even if it seems like it's table stakes. A lot of people don't view it as table stakes. Still, they've got some things to learn. Krissy Dyess (00:48:55): So Teams is empowering, it's a central hub, it's a window into all these other applications, the Power BI that brings the insights, the bots, the Power Apps, the drives actions. Tell me a little bit about the Veeva. I hear about Veeva, that whole human side. Tell me how you're seeing Veeva start to make its way to help balance, I think. Rob Collie (00:49:21): And what is Veeva? Krissy Dyess (00:49:21): Yes. Veeva. Shelly Avery (00:49:23): Yeah, sure. Microsoft Veeva is what we have marketed the name of our employee experience platform. If you're a Microsoft E, you've probably seen in the past years something in Outlook called MyAnalytics. MyAnalytics was the very early stages of what is now Microsoft Veeva. MyAnalytics was a analytics engine that had some AI in it that would give you insights about your day, or your week, or your month. It would tell you, "Hey, Shelly, you were meeting with Krissy like every week for a few weeks and you haven't talked to her in a while. Do you think it's about time to reach out?" And then it will even give you a button that says, chat with Krissy now, or schedule a meeting with Krissy now. Krissy Dyess (00:50:18): And I love that. Shelly Avery (00:50:19): Yeah. It would pop open your calendar- Krissy Dyess (00:50:21): Because I would forget. You have all your lists and you have all your things. And honestly, when those things would come across, I was like, "Oh, yeah, you're right." And I was like, wait a minute. The technology is getting on top of all this stuff that I can't keep track of. It's amazing. Shelly Avery (00:50:34): Yeah. That was the beginning of it. Microsoft also came out with another tool called Workplace Analytics, which was the next step of MyAnalytics, where it would anonymize the data and send it to your manager or to your direct report and it would go up the chain all the way. So if my manager had 10 people on it, he would get a daily or weekly report that said, "Hey, your 10 people, this is what they're doing. They're multitasking in their meetings or they're working after hours. Hey, maybe you should encourage them to close the lid of their laptop at night. Let them have better work-life balance." It provides the manager with insights. Right? Krissy Dyess (00:51:17): That's right. Because these are important. This is important to your overall health of your business, your company, your culture. Shelly Avery (00:51:24): Exactly. So Microsoft Veeva took MyAnalytics and turned it into what is now called Veeva Insights. And then there is Manager Insights and Workplace Insights. And so insights is really just a rebranding and a movement from MyAnalytics in Outlook. And it's now insight of Microsoft Teams. Because Teams has that developer side of it, there's so much more that you can do with that information in Teams than it is within Outlook. And so it gives you nudges also to set focus time on your calendar or set learning time on your calendar, and it updates your status, your green, yellow, red, to focusing or away or things like that. And so it uses AI to help you know maybe when you're overworking or who you might need to collaborate with. Recently, Microsoft made a investment with a meditation company, Headspace. Krissy Dyess (00:52:30): Yes. Yes. See, this speaks to me. I love it. Shelly Avery (00:52:33): Yeah. It's built into Microsoft Veeva. What I use it for is there's a feature called your Virtual Commute. We all used to drive in and drive out of the office and you had, and I forgot about it, but you had that me time in the car. We could listen to a podcast or veg out on the radio or something, but it was some me time while you were in the car, going home from work. And we lost that when we all went remote. It's like I literally shut my computer and then I walk in the kitchen and start cooking dinner. It's like, where is that me time? And so I use the Virtual Commute and I don't use it every day. It's about a five to seven minute decompression. It says, are you ready to wrap up your day? Krissy Dyess (00:53:17): I need this. Shelly Avery (00:53:17): Do you have any last minute emails you need to send? Do you need to create any to-dos? And it integrates with Microsoft to-dos, so you can click on things and say, add to my to-do. And then it walks you through a little meditation. Yeah, Rob's got it on right now. Krissy Dyess (00:53:38): This feels amazing. You just took this conversation to a new place and adding in the music. I'm feeling it. This is just taking work to a new level. Rob Collie (00:53:50): Imagine a world of Raw Data. Data with the human element. Krissy Dyess (00:53:58): No, no. Make it come back. Shelly Avery (00:54:00): Yeah. Krissy Dyess (00:54:00): Oh, no. Can we get that? Rob Collie (00:54:06): I couldn't help it. Krissy Dyess (00:54:08): No. This is what people need. Honestly, when I heard about this, and I'm surprised when I say Veeva, people are like, "What's Veeva?" And I loved your explanation because it gave so much more detail and history, people need this. Think about like, it gives tap it into how long you've been sitting and giving you that balance. This is amazing. Wow. I'm even more excited about this. Shelly Avery (00:54:31): Well, and I think- Krissy Dyess (00:54:33): I think I can make it another 50 years in the work environment now, like [inaudible 00:54:37]. Rob Collie (00:54:37): I said, that was the plutonium battery that you needed. Shelly Avery (00:54:41): Well, and it's so cool because just like there's a Teams team, there's a Veeva team and they are just getting started. They're integrating LinkedIn learning into Veeva learning and all these other learning platforms. So you can learn right in the UI of Teams and you don't have to single sign on and then MFA and forget your password to log into all these other learning tools. And it allows you to share it right inside of Team, say, "Hey, team, I just did this great learning. I think it'd be great for you." Shelly Avery (00:55:11): And customers can upload their own learning modules to it. There's Veeva topics, which is this Wikipedia where it's self-curated information. And what is great, like we've talked about acronyms at Microsoft, every acronym has a topic page now at Microsoft. So anytime you type an acronym, it hyperlinks it. So I'm chatting you in Teams and I say FHIR. And it's like, what the heck is FHIR? You hyperlink it and it gives you an explanation of what FHIR is. Krissy Dyess (00:55:43): That's game changer in itself. Rob Collie (00:55:46): So, does it also pick up pop culture, like if I type IKR, I know, right? And someone else doesn't know what that means. Usually I'm on the receiving end of this. Someone used an acronym yesterday in a chat with me that I'm sitting there going like, "Oh, what new hipsters saying is this?" And it turned out, no, no, no, no, no. That's the customer, Rob. Krissy Dyess (00:56:08): Here's something really weird too. I love this Veeva thing. I love Teams and all this productivity and pulling all the pieces together. Gosh, back in the day, when I moved from back east to Phoenix out west and I started working at the company I was with, they actually had a meditation person that would come in every so often and they would have us stand up and do exercises. And then even to just like little chair massages and it all- Rob Collie (00:56:41): Please continue. Krissy Dyess (00:56:42): Right. Oh, this is just as amazing. I don't even know what track you got, what meditation track, but I just need this in my day. And so many other people do. Rob Collie (00:56:55): Do you see that? I feel compelled to not even hold the phone steady. I have to move it in a circle, a very gentle circle as I play it into the microphone. I didn't even know I was doing that. Shelly Avery (00:57:06): It makes you want to sway. Rob Collie (00:57:11): Yeah. In the middle of the meditation music, you heard my reminder for my next meeting go off. Oh, it really spoiled the mood. Krissy Dyess (00:57:21): And you haven't reviewed that 50 page slide deck. And then- Rob Collie (00:57:25): That's right. Krissy Dyess (00:57:26): Here it goes. Reality comes right back in. You're like, "Oh, okay. Veeva, Veeva, help me." Shelly Avery (00:57:32): I Mean, not to pitch, I'm not selling Veeva anymore. I'm a user of it, but those are also things it does. It gives you alert in the beginning of the day that says, "Hey, Shelly, here's what your day looks like. You got these six meetings. Here's a PowerPoint that you were working on, that might go with this meeting. Do you need to review it?" The Outlook team has also built in, I don't know if you guys have seen this. In Outlook now, you can create 25 minute meetings, 45 minute meetings or 55 minute meetings that either start five minutes late or in five minutes early to give that bio break meeting buffer between meetings. Krissy Dyess (00:58:14): That's right. Shelly Avery (00:58:14): Because when you're fully remote, all I do is sit around and I click the join button all day. I need to go refresh my drink, I need to stand up, I need to stretch. And so, again, we talked about culture at Microsoft earlier, and Satya has been on multiple news outlets talking about how we were the customer zero for Veeva and for this workplace balance. And it's so incredibly crazy to me how much people care about people. It's what we need to do as a human race. We just need to care about people more and allowing technology to play a part in that. It's so cool that we have that. Hopefully organizations take advantage of it for their employees. So more people can have ... It's just the little things- Krissy Dyess (00:59:06): It is the little things. Shelly Avery (00:59:06): You mentioned, Krissy, earlier, it's the little things, like five minute less meetings. It's a sign of respect. Let me use the restroom. Don't be mad at me if I'm not on at the top of the hour. I need two minutes to jump from my last meeting to switch my train of thought to get into the next one. And I think that it's super cool that I get to be a part of a company that's offering that to others. And I hope the rest of the world sees it and gets to take advantage of it. Krissy Dyess (00:59:35): This week, just recently, because I am seeing the five minute grace period, the meetings start five after, but I just, this week, because now people are starting to creep in at 10 after. So it's like everybody expects that five minute because exactly like you said, you're on back-to-back meetings, you don't get a break, but now that five minutes, now it's okay if you're 10 minutes after. Then it's going to be 15. Right? Rob Collie (00:59:59): Yeah. It's like back when I used to teach classes, I would tell people we're going to take a five minute break and we'll resume in 10. Right? Shelly Avery (01:00:08): Yeah. Krissy Dyess (01:00:09): That's right. Rob Collie (01:00:10): But if I tell you it's a 10 minute break, it becomes a 15 minute break. You can't have that. So just say, "Five minute break, but I'll see you in 10 minutes." Krissy Dyess (01:00:17): When I was training, there was no break. So all my students out there- Rob Collie (01:00:20): You just powered through? Krissy Dyess (01:00:22): Because there was so much cool things that I ... I was like, "No breaks. Let's keep going." And they're looking at me. Rob Collie (01:00:28): In the morning, everybody please come in, sit down at a seat that has a book in front of it. And in the bag next to it, is your astronaut diapers for the day. Krissy Dyess (01:00:38): Don't drink water or you might have to go. Rob Collie (01:00:41): Yeah, yeah. We have capitas. Krissy Dyess (01:00:43): I was a different person back then. Now I'm embracing the Veeva and the breaks. I feel sorry for all my students, but that's what I did, because there was so much cool stuff. No breaks. Rob Collie (01:00:52): While we're on this topic, just briefly, this Veeva thing, it seems like one of those technologies that it's not the only thing like it, for sure. But it can be used for good, but it could also be used in a very dark way, if we're not careful. When we were talking to Jen [Stirrup 01:01:08] on a recent podcast, even dashboards reports and things can be used as a form of workplace surveillance. I do see all of the glass half full potential here. Are there any concerns about customers saying, "Yeah, yeah, yeah, we'll use this for the positive, the meditation and the humane," but then they just turn around and roll it out as like the Amazon horror stories of the driver's not allowed to take bathroom breaks. And this is a means of enforcing that. Shelly Avery (01:01:36): Yeah. I think there is fear of that. I mean, I know a ton of people they put duct tape over their cameras and they don't want windows hello because they think the world's spying on them. There are just people that have that fear. Rob Collie (01:01:49): I don't know any of those. Shelly Avery (01:01:51): Yeah. But I think Microsoft is trying to protect customers a little bit in this area, that you are the only one that can see your data. Everything else is anonymous. Now, if you're a team of one and you report to your manager, obviously the manager is going to know it's you or a team of two, there are those things. But as you go up from a manager one to an M two, to a director, to a VP, and then all the way up to HR, unless you're a very, very small company, the data is segregated into demographics, and geographies, and departments, and roles, and skills, and tenure. And they slice and dice that data to learn insights as to how one population is performing or working over another population. Shelly Avery (01:02:42): I think it was one engineering group at Microsoft that was really, really being overworked. Not that they weren't all being overworked. I'm sure everybody is overworked in every position at every company everywhere. But there was this one in particular organization at Microsoft, I think they were putting in like 18 hour days. It was ridiculous. And the feedback they got from these individuals was, "We have to work after hours because we are in meetings all day." And they were individual contributor. They were coders. They needed that three to four hours to get that line of code written or tested or whatever. Shelly Avery (01:03:17): They made a meeting free Wednesdays. They literally wouldn't allow people to have meetings. Now you could collaborate with people and set your own, but no internal or manager type meetings those days. And the productivity of that group after three or four months, just completely changed. And so using the data, that's what the data is meant to be there for. Now, there are people in the world that are just going to make Ponzi schemes. They're just evil people. Data can be used, I'm sure in malicious ways. I think Microsoft is trying their best to make it so they can't be super micromanagement at least down to the individual level. Rob Collie (01:04:02): It's a certainly a very, very challenging frontier for a technology company, right? We're going there as an industry. It's inevitable. It's happening. There's no point in trying to say, "Oh, no, let's put up the firewall here." We're seeing this thing. This goes back to my original, something I said a long time ago in this discussion about how certain things don't go through a computer very well. I think this is one of those examples. We're seeing it with Facebook and YouTube. Technology companies, they're in the position now, these companies, of being the arbiters of truth and there's no algorithm. Rob Collie (01:04:36): There's actually a really great YouTube video, or this one guy in the UK talks about, there is no algorithm for truth, but we've created these platforms that are the primary disseminators of information in the world and they're completely and forever ill equipped to be arbiter of truth. Wow, look at the world that we're in. So, I don't think this particular topic is on that scale. It doesn't have that same reach. I don't think as the other things, but I think it's a cousin of those problems in some ways. It's a more solvable problem, I think, than the Facebook and YouTube problem that we're seeing. But this is where the real stuff is. Is like, how do we deploy these things in a way that is a net benefit to humanity? And not just as a net benefit to shareholders. Shelly Avery (01:05:27): Exactly. Rob Collie (01:05:28): That's attention, especially I think in the United States. It's a very different dynamic like in Europe, for instance. I can imagine the adoption profile of something like Veeva in Europe will be very different than in the USA. Shelly Avery (01:05:40): Well, it will have to meet European standards. European has GDPR around privacy laws. And so there might be different settings or features that can or can't be enabled in a product like Veeva in UK or in Europe to comply with those. Rob Collie (01:05:58): A lot of consumer products in the United States, they have to meet California standards. Shelly Avery (01:06:03): Exactly. Rob Collie (01:06:04): And then because of that, the whole country is California in terms of its standards, because you're manufacturing product. Software's a little different, it can be tuned differently in different places. Shelly, I have really enjoyed this conversation and thank you so much for making the time. You also get a gif of yourself. Why don't have to be mentioned that. Krissy Dyess (01:06:19): A G-I-F not G-I-F-T. Gif. Rob Collie (01:06:22): Right. Not a gift, but it is a gift- Krissy Dyess (01:06:24): It's a gift or a gif. Rob Collie (01:06:29): Or a gif. Yeah. Shelly Avery (01:06:29): Yay. Fun. Rob Collie (01:06:29): Yeah. Krissy Dyess (01:06:29): And you could frame it. Rob Collie (01:06:29): It needs to be a movable frame. We could sell it as an NFT. Shelly Avery (01:06:32): Yeah. Rob Collie (01:06:37): And I also want to say, I really, really, really detected just a tremendous amount of wisdom in you, in this conversation. And that's not something that you necessarily run into all of the time in technology, but I think there's something about the way that you approach problems, the way that you think about them, that I find very valuable and special. And I wanted to make sure that I said that before wrapped up. So, thanks for being here. Yeah. And thanks for bringing your perspective, which I really appreciated it. Shelly Avery (01:07:04): Thanks so much guys. Announcer (01:07:05): Thanks for listening to the Raw Data by P3 Adaptive Podcast. Let the experts at P3 Adaptive help your business. Just go to p3adaptive.com. Have a data day!

We welcome Power Platform expertise in the form of Two Alex! Alex Dupler and Alex Powers both work at Microsoft. The organization they work for and their first names aren't the only thing that these two share! They also both have a lot of experience with and passion for the Power Platform. Alex Powers is a member of the Power BI Customer Advisory Team (PBICAT), and Alex Dupler is a Program Manager focused on BI & Data Infrastructure. These guys know data! Follow Two Alex: Alex Dupler Twitter Alex Powers Twitter Two Alex Youtube Channel References in this Episode: Raw Data with Brad and Kai from Agree Media Episode Timeline: 7:00 - The woes of Stack Ranking, Data storage options, more fun with names! 22:00 - What draws you to data?, The value (and drawbacks) of Excel, and the path to Power BI 36:40 - Two Alex-similarities and differences, Rob tells a story of someone crossing him, and one of Rob's favorites-the art of using BI to drive action 59:00 - When BI and IT collide, the 2 Alex's non-traditional BI path, the value of being an expert even if you aren't THE expert 1:16:00 - Two Alex LOVE helping people, is there value to documentation?, knowing the Business portion of Business Intelligence 1:37:00 - Advertising performance discussion Episode Transcript: Rob Collie (00:00:00): Hello friends. Today's guests are Alex Powers and Alex Dupler, collectively known as Two Alex. They're both Microsoft employees in very different roles, but both have their feet rooted firmly in the power platform. You might be familiar with their YouTube show. I interact with them primarily on Twitter and a little bit on Reddit. And this is the first time I've had really any conversation of length with Alex Powers. And it's the first time I've had any conversation at all with Alex Dupler. And no surprise here, really, really cool people. We had a lot of fun, really dynamic and inspiring, interesting conversation that wound through a number of topics, including some show favorites, like non-traditional backgrounds, and closing the action loop, and imposter syndrome. We talk about how years ago Alex Powers wrote a review of my book that called out the intermission in the book and how, what a delight that was at the time to read. Rob Collie (00:00:57): And that leads to a conversation about how we're always essentially at our own little intermission in our expertise curve. You're always in the middle somewhere. And if we started doing metrics on this podcast, you'd probably find that this one ranked very highly in opinions expressed per minute. Ooh. What could he mean? Let's get into it. Announcer (00:01:21): Ladies and gentlemen, may I have your attention please? Announcer (00:01:25): This is the Raw Data By P3 Adaptive podcast. With your host, Rob Collie, and your cohost Thomas LaRock. Find out what the experts at P3 Adaptive can do for your business. Just go to p3adaptive.com. Raw Data by P3 Adaptive is data with the human element. Rob Collie (00:01:49): Welcome to the show. Alex Powers and Alex Dupler. How are you today, gentlemen? Alex Dupler (00:01:54): I'm doing great. It's great to chat with you. Alex Powers (00:01:56): Rob, back-to-back meetings. I'm glad that Luke found us some time here. I was so hesitant about this podcast, just cause I love listening to it. I was like, "I don't know, should I do it? Should I do it? Should I do it?" Rob Collie (00:02:08): The answer is yes, you should do it. Alex Powers (00:02:10): I appreciate Alex D and Rob just pulling us all together. Yeah. Rob Collie (00:02:13): We've already backstage a little bit been laughing about this. So let's bring it out to the front stage. The two of you combined, what do we refer to you as? Are you the Two Alex's? Or something different? Alex Dupler (00:02:23): So we learned separately from our wives that the correct pluralization is two Alex. Rob Collie (00:02:30): See, I just don't buy this. I still think Alex's. I mean, we could get really funky and say, Alexi. Tom LaRock (00:02:36): I was going to say, that's what I think. Yeah, Alexa, Rob Collie (00:02:39): But I mean, think of it this way. There's fish, and that's plural. But even there, there's still fishes, which refers to different species of fish. Yes. I think. Is that what it is? Alex Powers (00:02:51): Yeah, that's right. Fishy. Yes. Rob Collie (00:02:53): I don't know. So the two Alex, are you guys seriously going to go by that now? Is that going to be the new thing, or? Alex Dupler (00:02:58): Well, the YouTube channel is called Two Alex. Rob Collie (00:03:01): How'd the two of you come to know one another? Is it just like, oh, we're both working in data and we're both named Alex. So you're like, you see each other from across the room and your eyes meet across the internet? Alex Powers (00:03:10): I would say across the internet, for sure there. Just because he's up in Redmond, I'm kind of located in St. Louis, Missouri. From there it was kind of this, I think natural, just both being active in the community. Alex D you can keep me honest there, I'm sure we were connecting on Twitter a little bit there before, definitely in the subreddits. One of my earliest memories of was, Hey, this thing isn't folding. And I was like, oh my gosh, it's Power Query. I've got to tackle this. I've got to answer this question. Reddit is where I hang out at. I would say from there that's when we really started coming chat more and more, but Alex D I'll let you kind of tell your side of the story. Alex Dupler (00:03:43): Yeah. Yeah. My recollection is that the first time we interacted with each other, where it wasn't just some random poster on Reddit, was side conversations on Microsoft teams within Microsoft. There's some internal discussions where salespeople can get their question answered and sometimes the questions are interesting. And so, yeah we had some side conversations. Plus back then, when Alex wasn't on the product team, he didn't always have full visibility into the roadmap. And so we would chat on the side about what we would do with the roadmap. Not that we would do a better job, just a different job. Rob Collie (00:04:19): Yeah, I get you. Yeah, I understand. I understand. What are the two of your roles at Microsoft today? Alex Dupler (00:04:25): I work for Microsoft advertising. We're the organization that sells the ads that go on Bing, as well as some partner websites like Yahoo search and AOL search and stuff like that. And I work in the business function of the sales org. So I do BI for a sales team. And it just happens to be at Microsoft, and that influences the technologies that we use. IPM are like data warehouse and big cube stuff. Rob Collie (00:04:50): Cool. We're going to have to circle back to that for sure. And Alex P what are you up to these days? Alex Powers (00:04:56): Yep. So senior program manager on the Power BI customer advisory team, so PBI CAT for maybe those out in the community. I'm called as kind of that last bastion of hope sometimes, where I'm not very close to the solution, not close to the architecture, just come in and fix it. Where Alex D, he owns the solution, he owns the finished product. That's a line of visibility that I completely lose in my day to day. But you get variety, you get to do different things. Some days it's maybe a DAX challenge, next day I'm writing C#. The next day, I'm writing kind of new report, kind of clicky, clicky, draggy droppy experiences. So a vast rich tapestry of Power BI. Rob Collie (00:05:32): So you're on the CAT team with a number of people that have already been on the show, right? Adam Saxton, Casper, Chris Webb. You're part of that crew? Alex Powers (00:05:41): Yep. Rob Collie (00:05:41): I hear that that crew continues to expand, it's like this great gravitational attractor. It's like just hoovering all of these people. Let's just have it on the record. Does the Power BI CAT team have ambitions of world takeover? Alex Powers (00:05:53): Every day. And I think what you're seeing right now is a lot of formality. Community contributors, experts, decades of experience. They're now turning into bosses, they're now turning into managers. So they're getting further away from the technology and kind of now being people managers. I'm enjoying our livestream here because Rob is laughing. He's like, oh, I know that exact feeling. Rob Collie (00:06:14): I do. I do, right. I got a request today from some media outlet to interview me for Power BI tips. And I'm like, gosh folks, I'm probably not that person. You want to talk about strategy, okay, that's different. But I have gotten further and further. I still build some Power BI stuff for sure, for my own purposes. But I don't have that day to day, like, this is my life. That's not how my day goes anymore. I'm back to the management game after years of being out of it. Yeah. Growing a company tends to keep you out of the actual hands-dirty data trenches that started the whole thing. Alex Dupler (00:06:52): Well, if you ever start stack ranking, that's when it's going to be time to sell it. Rob Collie (00:06:56): True story, stack ranking was the reason why I actually stopped being a manager at Microsoft. At one point, I just said, I'm done applying the system for you. I was sick of it. And I understand it's gone now. I found out the hard way that stepping back from a management position didn't just relieve me of that stack ranking thing that I found immoral and uncool. It also took me out of a lot of the important conversations. I just didn't have nearly the input or influence that I had before. And that was hard. If I was still at Microsoft today, my career at Microsoft would still have suffered like a multi-year setback because of this era where I just said, I'm done. I know that at this point, the whole stack rank thing has been gone for a long time, but it was still a number of years later after I left that it still persisted. No, we're never going to do that. We're never going to play lifeboat with human beings. I mean, it really sucked, right? Basically, if you built a really good team, either by recruiting or by development or both, you were punished for it. Alex Dupler (00:08:04): Yeah. Apply this to Alex's team. You want to stack rank Chris Webb and Casper and Adam? Tom LaRock (00:08:09): I will. I'll do it. Rob Collie (00:08:11): Which one of them gets told that they had a terrible year? Right? Tom LaRock (00:08:16): I'd be happy to do it. Rob Collie (00:08:20): Hey, listen. As long as we put that kind of phenomenal power in the hands of a benevolent tyrant, like Tom, it's perfectly safe. What could go wrong? Alex Dupler (00:08:29): That is what they said about solar winds. Tom LaRock (00:08:34): My first criteria, having known them for many years, is Jaeger consumption. So we'll just start with that and work our way down the stack. Rob Collie (00:08:44): Which way are we going to sort that list though? We sort it largest to smallest, or smallest to largest? I mean, I could see that list being sorted either way. Tom LaRock (00:08:50): We'll try it both ways and see how it shakes out. Rob Collie (00:08:53): Yeah. I mean, it could be like a honeypot, right? Put some Jaeger out there, see who goes for it? You're getting the 3.0. We won't be doing any of that, thankfully. Now, Alex P, you were previously in a different role, right? Alex Powers (00:09:10): Yes. So, here at Microsoft less than two years, came in through the premier field engineer side to support, really had a blast there kind of proactive engagements training, probably train like 4,000 Tableau users on the Power BI. So just like the grind of doing it day in, day out, talking about the product, I just absolutely loved that. Transitioned to kind of field sales roles. There it's competitor competes, a lot of disinformation where they're saying, well, Power BI can never do this. What do you mean it can't do that? Here's an article. Here's me, kind of the whizzbang demo. That's probably where I got my hyperlink chops for those that kind of know me on the community. Alex Powers (00:09:44): This is the good and bad of the pandemic is like, Hey, we're making some career advancements, we're working long hours, whatever else it may be. A lot of my goal whiteboard over here was, Hey, I want to be on the Power BI CAT team. Had that visibility, just kind of did those grinding over the fall and winter months when we're all stuck inside. But I'm sorry, Thomas. I don't know how good I would be at the Jaeger thing, just because I don't have that peer connection. I haven't met my coworkers. So that's tough for a lot of people that I think are just making career jumps during the pandemic right now. Rob Collie (00:10:16): Yeah. I mean, it's weird. I live in a completely altered reality where we've been a hundred percent remote, I've been a hundred percent remote for 11 years. Probably more closer to 12, actually. Our company was a hundred percent remote from the beginning, basically out of necessity. To me, it's shocking how many people who've been at this company for a long time have never met each other face to face. We did a gathering, a team gathering in 2019. We didn't do one in 2020. I don't remember why we didn't do that. We haven't done one this year, either. We're hoping to maybe do one in 2022. We've hired so many people in the last year that there's like half the company that I haven't ever been in person with. Alex Powers (00:11:02): It's tough. Rob Collie (00:11:03): It's different, isn't it? Alex Powers (00:11:04): Yeah. I think it was like the good meme the other day where it's like, Hey, here's your company culture, it's just like an empty cubicle. And it's like, well, people don't even have that anymore. It's just, here's your new job, here's your new email. Log in, welcome to the company. Great friend of mine, Mark Beedle, I know kind of joined T3 adaptive. I love that he's like, this is where I want to be. I think of the P3 of the past, where you take the group, I think, up to Seattle or some of the different areas. And then it was like, oh wow, they're all getting together and having fun. You know, I tried applying for the job, but unfortunately your Excel file was corrupt and I couldn't pass the test. Rob Collie (00:11:36): Oh, I see. I see how this [crosstalk 00:11:38]. Alex Powers (00:11:38): Yeah, what happened with that, Rob? Rob Collie (00:11:39): I don't know, man. Alex Powers (00:11:40): That's really what I wanted to corner you on today. Rob Collie (00:11:43): That might've been part of the test, Alex. Alex Powers (00:11:45): I literally thought it was, that responded that way. I was like, I don't know if they're testing me with a corrupted file. Alex Dupler (00:11:50): Yeah. You need to have mastered the Open XML format of the Excel file, and be able to track down the corruption in the Power Query. Rob Collie (00:12:00): I saw a joke or a meme on some social media a couple of years ago about cast iron, the hipsters with their cast iron and how you have to take care of it and everything like that. And then after you're done with that, you have to dry it in the sun for 24 hours. And someone goes, 24 hours? And they go, yeah, if you're not willing to go to the Arctic, you don't deserve cast iron. So it's like that kind of test. Yeah. Alex Dupler (00:12:23): We beat the crap out of our cast iron, it's just fine. Rob Collie (00:12:26): Okay. And now Alex Dupler. You're working in BI in the advertising wing, within Bing but also the affiliated networks like Yahoo and things like that. And so you mentioned that you're in charge of the data warehouse and you're in charge of, you said big cube. Alex Dupler (00:12:42): Yeah. Rob Collie (00:12:43): For a year I worked on Bing, and maybe this is a completely different dataset than what you actually end up caring about, but the state of the world back then was there was this giant distributed commodity hardware database system, data storage system called Cosmos. Alex Dupler (00:12:58): Yep. Rob Collie (00:12:58): One of the world's foremost write-only data stores. It was amazing at storing data. You could never get anything useful out of it. There was only one person in the entire organization, named Jamie Buckley, who was capable of actually running queries against this thing. And so if you wanted any information whatsoever about what searches were being run and things like that, yeah sure, you could try to write a query against this thing. And what would happen is you'd get syntax error after syntax error after syntax error, and then eventually you kick off a query and it wouldn't give you any errors. And you're like sweet. And it would run and run and run and you go away and you'd come back like a day and a half later and then you'd get a runtime error. Alex Dupler (00:13:38): Yeah. And when it works, you get a CSV. And so we still have that. I think when I was getting trained on it, which they said it had something like 5% of the world's data in it. Cause it's not just Bing, it's X-Box and a whole bunch of stuff. It's this really cool exabyte scale thing. But nobody knows how to use it, partially because it uses scope scripts, which the only commercial product they've ever been used in is the ATLS gen one analytics feature, which was not a successful product and is being deprecated. And so you can't hire people that know how to use it, there's just like a bunch of vendors that have learned it. And I can't write it either. Also, I don't know if this was your experience, but the engineers are allergic to writing documentation. It's got these petabyte sized tables with 400 columns and there'll be a data dictionary and it doesn't have any descriptions of any of the columns. Rob Collie (00:14:33): This does match my experience, yes. Alex Dupler (00:14:35): So we use that some, we also have other partners. I mean, it's a huge organization. We just missed getting touted in the quarterly earnings as having crossed $10 billion for the last fiscal year. I think the public number is like 9.95 or 9.5 billion. Yeah so it's a real business, even though the market share is pretty small. It turns out advertising is just a really, really good business. So we take a bunch of data out of there, and then also from partners that take data in there, and put it all in Databricks and make it available to folks that way. And we love Databricks because our analysts, they can come with whatever skills they have, and they can be successful on day one. Because they don't have to learn SCOPE or KQL or whatever. Alex Dupler (00:15:22): They can write Python, they can write R, they can write SQL, there's a cube so they can do Power BI, they can do Excel. They can do whatever they want, all in the same data. Now, if they want to do things that are super fancy, they may have a hard time using the cube. So they got to write something. Rob Collie (00:15:41): Yeah. Alex Dupler (00:15:42): But if you're a PM owning a project, you can drag and drop in that cube all day long and have a good time. And then the other thing that we like about the setup we have is, with the data in Data Lake, our partners that have their own generous Azure budgets, they're not running queries against our server. Whereas if we put it in Synapse or SQL, when they want to query our data, we're paying for this compute. But here they just mount it onto their own compute system, and they pay for it. And that's great. We like when other people pay to use our data. Rob Collie (00:16:15): So it's funny, I actually expected that the answer to the question was going to be, oh no, no, we fixed all that. That original system is completely straightened out, it's got a much more human friendly interface. But it turns out that you just have other systems that are human friendly. And those things have to... on the order of one-time investments to figure out how to populate those things from the great Oracle that is Cosmos. Alex Dupler (00:16:41): Yeah that's largely true. I mean, in Cosmos, they've implemented the ATLS APIs. So you can mount data in Cosmos directly to a Spark engine and do stuff that way, if you want. Yeah. Basically that's how they've done things. You will not be surprised to learn that Microsoft likes to reuse names. Maybe you've seen this phenomenon before in the word power, but yeah. Cosmos, the internal exabyte scale data platform is not the same as Cosmos DB, the Azure product, which is for, I couldn't even describe it. It's for, like, everything. Rob Collie (00:17:19): Yeah. I mean, there's only so many cool nouns. And furthermore, the set of cool nouns in the world is further refined by the ones that computer scientists gravitate to. So you end up with a really small population of words. And the chances... It's like the pigeonhole principle from math, right? You need 450 names, you only have 300 words. So you're screwed. And so you end up with things like the word dashboard being repurposed to mean something kind of niche in Power BI. That's one that I wish we could get a do-over on. And you know, I'm a sinner. I named some things poorly in my day. I'll give you an example. When PowerPivot V1, and actually several versions of PowerPivot, at least in 2010, there were those two drop zones, extra drop zones in the pivot table field list, for slicers. Rob Collie (00:18:11): Cause Amir insisted that we make slicer layout really easy as opposed to tedious. So we had these extra drop zones, and one drop zone put the slicers down the left-hand side of the pivot table and one put them across the top of the pivot table. What did I name those two zones? Horizontal and vertical slicers. For years after that, when I taught that product to classes, they go, oh, what does a horizontal slicer do that's different than a vertical slicer? And I just sit there with my head in my hands like, it should have been left and top, Rob. Why did you... Previous Rob, why were you so nerdy and stupid at the same time? Left and top. Alex Dupler (00:18:48): Well you see, in an indimensional cube, there are some things that are horizontal and some things that are vertical. Once you understand what the tubal is, it'll all make sense. Rob Collie (00:18:59): Yes. So let's go back to basics and... Yeah, no. It's just left and top. Yep. These are what you call own goals. Can't make these things up. It's even funnier, by that point in my career when I made that mistake, I was already kind of like this rabid high priest of naming. Like, we should be better. And here I was in the course of delivering those sermons, just committing tremendous sins out the back of the church. It's just like. Alex Dupler (00:19:31): Yeah, it turns out we should be better in, oh crap, I got an hour before this presentation, what am I going to call this thing? Those are two overlapping states of being. Rob Collie (00:19:41): You know, people's hearts are in the right place. So I still think that the two of you probably might've gravitated toward each other just a little bit, maybe like 1% more, because of the shared first name. Can I be allowed like an extra 1% gravity on this? Alex Powers (00:19:54): 99. I mean, a lot of Alex's within Microsoft that are doing Power BI, we've all kind of banded together. Rob Collie (00:19:59): There's like an Alex crew? Alex Powers (00:20:01): Hell yeah. Big time. There's multiple Two Alex's, too. Rob Collie (00:20:04): As we've established, once you get above like three or four Alex, it's suddenly Alex's. That's when it becomes plural. Alex Dupler (00:20:10): There are at least two Alex's working at Microsoft in the Power BI ecosystem that are smarter than either of us. Rob Collie (00:20:17): Well I mean, going back to something we were talking about earlier, every single person, every single consultant at P3 is a hell of a lot better at Power BI than I ever was. I can't even argue that it's like, oh, I'm off my peak. It's not that at all. They were always going to be much, much better. It's very humbling. Like in the real sense of the word, when you sort of get put in your place. Alex Powers (00:20:40): Is this like a time thing, Rob? Cause I feel it too. It's like the early days, Power Pivot and Power Query were something like, I'm digging, I'm learning all of these things. And then like everything else is kind of passing me by and it's like, yeah I'll catch up to that at some point. And I see the wild stuff that people are doing nowadays, like, I don't know what nights and weekends they're spending learning this product, but I'm working twice as hard and I'm still not catching up. Alex Dupler (00:21:00): Yeah. I was watching the other demo the other day. And he was talking about how you should have your report and your data model in two separate PBIX's. This was Mike Carlo. It was a great demo. But then he was like, and to make this really easy, what we're going to do is we're going to edit the PBIX. And I was like, hold on a second. You can't do that. That's not allowed. Rob Collie (00:21:22): [crosstalk 00:21:22] Like actually hacking the file? Like he got into the file structure? Alex Dupler (00:21:25): Yeah. Rob Collie (00:21:26): I do love me some file hacking. For me, I think it's not necessarily a question of time. It's actually that the universe has returned to its default state with respect to me. Which is, the whole time I worked at Microsoft, in all the years I was on the engineering teams, I worked with plenty of people who were super technical, but also enthusiastically technical. When VB.Net came out, and ASP.NET, I had some colleagues that just dove into that. They loved it, it was the most amazing thing. And I just could never... I was still at that point going like, okay, well I learned how to write my VBA, and I'm sticking with it. That's where the frontier of my coding, actual procedural coding, is still VBA six. Rob Collie (00:22:09): For some reason, DAX and data modeling, as technical tools go, DAX and data modeling really, really spoke to me. Like I freaking loved it and still do, still do to this day. And Excel formulas are kind of the same thing, right? This is the handful of exceptional technologies that really seem to appeal to my nervous system, and none of the others do. And by the way, M is another example that does not appeal to me at all. Alex Powers (00:22:38): I am the opposite. I love M. Rob Collie (00:22:39): Really? You love M? Alex Powers (00:22:40): I love, love, love. Hell yeah. Rob Collie (00:22:42): You're not my species then, you're something completely different. Alex Dupler (00:22:47): So I think one of the big things that drew me to data modeling, so there's a lot of constraints. And with programming, it's like, there's such an open world. Like the only programming I could ever really get my head around was VBA. That's where I started. You didn't have to have a big, complicated object model. There was just Excel. That was your object model. And it made everything so much easier. And you're like, okay, well, what I'm trying to do is move these cells to those cells. And with data modeling, especially in Power BI, it's like, well, I need one column for these relationships. And I need these relationships to flow in one direction. The constraints make it a much more manageable problem, but also opens up room for more creativity. Rob Collie (00:23:30): I agree. And also VBA comes with a macro recorder, the world's greatest set of training wheels. It's like, if I want to build an app from scratch, I can't like act out like pantomime what the app will do, and have something spit out code for it. Alex Dupler (00:23:48): Draw some stick fingers in Figma and just drag them around, and get some code from that. Rob Collie (00:23:51): Yeah. It's like, mock up the UI in Balsamiq or something, or Vizio, and then start mashing on the screen with your finger and say, okay. And then speaking out loud, what should happen at that... there's no macro recording for actual software developers. Alex Dupler (00:24:04): I think we got to tell Charles that, that's what he's got to do with his AI driven power apps development. Rob Collie (00:24:09): Yeah. It's we need to turn this into a LARPing thing, right? You just act out the application in the real world with these cameras... Holo lens. There it is. We've solved the world's problems. Take that for low code development. Alex Powers (00:24:26): Well, I like how Power Automate's now watching your points and clicks, and generating flows for you. Rob Collie (00:24:32): See, I didn't know that it did that. Alex Powers (00:24:33): Oh yeah. You're training the machine. You don't even have to write the code anymore. It's like, oh, automation is here. It's really here now. Rob Collie (00:24:41): It's always a feel good moment to meet a fellow VBA 6-er. The world used to be lousy with us. We were just everywhere. It's kind of a dying art. Office has got this new JavaScript API, Office Scripts. That's incredible. Again, in theory. I haven't touched it, because it's not reaching out and grabbing me by the eyeballs. I'm tempted though. It's sort of like, oh, a new VBA six and they have a macro recorder and I'm like, okay, maybe, maybe. This might be the way I learn JavaScript someday, is Office Scripts. Alex Dupler (00:25:09): Yeah, that sounds like how I'd learn it, except Excel is dead to me. I mean, I use Excel for note taking and PM stuff, but data work, I don't use it. Because first of all, Power Query is the way to go. And in Excel, when you have Power Query over, you can't save the Excel file. Rob Collie (00:25:28): Really? Alex Dupler (00:25:29): Yeah, Power Query takes a lock, like a lot of the old school windows. And you can't get back to the main- Rob Collie (00:25:34): Modal window. Alex Dupler (00:25:35): Yeah. So you can't save, you can't refer back to the data. You can't open stuff. And it's not like Excel ever crashes when you're working with lots of data. So saving, it's not that important. And if you want to say, first you have to evaluate your queries or set them to disable load. But if you've already loaded some, if you do something to disable load, it destroys the cells. I just said, I'll do it all in Power BI. No more Excel. Not because there's anything wrong with Excel. It's just that that user experience was just so unacceptable to me. I lost so many hours of work. Tom LaRock (00:26:10): Wait, what do you mean, not that there's something wrong... Clearly there's something wrong with Excel. Alex Dupler (00:26:14): Yeah. Rob Collie (00:26:15): Alex, you're cut from a cloth that I understand very well. Your sarcastic cynicism is, ooh, it speaks to me. Yeah, we've come to the right place. Even I, team Excel guy, I am really on team Excel. I haven't written any DAX in the Excel environment in several years. It's all Power BI, all the time now. Alex Dupler (00:26:38): The other big thing is why would you want to write DAX in an environment that you can't schedule to refresh? Unless you don't have pro licenses, like... Alex Powers (00:26:47): Hold on, let me challenge you now. Here we go, this is a little taste of Two Alex. So I love Ken Puls, where he's saying, Hey, I don't want the heavy weight of Power BI. If I can do as much as possible within Excel, be it Power Query or even Power Pivot. I would agree that. Alex Powers (00:27:03): Be it kind of power query or even power pivot. I would agree that the development experience is severely lacking. That's not to say that the power BI side is the best in the world, obviously Dax studio, et cetera. But I would much rather take a lightweight application over a heavy one every day and then just import that data model into power BI when I'm ready. Rob Collie (00:27:19): To me, the primary value of these technologies in Excel is as an on-ramp to the power BI universe for the authors. Tomorrow's power BI authors are today living in Excel. And the reason, I've said this multiple times on this podcast of multiple different people at Microsoft, but the reason why I'm, I don't want to use like the passive aggressive version of the word disappointed. Let's use the completely neutral version of the word disappointed. The reason why I'm disappointed that there isn't more investment there is because that is the gateway drug, and as a universe, as a community, like we really need to care about bringing those new people on. And that's where they're going to come to. To tell those same people, "No, put Excel down and start learning this in a completely new environment," their immune systems reject that because they've been sold a million times on the idea that something's going to replace Excel. They know better by now. Rob Collie (00:28:23): But no one in that category, like the V lookup and pivot route, none of them resist the idea of there being crazy, powerful new versions and features of the things that they're already doing. You get them 48 hours into that new world, and they're more than happy to switch to the power BI environment. They're excited about it. Those same people who would have rejected it 48 hours before. You got to take them on that path and this thing not getting the love that I think it deserves, I understand it's from the perspective of our real production environment is the power BI environment. I get that. But the on-ramp, they are doing some things about that, even things that I didn't know, because they're targeted at people who don't know about this stuff and I already do. Brian, when he was on the podcast, was talking about how they're using machine learning, advanced clippy generation seven, to detect the people who should be interested in this stuff and sort of pointing them to power BI. And there actually was really good uptake of that. That feature didn't fire for me because I don't use V Lookup or regular pivot tables anymore. Alex Dupler (00:29:23): That's almost exactly the journey that I went on. Like many of your guests, I did not go to school for power BI. I actually, I went to school for chemistry and I worked as a chemist for a couple of years. I was doing lab work and I was very bad at lab work. I mean, I understood the chemistry, but I would break glassware that was expensive and stuff like that. Which when you make $15 an hour, breaking expensive glassware is a good way to get in trouble. So I was like "Okay, well I grew up in a very computer centric family. Maybe I can do some of this Excel stuff." And so I was doing visual basic, and we were doing some dashboards, like operational reporting. And I had Excel in this company. I loved the people there, but it was not a successful business. We had maybe a hundred thousand dollars in revenue per employee with high CapEx, because we had these big, expensive instruments that we had to buy and chemicals and all sorts of stuff, lots of HVAC. So there was not enough money to pay people to live in Seattle, so every office license was a battle. Alex Powers (00:30:31): Wow. Alex Dupler (00:30:31): I was looking at, okay, what can I do with Excel 2007, because we had some of that I think we had enough licenses, but it didn't really check. So we didn't really pay too much attention. But then I was like wanting to use power query because I had sort of discovered it was easier, but I couldn't. So I was like, "Okay, how do I get this macro to run as a service so that I can refresh these dashboards on these dowels that we bought second hand?" Rob Collie (00:31:01): You know, if it weren't for the a hundred thousand dollars of revenue per employee, at a certain point, that story sounded like season two of Breaking Bad. The HVAC, the cap ex, oh you mean a hundred thousand dollars per employee per week? Okay. Alex Dupler (00:31:18): No, no, no, per year. Rob Collie (00:31:19): Then it's meth. Alex Dupler (00:31:20): Yeah, no. So this is the environmental testing industry. And the way it works is your tests have to be defensible to the EPA. So the EPA puts up a spec and says the test needs to be done this way. And when it's done, it has these parameters in terms of statistical reusability. And that means that one lab's product is a commodity compared to the other lab's product. And so you can't get outside profits. All you can do is compete on service and price. And if you take a high CapEx business and bolted to professional services, you're not going to get good margins. Rob Collie (00:31:59): Unintended consequences of everything, right? Alex Dupler (00:32:01): Yeah. I mean, Rob, can you imagine your business, if you are charging professional services business model, but you bolted on a whole, huge amount of consumable costs to every delivery? Rob Collie (00:32:14): Yeah. It sounds like we can safely not choose the wine in front of me. Alex Dupler (00:32:19): That's how I first encountered the 2017 standalone web, maybe it was 2016. The first time power BI was split out. I was doing office 365 admins and I got like a push notification. I was like, "This is cool." And I built some stuff and I showed it to my manager and he was like, "That's cool. How much is it?" "$10 a month." "Nope, can't afford it." And that's when I started looking for jobs anywhere where they had good Excel people. Rob Collie (00:32:46): Yeah, and to put that in perspective, this is the punchline to many jokes when people ask us how much it is. We go, "It's $10 a month per user." We all just start laughing. Like, "Oh my God, it's like stealing. It's so cheap." Alex Dupler (00:32:58): I didn't even have that many users. Rob Collie (00:33:02): I mean, this might be $30 a month. You know, like, nope. Alex Dupler (00:33:06): No. Rob Collie (00:33:07): It's like when we first moved to Cleveland back in the day, it was right in the middle of the financial crisis. We were looking at real estate and everything. And there were houses for sale in Cleveland for $10,000, like $10,000. And I started laughing. I'm like, imagine the deal going down. This house has been on the market for 180 days at $10,000. And you come in and say, "Look, I've got a cash offer. I'm willing to pay asking price. But the grill out back? You need to leave it." And the owner's like, "Mmmmm." Alex Dupler (00:33:43): My wife's best friend lives in Cleveland and they recently bought a house. And so we looked at a bunch of Zillow listings. I'm like, "Oh man, we could pay cash. Move next door." And they're sort of north of the Cleveland Clinic, that super nice neighborhood up in there. I was like, "Oh yeah, we could buy a very nice house, but our family is not there." Also, have you looked at the weather? It's not Seattle. Rob Collie (00:34:05): No, it's not Seattle, but I'll tell you what, here's an interesting description of statistics. When we moved to Cleveland, it wasn't because we wanted to, it was because basically my kids had been taken to Cleveland and so we're trying to console ourselves. We're like, "Okay, well, okay. It's going to be colder. There's going to be snow. Okay, okay. But at least it isn't going to be as overcast." And then we looked it up and Cleveland has more overcast days per year than Seattle. So we were like, "Damn, that sucks." However, it turns out that the definition of overcast days is very, very, very important. Because like an overcast day in Cleveland is like 75% of the sky is covered by clouds. That's an overcast day. At no point in time ever is Cleveland under a one mile thick, oppressive blanket that starves you, where you don't even have any idea where the sun is in the sky. So number of days is one of those misleading statistics. Total amount of oppressive cloud cover needs to be a different statistic. Trust me, there's more sun in Cleveland on an ongoing basis than there is in Seattle. Those winter and fall months, man, those are rough. Alex Dupler (00:35:16): That's not the part that bothers me. I was born and raised in Seattle. It's the shoveling your driveway in March, that part of it. Rob Collie (00:35:24): You could just be the delinquents that we were and just get a four wheel drive vehicle and say, "Screw it." Alex Powers (00:35:31): You don't shovel in March. After February, you don't shovel. It's in my contract. I don't shovel after March 1st. That's it. Because it's going to melt. Alex Dupler (00:35:40): Eventually. Alex Powers (00:35:40): It'll melt by the end of the month. Rob Collie (00:35:43): By the end of the month. Alex Powers (00:35:44): By the end of the month, it'll be gone. The rainstorm's coming. Sunshine's going to happen. I ain't shoveling. No, I put that in my contract years ago. Rob Collie (00:35:53): Cleveland's where I learned the rule, we do not adopt dogs that require walking. Alex Powers (00:35:58): Yes. Rob Collie (00:35:58): They need to be able to go out in the backyard and come back in. I fell so many times on ice. I eventually got, they're like crampons essentially. Alex Dupler (00:36:06): Yak tracks? Rob Collie (00:36:08): Yak tracks, that's what they are. Yak tracks or something else. You can't intentionally slip on yak tracks. It's crazy. But without them, just any day now broken hip. Alex Dupler (00:36:19): Our friends that lived there, they just got a golden retriever. We met the puppy when we were visiting with them this summer. Very cute, but I think they have some of those walks in their future. Rob Collie (00:36:29): So you start looking for a job, that's what led you into Mount Redmond? Alex Dupler (00:36:34): Yeah, I literally went looking for jobs good with Excel in Seattle. I found a contract position into Microsoft, making sure that the salespeople were assigned to the right customers and got paid on the right quota because advertising the agency model, it makes that much more complicated. Because we were in this model where we'd try and keep all the customers of an agency with the same salesperson which makes a lot of sense, especially when you're the underdog and you have relatively few sales resources, you get more leverage. But customer's change agencies all the time, have no respect for our compensation cycles, and so it was quite the nightmare. Rob Collie (00:37:15): Yeah, I love that. Like, so here's how we'll define the world for our benefit, "Oh world, you did not get the message. World, please don't change. Don't have your own things going on." Yep, that sounds like a software engineering problem from the nineties back before the industry kind of got wiser. So you start talking about the show. It seems like with a format like that, it's got to wander, which is what our show does too, by the way. So what are some of the most entertaining or valuable corners that you found yourself wandering into over time? Alex Powers (00:37:48): I'm still excited with our first episode where we talked about kind of beyond the desktop where it's no longer just development in [inaudible 00:37:56] desktop. It now almost takes like five different applications to build something at scale, which is like a good and bad thing. Well, you're getting more tools, seeing new things faster, more performing, et cetera, but why do I need 10 tools? Can we solve that within the desktop application? And we just had a really good conversation, a lot of attendees there, providing their own thoughts. And it kind of comes back to like this overwhelming feeling of learning power BI it's. Like I have to learn 20 new things all the time, learning, learning, learning. It's just never ending. That was my key episode. Alex Dupler (00:38:26): I agree. I mean, I think that's been the central theme of the whole show. I mean, we did that first episode and then we've talked, we've had the same conversation about these tools in so many different contexts. What are the different ways to do dev ops in power BI? What are different ways to measure how you're doing in terms of the effectiveness of your models? And so all of that is sort of external to the desktop application. Alex Powers (00:38:52): I think the best part, too, is that we're not from these traditional backgrounds of 20 years of BI or 20 years of kind of dev ops. We're learning this real time, sharing our experiences of, Rob, I think you call us power users or business users that find these tools, that get empowered by this technology. That is the seat in which we sit in. Hey, I found Excel 2010 power query add in 2013, 2016. I'm fighting with my IT admins. Can you please just upgrade to the next monthly release that will solve all my problems? Where Alex D is on the other series fighting tooth and nail for a 2007 license. It's kind of funny to hear that conversation. Rob Collie (00:39:32): $10 a month. We need those charity commercials like Sally Struthers used to do. This is Alex Dupler. For $10 a month, less than the price of a cup of coffee, you could get him an O 365 license. Alex Dupler (00:39:50): Yeah. So we ended up getting some E3 licenses and some E1 licenses, which meant I could work in power query, not using my personal license, but using the company's license. And then when I tried to share it with my coworkers, they only had E1. They couldn't use desktop. They had to use power query online or Excel online. And there was no power query online. And so even once we sort of modernized, we installed like a windows server 2012 and it was already 2015 and I was okay, this is our modernization. Alex Powers (00:40:26): So Rob, I'm going to steal one of your quotes here if you don't mind. Rob Collie (00:40:30): No, please do. We have an open source quote license at P3. Alex Powers (00:40:35): Well, I'll buy you a 2007 Excel license too, if I have to. Rob Collie (00:40:39): Fantastic. Alex Powers (00:40:39): But one of the items you had said a long time ago, I believe it was either in your book or maybe some of the video recordings you used to do in the studio with a nice button up shirt. You said, "There are two types of people when it comes to technology, those who can see the possibilities and bring about change and those who are about to be affected by it." And I always look at this and I look at things like had kind of talked about with the Excel users who haven't even gotten to this experience yet. There is still somebody out there today that is using Excel 2007 and their employer or their this, their that, or whatever their situation they're red is like, "I just see this little rectangle. It hasn't changed. Why do you want me to go invest in any of this stuff?" Like how often are you still seeing this? Rob Collie (00:41:17): Anecdotally in our public classes, which I haven't taught in a while, I've taught one as recently as let's say two years ago. And when I taught public classes for P3, I still stubbornly insisted on using the Excel version of this stuff to teach. Again, because of that onboarding effect. Alex Powers (00:41:34): Yep. Rob Collie (00:41:34): I think I was the only one left at our company that was still stubbornly doing that and I wasn't bothering to argue with others and whatever. So I did this for years, probably the first one of those classes I taught would have been like in 2011. So fighting with different versions of Excel, all the different students showed up with, for a long time, that was like a quarter of the class. The instructions for the class were very clear, show up with this version or don't bother. And they'd show up and no, they had a version, didn't even have power pivot and couldn't get power pivot. And so it was so bad for a while, we would bring spare laptops. If we traveled to another city, we'd be lugging spare laptops with us and they'd just be there ready to go like the hot swap with a student. That problem really went away though. I reached the point where I'd survey everybody at the beginning of class, "What version of Excel are you on," or whatever. And everyone, every single person in the class would be on the basically some version of recent 365. Rob Collie (00:42:32): I really do think that the person who's trapped on 2007 or hell even 2010 or 2013, they're out there, but they are really a tiny, tiny fraction of the world now. Whereas that used to be an overwhelming problem. So it's really testament to how successful O 365. Alex Powers (00:42:52): I would agree. Rob Collie (00:42:53): It's like, I was kind of like cynically betting against it forever, like the tortoise and the hare. Like I woke up one day and that's the world. The world is O 365. I think everyone's on the modern, not everyone, but it rounds to everyone, is on the modern wave of the tools. But they're still shocked when I show them, when we show them, "Did you know that this is in here?" And they're just like, "What?" Alex Powers (00:43:19): How is that in here? Rob Collie (00:43:20): They get angry because they start to realize how much of their life they have lost by not being told. Alex Powers (00:43:27): What I was always seeing was people had to live in two worlds. Like I went to some of the Excel boot camps, Michael Alexander, absolutely transformed on my personal laptop. I'm having the best time of my life in these three-day boot camps. I'm loving, loving, loving. At the very end though, I have to go back to work on Monday. I saw what could be, and I'm now back to what is. And it's just very difficult to kind of live in that middle space. For those that are still out there and listening to this, Hey, look at your surroundings. Hopefully Office 365 is coming within your organization. But if not, kind of like Alex D's story, I just went looked somewhere else. I saw the future that was coming, and I bet it all myself and I went for it. And I think that me and him both kind of share those stories, too. Alex Dupler (00:44:08): Inside of Microsoft, in my little corner of the Microsoft that most people in Microsoft don't even know about, I put together a class that I've given a couple of times, Modern Excel for Managers. And basically I would just show them power query and X Lookup. We didn't even talk about Dax. But just to like get them thinking like, "Hey, if you're doing some annoying thing in Excel, maybe there's a couple ways to make it a little bit better. Maybe you've never even seen the formula bar before." I had one person that I worked with who I was like, I didn't handle it very well at first. But she was like, "Can you add these numbers together for me?" And I was like, "Yeah." Alex Powers (00:44:52): Just a standard Excel formula bar? Is that what you're talking about? Alex Dupler (00:44:55): She was like, "Can you show me the difference between these two numbers?" "I can do that for you, but here, let me come over here and show you something." Rob Collie (00:45:04): So there was another program manager on being, because I was such an Excel, I'd come from the Excel team, I'm such an Excel zealot, that all someone had to do was say that they needed Excel help and I was there. Alex Powers (00:45:17): Oh yeah. Rob Collie (00:45:18): This person, they developed a habit of having me do all of their Excel work for them. This is one of my peers. And then of course passing off the work as their own. Fine, I wasn't that career minded, really. Six months after this is when I volunteered to no longer be a manager. So climbing the corporate ladder wasn't some voracious appetite of mine. So, okay, fine, fine. I knew what was happening, but I was still okay because the Excel problems were so fun. Keep them coming. Then one day this person asked me for Excel help. And there were these two columns of numbers. And this person had subtracted column two from column one to create column three and then added up column three to get the difference. Alex Powers (00:46:04): I'm waiting for the reveal, because there's a big story here and I'm loving it right. Rob Collie (00:46:09): I said, "Well, you know, you could have just summed column one and column two, and then taken the difference between the two sums." And they said, "But wouldn't the answer be different?" There was this moment of silence. I'm looking. I'm looking at them. They're looking at me. I'm looking at them. They're looking at me. At that moment, they realized that they couldn't use me anymore because I was now dangerous. I now knew that they didn't know math. They didn't just not know spreadsheets, they didn't know math. They were exposed. This person is now an executive at Google. Tom LaRock (00:46:48): This person being the executive at Google. I have no doubt probably doesn't know math. However, as somebody who uses technology and knows that data can be dirty and whatnot, I would actually, if it was me Rob, I would say do it both ways and make sure the answers match. Because you know what? We both seen it where it didn't work out. Rob Collie (00:47:10): That's true. But like when you see all the numbers in front of you, you physically see them all. You've got access. There's nothing hidden going on here. Oh, by the way, Tom, what's your degree in again? Tom LaRock (00:47:21): I have a master's in mathematics from Washington State University. Rob Collie (00:47:23): Masters, yep. Yeah, the masters in math is what allows Tom to say, "I'm not sure." Tom LaRock (00:47:29): Now hold on. Hold on. We've seen it. We've seen it. Rob Collie (00:47:35): There's a name for this. It's like the distributive property or associative property or something. There's some property that we learned in middle school. Tom LaRock (00:47:41): See, that's math with paper and pencil. Now we're talking about using Excel for math. So the tool, there could be something like, "Hey wait," and that's why we tell you, "well, just do it both ways." Even Wayne Winston would probably say, "Yeah, well have two columns. They should match. If they don't match..." Rob Collie (00:47:58): No, no he would not, not in this particular case. Tom LaRock (00:48:01): You're right. He wouldn't. Rob Collie (00:48:02): Every time I tell this story, someone always sort of like takes a sympathetic stance towards the antagonist and I end up feeling like a heel. Tom LaRock (00:48:09): You shouldn't. You shouldn't. Rob Collie (00:48:12): But come on. Tom LaRock (00:48:15): I'm with you. I have no doubt that they don't know math because I come across the same people. I do. Rob Collie (00:48:22): It's think it's the intersection of all of those things, That I was being used the whole time. Tom LaRock (00:48:27): Yes. Rob Collie (00:48:29): Which I had kind of made my peace with. But then on top of that, this incredibly aggressive ladder climber, the kind of person who really was kind of like willing to climb over the bodies of their colleagues. There's something delicious about, even though I was the rube in the whole story up until a certain point. I was being taken advantage of and I knew it. But even me in that situation, there was that moment of just like jaw dropping dumbstruck, like just looking at this person going, "Oh my God, you did not do that." Alex Powers (00:49:09): I'm going to lift us up from the depths here of career and everything else. I thought you were going to take us into that they didn't use cell references, which I've seen people type in column A plus column B's value in an equals. And it's like, "Well, why didn't you just do A1 plus B1?" Mind was blown. So I love that those moments still exist and you find them out in the wild every once in a while. And it's not massive warehouse MPP processing, et cetera, et cetera, that everyone's like, "Oh, this is the," I call it the BI bubble. Everyone's out here living in the BI bubble, writing C sharp, doing tabular and coding, blah, blah, blah. People are still excited about the very simple things that technology can achieve for them. Alex Dupler (00:49:56): My in-laws, they own a brewery in Rinton and they make great beer. I offered to help my mother-in-law with some of her bookkeeping that she does on inventory. And she was showing me how she was doing it. And she was like, "Okay, I get these numbers in Excel. And then I get out my calculator." And I was like, "Okay, let me show you how you can do this differently." And I showed her. She was like, "No, no, that's going to be too hard. I'm going to stick with the calculator." And I was like, "Okay, that's fine." Alex Powers (00:50:20): I still get the, "I don't trust Excel, so I double check with the calculator." Rob Collie (00:50:25): My first exposure to that, I was in college. I was working for a construction management firm that was building the new chemistry building on Vanderbilt campus and I was working in the management trailer. I was sort of all purpose ... we called me the lackey. I would just do whatever anybody needed. Sometimes I'd go out in the building and take measurements for things or whatever. But most of the time, I was just doing paperwork and stuff. They turned over the spreadsheet for this latest change order to the project to Vanderbilt management. And the price tag, it was an Excel spreadsheet and it had a column of values that were summed and there was a number at the bottom of it. And I remember the guy Tony who worked for Vanderbilt going, "Well, someone's going to have to double check these numbers. We can't just pay this contract." And my boss was just looking at him going, "Come on. That's what the spreadsheet is for is for doing that." And Tony's like, "Yeah, yeah, yeah, yeah, I know. But still, I mean, we can't just pay this number." I can understand that stance a little better, anyway, I just looked like a giant meany. But remember. This was someone who was taking advantage of me. Alex Powers (00:51:36): I agree. I agree. Alex Dupler (00:51:38): One of the things that I wanted to touch on in this conversation, something you've brought up a lot, which is going from BI to taking action within the report. And I got to tell you, this concept terrifies me. As a BIPM, I'm terrified of it. And I totally agree that the value is there, but in the BI space, we are really bad at testing. And if I think about how going from, Hey, I've got a report and these are the numbers to someone's going to click a button and it's going to change something in a system of record, the level of quality and testing goes up and I think really threatens the quick solution thing that you've also talked about is your bread and butter of like, Hey, we're going to do this really fast and it's going to blow your mind. But if I got to throw all that testing in there to make sure I don't blow up your source system instead, I don't know how those two things coexist. Rob Collie (00:52:39): Yeah. that's a fair point. I mean, for a moment there, when you were saying the taking action part and this terrifies you, before I understood the subtleties of your point, I was going to make the joke like, "Oh, you want this to be like the psychic hotline. It's for entertainment purposes only. Please don't use this report to take any action." Alex Dupler (00:52:57): It does make my job easier, I will admit, but it is a little bit more nuanced than that. Rob Collie (00:53:02): Okay, okay, fine. So anyway, I still managed to sneak the joke in there without ... it's not a joke at your expense because your point is different. Okay, there's escalating versions of this with escalating versions of responsibility and test implications and things like that. So you can just start with report design and working backwards from the types of action, your constituents, the users of your report. In classes what I would teach this concept on the end of the last day, as sort of like a religious sermon. I would encourage people to think of the users of their reports as each one of them sitting in front of like some gigantic cartoonish bat computer looking thing with these giant oversized 1960s, pop art colored buttons and they're labeled things like "open more stores" or "adjust hours of locations" or "increase head count, reduce head count" or "change product mix" or whatever. It's actually kind of interesting, when you imagine e ... Rob Collie (00:54:03): ... mixed or whatever, right? It's actually interesting when you imagine it that way to give it that physical manifestation, it actually becomes a little bit easier, for me anyway, to imagine what these people can do, because every role in a company really has a finite number of actions that they can take. Now, finite in terms of their categories of actions. It's certainly infinite when you get into the details of what are you going to do. And if you start to think of them from that perspective and you think, okay, what I should do is build reports that advise them or at least are helping inform them as to which control they should touch on their dashboard and directionally which way they should move it. Rob Collie (00:54:45): And it sounds like not that important of a trick, not that powerful of a trick, but if you actually apply that methodology faithfully, you end up with a vastly different portfolio of reports that you have built. Even I, very often, don't live up to my own principle in this regard. Because it's so easy. It's so seductively easy. It's the path of least resistance to grab all the data, load it up, make the model, that's fun, and then it's like flowing downhill. It's just like, oh, this is the easy and fun part, right? And then inevitably, you just start slapping together some reports. And those reports, in some ways, are just exposing the coolness of what you've built. Rob Collie (00:55:29): Now, that still leads to some very, very useful things. That's mind-blowingly better than what you ended up with in the old dark ages of Excel or even traditional BI. But I mean, oh my God, we were just talking about it on the last podcast. Some of the things that I have seen in the world that were supposed to be helping people make decisions were better described as their opponents in the process. This report was something that you had to fight to figure out what you should do at your dashboard. Rob Collie (00:55:59): So even before we start with any sort of actual software integration and taking action and things like that, that's a really, I think, important religion to develop. And again, when you're at your best, your absolute gold medal in the Olympics celebrated by the world best, maybe 30% of your output will live up to this. You just can't, you can't execute that way all the time. It's really, really hard. But it's software development. You are building software when you're building reports. You should have the same sort of mindset, if you can, as the Power BI team has when they sit down to design a new feature in their software. Alex Dupler (00:56:38): I totally agree. One of the questions I've been asking a lot, because I've been working on reports for the salespeople to take to the customers is, what is the conversation you're going to have with the customer? Not, what is the metric, but how does this fit into the conversation? And part of this is because my superpower and my career is going and building tools for the thing I used to do. And I think a lot of BI people come from that, where they were in the business and they were doing a thing, they just started making the reports for that thing. And somewhere along the line, they either work away from it for too long or they solve those problems. They had to learn how to make reports for something they haven't done for years. And I think that's a difficult transition and one I've been going through. Alex Dupler (00:57:26): But yeah, learning how to ask questions of the user, because they're not just going to tell you... what they tell you isn't what they need. You have to learn how to learn from what they say, what they actually want. Rob Collie (00:57:42): Yeah, it's a fine art. And by the way, when you've been in "BI", building the same reports for a long time, generally speaking, looking backwards anyway, those reports also sucked because they were constrained by what was possible at the time. And so they were never very ambitious. And most of those reports amounted to... A lot of times they just amounted to the data dump import that's used for something else. It's just, again, it's the opponent. It's better than nothing, but it's meager, meager help. And suddenly you're given this brand new tool set that's capable of so much more. Rob Collie (00:58:22): And unfortunately what I see a lot of times, when you give Power BI to an IT department, they go, "Oh hot damn, the new SSRS." This is the new reporting services. We're going to use it like reporting services. Load that big one flat wide table and pigeonhole it as visualization. It's just like, "Come on." Alex Powers (00:58:45): I'm telling you my favorite DAX is always written from the IT department. It's just written like a massive sequel statement, 400 lines. None of it makes any sense. It's like, "Can we just calculate, maybe another table here or there." Alex Dupler (00:58:59): The folks coming from the IT department, the one thing they do have going for them is that they did learn to format their code. Sometimes people coming from the Excel world, they learned that they can't format their code. And so I'm not sure that I would agree that the worst DAX comes from the IT department. Because you take a DAX statement and you take all the formatting out, and you've just made it 10 times worse. Alex Powers (00:59:21): From readability, yeah, I would agree. Rob Collie (00:59:24): I gave a talk one time where I asked the trick question of, what's the number one programming language in the world? And the punchline is it's Excel formulas. In terms of usage, it's overwhelmingly Excel formulas. And then I showed them, the audience, just to underline it, I take a chunk of Java or something off the internet. I put it on the screen. And I take an Excel formula, a really complicated Excel formula, and do the same. Put it next to it. But when I put that Excel formula on the screen, it's formatted, it's indented, it's got line breaks and everything. So that it looks, and it does, it looks ... suddenly, it looks a lot more like the Java above it. And then I go, but of course, the Excel programmers, they like it the hard way. And so then I clicked next on the slide deck and it all squishes down into one paragraph. It was randomly line broken. Alex Dupler (01:00:11): So at some point, Excel started supporting white space and formulas. And I don't know when it was, but it used to be, if you did that, you'd just break the formula. At some point in the last five years, you can add line breaks and indentations to your Excel formulas. Now, the code editor makes that very difficult, but at least the formula will still work. Alex Powers (01:00:34): I was doing this in 2013 so it's been around for a while. Rob Collie (01:00:36): Why don't these tools make this automatic? Alex Powers (01:00:38): I would agree with that. Why doesn't... Whenever you hit that checkbox and Power BI, why doesn't it just format it right then and there? Alex Dupler (01:00:44): I don't know a power BI product team person. Alex Powers (01:00:50): Well, I'll tell you what: ideas.powerBI.com. Rob Collie (01:00:56): So there was something you guys were talking about earlier that I wanted to come back to. Having both come from non-traditional backgrounds, by the way, that's the majority of backgrounds. Alex Dupler (01:01:04): That's the traditional background for BI. Rob Collie (01:01:06): That's exactly. Oh, that's right. Non-traditional is the new traditional. So do either of you ever find yourself wrestling with some version of imposter syndrome because of that lack of a credentialed background. Does that ever haunt you at all or "Nope, not bothered"? Alex Dupler (01:01:22): I am optimistic as a fault. So I got to Microsoft and I was like, "Okay, great. I'm going to find the secret Excel people in the internal stuff." Similar thing with Power BI. I'm like, "Okay, great. I'm at Microsoft, I'm learning Power BI. There's got to be people that are better than me." And it's been almost five years that I've been here, one color badge or another. And except for the folks on the CAT team, I really haven't met that many people that are better at it than me. Now, on the other hand, when you look at Alex's team and you're like, "Okay, well, I've been reading his blog for years. I've been reading his blog for years. I've subscribed to his YouTube channel" All of a sudden that job intimidates me. And similar to Alex, I wouldn't mind working there, but man, that's quite the hall of heroes. I definitely feel some imposter syndrome when I'm like, "Okay, well I think I have some of the experience necessary to do this, but I am not Chris Webb." Alex Powers (01:02:25): So I'll speak from that end words like, "Hey, I'm surrounded by the experts." Within the industry, I'll say that because they still have their own challenges. They still have their own day to day where there is something simple that blows their mind once again. It's like, well, I didn't even know that that would have been there for the past 14 months. I come from an Office background. This is part of my journey. And so I learned Excel. I learned all the modern Excel capabilities. Learned Power BI, I'm learning Azure. I'm within that learn at all journey and I am not the expert. My job is to find expert answers as best as I possibly can. So this is advice for everyone and I am the best at what I am the best at today. Alex Powers (01:03:06): Those around me, who I can help, I absolutely love helping. So Rob, when you were talking about, "Hey, someone reaches out with a small question like you do want to jump in and you do want to help. Obviously, it comes back to a point of scalability like, well, can I help everyone in the world? No. I just have to find new ways to spread my message or get my voice out there. There are so many things I still want to learn tomorrow. There are so many things where I'm just pacing myself. I'm not going to go become the expert at C# to write DAX. Power BI should just be simpler. It should have an advanced ribbon. It should have all these capabilities built in like the ... Alex Powers (01:03:36): I'm really excited like the DAX auto generator coming out in the future here. I want to say February next year. I don't think it'll be that great on day one. Because obviously it depends upon an incredible model to get incredible system written DAX. But, my God, everything should be made simpler with technology. If we require experts to use this stuff, I'm sorry, it's just not for me. It's not where I want to be. I love the low code side of things. Alex Dupler (01:04:01): It doesn't have to be good. It just has to be the macro recorder. Coming back to what we were talking about before. Rob Collie (01:04:07): Like LARPing, acting it out, pantomime, HoloLens. That's where the industry's headed. Alex Powers (01:04:11): I love, Rob, that you also said, "Hey, don't write any M." I would agree. I want people to do as much as humanly possible with the lowest barrier for entry. Go, Thomas. Tom LaRock (01:04:23): So Alex P, when you just said that about being the best person, that's something I've talked about before as well. No matter what you have me here for and if I know what I'm doing or not, whatever it is that you need help with, I'm willing to help. And the fact is I'm the best person for what we're doing right now until the best person shows up. And that's the attitude, I think, any data professional has to have. Even with Rob, and the guy who couldn't do the math, he's the best person at Google right now until the best person shows up. Rob Collie (01:05:03): They're the best at getting other people to secretly do their bidding. Tom LaRock (01:05:08): Until somebody else shows up that does it better. Rob Collie (01:05:13): I like to think that I educated that person that day though. Tom LaRock (01:05:17): Oh, you sure did. Rob Collie (01:05:18): They learned something. Their Grinch brain grew two sizes that day. Tom LaRock (01:05:23): They learned they needed to put some distance between you and them. Rob Collie (01:05:26): That's right. They knew that they had to give me a wide berth from that point forward. Anyway. And they probably then also proactively went about character assassinating me behind the scenes in case I leaked that knowledge. Knowing this person, that probably happened. Alex Dupler (01:05:41): They didn't go Google order of operations and learned how to do a little bit of math. Rob Collie (01:05:47): How would they even know to Google for that? Like order of operations? Rob Collie (01:05:54): You said Alex, you're not the expert. Oh my gosh. You are an expert. You're not the expert. Of course. Okay. Like anybody that wants to call themselves "the expert" earns every bit of ridicule that's about to come their way. Right? And every bit of comeuppance that's about to come their way. Right? Every bit of it. Oh, eat it up. Right? You asked for this. So I agree. You're not the expert, but oh my gosh. Are you an expert? When we had Hugh Millen on the podcast, the ex NFL quarterback, he pointed out sometimes... I really like... Let's pick any activity that humans engage in. And let's assume that you could know the rank order of how good people are, which of course is itself a fallacy. Right? But there's one person in the world who is the best. Everyone else is in the middle somewhere. Rob Collie (01:06:42): And there's also someone that's the worst, but like the overwhelming majority of humanity is in the middle somewhere. So like just the fact that you can look up the ladder and see people that you think... And oftentimes people that you look up the ladder, you think they're up the ladder from you. They're not, they're not, when you really get up close and personal or more realistically they're up the ladder in some ways, it's not one ladder, it's actually a million parallel ladders and you've all got a coordinate on each axis. This is why I wanted to talk about the imposter syndrome thing, is that everyone that's listening is somewhere in the middle. We're all also somewhere in the middle and relative to where we all were, including me, when all we had was Excel like V-Lookup and single table pivots, Alex Powers (01:07:27): I just want to go on the record, I'm an index match guy, just so we can kind of clear the air. Rob Collie (01:07:32): This is another joke that I used to tell in my public classes. I raised my hand and say "okay"... so if you look up, right, right, right. But there's a few of you who are index matchers, right? Like 1 out of 7 people put their hand up. Right. And I go, "and everybody you work with knows you're an index matcher... You're really vocal. You're in everybody's little face about you being an index matcher, aren't you?" And their colleagues are sitting next to them laughing like "Yes!". Alex Dupler (01:07:58): There's one person who thinks it's actually supposed to be index match match. Alex Dupler (01:08:04): And then there's me on the other side, which is like, why are you using V-lookups? Have you used [inaudible 01:08:08] ? Rob Collie (01:08:13): Yes. But I have yet to meet the index matcher that isn't proclaiming to the world. Tom LaRock (01:08:19): Is that like the CrossFit of Excel? Tom LaRock (01:08:22): Did I tell you I do CrossFit? I wonder what the Venn diagram, the overlap of CrossFit and index matchers. I wonder... Rob Collie (01:08:35): You'd have to control for the Excel audience first. I'm not sure that in general, the index match crowd overlaps a lot with the CrossFit crowd. And where does Camaro ownership fit on this Venn diagram? Alex Powers (01:08:56): I've seen some stuff like, these are some old war stories you guys get ready to tell Rob Collie (01:09:02): And you know that. It's not like this is the first time you've come to terms with the fact that you have powers. Oh my God, I didn't even mean to do that. Your last name is Powers. I just said you had powers! Alex Powers (01:09:14): I think a lot of it comes back to... Just those of us who have been around for a while, those of us who are reading the books, you can't see it on the stream here, but Alex D. has got all of his books on the shelf. I have the exact same stuff in the background. I've got books on my desk. It's just like, we're always constantly learning. We're not stopping. I think when you look around, you're just like, there's so many people out in the community doing weird wild things. I would rather read their article, listen to their podcasts, watch their video. I'm still in my little nerd corner here with the things that still excite me. I like how you alluded to at the beginning, Rob, where yeah, that looks cool. That looks fun. It's shiny. It's awesome. But this is what excites me. That's what I still just love day in, day out. Alex Dupler (01:09:55): I'll also add that when Powers moved over to the Power BI CAT team, I saw that job opening and I thought about applying. And when I heard he was going to apply, I didn't even bother. Rob Collie (01:10:08): Well, you just mentioned earlier that this is Thunderdome. 2 power BI professionals enter, 1 power BI professional leaves. Alex Powers (01:10:15): I had the commitment to change my last name. So, for me, it was kind of a slam dunk. Rob Collie (01:10:20): You were talking about books, I think you're the only book review on Amazon that mentioned the intermission. Alex Powers (01:10:26): Yeah. Loved it. Rob Collie (01:10:27): That's why I remembered that review. I had no recollection of who had written it. Tom LaRock (01:10:32): I forgot the intermission. Rob Collie (01:10:33): When I saw the text of the review, I'm like, oh, that one, that one. Yes. Like halfway through the book, I put a one-page intermission in that said, "Hey, you can just put this book down, go do amazing things. You're already amazing. If you understand what's here, just know that what's ahead of you is also awesome, but don't feel like you have to judge yourself by how well you recall and master the things that follow this page. It's like this explicit reminder of always being in the middle somewhere, I noticed this in classes as well. People would, if I teach a two day class or whatever, people would tend to judge how well they understood it by the list of things that they didn't understand as opposed to, by the list of things that they did. And so, I also put this concept of intermission into my teaching, where I would be like building up this list of cool new capabilities that we're learning relative to regular Excel. Rob Collie (01:11:31): And then eventually near the end of day one, I draw a line under this and say, okay, "Everything above this line. I want you to leave here having understood. So we'll drill this. You'll ask me questions about it, whatever, everything below this line is gravy and more of a tour of what is possible if you remember that there were things like this that were possible below this line. Even if you don't remember exactly how to do them, you're going to be in good shape. But this above line is the foundation". It's basically the same concepts that happen pre the intermission page in that book. Rob Collie (01:12:04): I was really grateful. A couple of things just broke the right way writing that book. One was that I didn't choose to go with a traditional publisher. I decided to publish through Bill Jelen's company. And he asked me if I wanted a traditional editor and I said, nah, let's not do that. And the other one was is that I found that I wasn't able to write in the technical voice. I tried, I tried to emulate, again the imposter syndrome thing. You tried to emulate the stars. You try to emulate like every O'Reilly book you've ever picked up. And I just couldn't, it was exhausting. And so I wrote it in that quirky, like, why not put an intermission page in the middle of the book, right? That was just so I could get through it. Just so I could survive it. All these years later, it stands as one of the things that people really like about the book. And I'm just really grateful that people accepted it. It wasn't some stroke of genius, but I do look back on all that with a smile. Alex Powers (01:13:01): So I love that it is your voice throughout the book. Question for you, because I'm kind of the weird quirky style too, where I'm going to write something that really excites me and it's going to be fun, and I'm going to have a good time when I get to the bottom of this. Start out with a two paragraph thing that turns into like a 17 page diatribe. You're just like, ah, that's not what I wanted to do, but here I am. The style of reading that I enjoy most though, is the most dry technical, boring, where it's a labor, it's a labor to understand it. It's a labor to get to that final page. John Walkenbach was that for me, from the Excel world where you would talk about O'Reilly, what is your personal style of learning? Do you still enjoy like a deep spec page and getting into the nuts and bolts? Talk to me about that. Rob Collie (01:13:44): I'm just kind of a terrible learner, it turns out. And this is why my adoption of something like Power BI, a DAX, a data modeling to me just like so extraordinary. This is something that, I couldn't help it. I also think benefited tremendously from the fact that there weren't a lot of people in the space when I started. There wasn't anyone to judge myself against, really. I was the only person at that point in time, like back in 2010, I was the only person who was living this every single day. That was their only thing they were doing. Even back then, like Chris Webb, like he was still doing a lot of MDX, the Italians, Alberto, Marco, they were still doing a lot of MDX. They had a lot of existing clients and I had a clean slate to do nothing but this, this isn't a strength of mine. This isn't something that I'm proud of. I want to think that I would, but I'm not positive that I would throw myself into this stuff with the same vigor today because I would be judging myself against those who went before me. Whereas before in 2010, I got to treat it like an adventure. Like I was out there discovering the frontier. I was like Lewis and Clark. And I don't consume very much. I'm not an avid consumer of YouTube. I don't really read books. I don't read blogs. I could paint a very negative picture of myself, for the listening audience, if I wanted, I think maybe the positive way to describe it is that I found that I'm at my best like in a phone booth, like very focused. Rob Collie (01:15:12): So I've got to be hands-on, I've got to be struggling against it myself. And then I will go and run the search that's looking for a particular technique, but I don't even like YouTube as how-to, I don't have the patience to watch a video. It seems like 99% of humanity does. I have such bad ADD, like I'm trying to scroll the page, get to the point. Where's the answer. You know, I don't have much success scrolling through YouTube fast-forwarding videos. I always missed the point that I needed. It's like trying to fast forward commercials. I'm always over running. Rob Collie (01:15:45): I don't think anyone should ever try to actively emulate my learning style because it's just not a very good learning style, but because it's such a poor learning style, it made this stuff seem even more special to me because even I, with my awful learning style, devoured this stuff. A true statement about this was that even as I was doing all of this, people were telling me I should write a book. And I was saying, "no, I shouldn't". If I understand this, everyone understands this. Alex Powers (01:16:12): Yeah. That's always tough. You always think that too. We're like, "well, everyone knows this". That is like 1% of the time that that ever is the actual case, is what I'm finding day in and day out. Alex Dupler (01:16:25): We were talking about digging in and helping users. And this has become a really big part of how I run our BI practice, which is that I'm a PM building tools to make analysts successful. The way that I know what tools to build is I spend most of my time helping them be successful, building stuff. I don't have a calendar full of meetings with various stakeholders. I keep my calendar open. And then any time someone on our broader team of analysts has problems, I try and help them. Sometimes it's like, well, you just need to learn this function. And sometimes it's like, if we just added a table to the [inaudible 01:17:08] , your life would be much easier. I'm going to put that in. And, sure, it's going to be 4 months before we get around to it. But the next time that problem comes up, I know exactly that we've covered that scenario. So I feel like that's been a really successful strategy for us. Alex Dupler (01:17:24): Now, the downside of this, and this is something that Seth and Mike were giving me a hard time on the other day on Twitter, is that I don't create a lot of documentation because every one of those conversations is unique. I could write it all down and tell them, "Hey, just go read the docs", but then I don't learn what they're trying to learn. And I really struggle with that trade-off. Rob Collie (01:17:45): At least, you know, there's always a job for you as a Cosmos engineer. Have you thought about getting into software development? I mean like really... Documentation is overrated too, right? For end users, it is. Cause again, the end-users are flawed, right? They're humans, we're not talking about an API here. We're talking about tasks. So if you really wanted to write documentation for your data models and stuff, it's basically a book. It needs to have a narrative. It's pick up a copy of my book and go, "okay, this book is written about a system and it's got a very deliberate reveal order. And it talks about the human problems that need to be solved and why you have to make your peace with this". And you have to help guide them down the path. To be successful documentation for something as complicated as a sophisticated data model, it's more like a book than docs. And then, you know what's going to happen? Two-thirds to three-quarters of people aren't even going to pick up the book. So I'm not giving you a license to never write documents, but I'm giving you a license to not write documents. Alex Dupler (01:18:57): Well, one of the things that I've struggled with, because everybody's like, "oh, do you have a data dictionary?". And I'm like, 90% of the questions that we get, it's about the glue. It doesn't fit in that column, name, description framework. It's like, Hey, how do these things fit together? Why is this missing? That doesn't fit in a data dictionary. The other thing is of course reading the documentation is like a superpower, Alex, you clearly have that in spades, but lots of folks, like, they're not going to go looking for it. Tom LaRock (01:19:27): They're not going to look for it because they're not expecting it that it exists. Alex Dupler (01:19:30): But even the power BI documentation, they're like, "Hey, how do I do this thing?". I was like, "well, did you try looking same period last year, did you try looking for a function to do prior year?" It's like, "no". Oh, well. Rob Collie (01:19:47): And then you searched the Microsoft documentation for the ALL function. You're a brand new user. You're the person who doesn't understand the ALL function. So by definition, you're the person who should not be told "this is the table function". That's a terrible way to introduce it to someone. Okay, it's technically true, and I know that on Reddit, that's the best kind of true, but it's also not helpful. Alex Powers (01:20:12): It's funny. Cause you're talking about docs. Thomas had brought up as well. I'm imagining someone going out to Microsoft docs and typing 'Power BI ALL', how successful is that search result going to be? And then it comes back to people like Rob coming up with these weird wacky names, a data flow, a this a that, how are people going to learn these? Is it with a space? Is it not a space? Terminology is hard, just in the product. And then like you had said, Alex, where would they even go with his basic knowledge to get a correct answer? Rob Collie (01:20:44): Let's also not focus on the downside. We're talking about the no docs, we're having a discussion about whether that's okay. Let's just make our peace with it. It's kind of okay. Kind of not okay. But, look at the upside of what you've been doing. You've been having all of these dynamic interactive conversations with all of these users. Rob Collie (01:21:02): Interactive conversations with all of these users and that advances the art. That advances things so every one of these people is capable of engaging with and wants to engage with a conversation. And when they're not understanding it, most people will admit it. Those who don't, you can usually read it on their face. You get that two way conversation. The opposite approach would be like, oh, write the docs and I never need to talk to anybody, right? Okay. If you had to pick one extreme or the other for something like this, and we're not talking about an API, right? It'd be different if it were an API, right? Rob Collie (01:21:33): And even APIs with their documentation still require books and YouTube videos and training and all that kind of stuff. The amount of sort of human good that you create with this interactive model, this open door policy and helping people directly. I don't think we should get like target fixation on the downsides of this. The upsides of this are enormous and it's got to be super, super gratifying for you. Alex Dupler (01:22:00): Yeah. I have a lot of fun. Rob Collie (01:22:00): It makes you want to do your job as opposed to hate your job. On another podcast that we haven't released yet, it's coming up, we talk a little bit about how it's a better strategy to lean into one's own strengths than to try to mitigate one's own weaknesses. You don't have an excuse. It's not like you get to ignore your weaknesses, right? But you can't get through life mitigating weakness. Alex Dupler (01:22:27): Yeah. Yeah. You got to strive for being T-shaped. It's sort of what you were saying before, which is like, "Hey, there's a list of things you got to know, and then you need to know what's possible." If you come into a circumstance where you need to write some custom M, go look it up. But you got to know that, "Hey, there's a script back there. And if I get into real trouble, maybe there's a solution." Rob Collie (01:22:50): Conversation beats broadcast every single time and docs are broadcast. So ideally you have both. If you have to choose though, conversation wins seven days a week, twice on Sunday. Alex Dupler (01:23:02): The other story that I wanted to tell, which ties right into this, which is that when you're doing BI, it's really important to know the business. So I grew up in the Pacific Northwest. My dad was Boeing IT for many, many years. He wasn't in the BI space. He was in networking and communications. So in the eighties he was working on a DOD contract. The admiral he was working with told them he was the first admiral in the whole Navy that could email with his project manager back and forth because they were the first ones to get email that worked across the networks. And then later he was working in the Bellevue campus, so right next to where Advanta is, where part of the Power BI team is. Alex Dupler (01:23:45): His office was in the Darth Vader building right across from there. And I remember going to his office, this must've been in '93, '94, and under his desk he had a server. And he said, "Oh, that's got our pilot for Exchange." It was the first beta version of Exchange. And he was running, self-hosting it in his office for Boeing. So they were one of the first customers getting in there. But he used to rail against that Bellevue campus, partially because the traffic sucked but mostly because they had IT in Bellevue and they don't make any planes in Bellevue. And he always thought that that was one of their biggest mistakes was not having the IT folks where the business was. And I think that directly translates to BI. Rob Collie (01:24:34): I want to go back to '93 or '94 and tell your dad, "Oh, you think traffic is bad?" Alex Powers (01:24:44): It was. Rob Collie (01:24:44): Just wait. Alex Dupler (01:24:46): We lifted in View Ridge and sort of north Seattle, and so to get to Bellevue, he'd have to do one of the two bridges. And yeah, obviously it's so much worse now. Rob Collie (01:24:58): I mean Microsoft still had like 10,000 employees at that point in time. When I got there in '96, they had 17,000 in the Puget Sound, right? It's like, we're just getting started. This virus is really just come to shore. Alex Dupler (01:25:11): Jeff Bezos hadn't even moved to town yet. Expedia was still a twinkle in someone's eye. Rob Collie (01:25:17): This is a very powerful concept. It's super important, right? This goes hand in hand with that other question about non-traditional backgrounds. Alex Dupler (01:25:24): Yeah. Rob Collie (01:25:24): Being the tweener, having the business domain in your head and the technical skills to execute is everything. My old joke, and it's not even a joke, I think it's the truth, that greater than 99% of a traditional BI project was just lost on communication of requirements and clarification and mistransmission and misunderstanding and iterating needlessly because human beings don't merge brains. If we had Vulcans with mind-meld, we'd be different, right, but we don't have that. We have a very narrow bandwidth between us, unfortunately. Alex Dupler (01:25:58): If anything, it's worse now because the tools have gotten so much more powerful, but we still have the same communication challenges that we always did. Rob Collie (01:26:07): Yeah, but I think you're closer though. You're closer, right? You understand the business. Alex Dupler (01:26:11): Yes. Some of the self-serve revolution has also made more business people do the work, but in general it used to be that you'd spend all this time getting the requirements right, and then you'd have to go write code for a month. And now you spend all this time getting requirements right, and then you would drag and drop a few fields. And so it takes so much less time to build the thing that the communication is an even bigger percentage of the job. Rob Collie (01:26:37): Right. Alex Dupler (01:26:37): And yes, you can get a nice feedback cycle going, but that still means that it's the communication that's the job. Rob Collie (01:26:45): Yeah. But you're not doing it that way, right? You've already described how you work. You're not going out, sitting down with them and saying, "Hey, write me a doc. I'll be back next week." Alex Dupler (01:26:54): No. Rob Collie (01:26:55): You're building in real time in front of them and saying, "Oh, do you mean like this?" Right? Picture worth a thousand words and all of that. You're not writing documentation on the data model. You're also definitely not writing requirements documents. Alex Dupler (01:27:07): No. Rob Collie (01:27:09): So the proof is in the pudding. Tom LaRock (01:27:10): So what you're missing is a layer, an organizational layer, of what I would call is a functional analyst. And it doesn't exist for the reasons you've stated. The tools are easier to use, it's self service, people can do these things. So you've lost that bit of communication because traditionally, I would say you would have people there who were the functional analyst that would bridge the gap and with them gone, this is the pain that you feel. You are essentially the functional analyst. You just don't have that title. Alex Dupler (01:27:40): No, we do have analysts. That was when I first went from Fender to Blue Badge, I was an analyst focused on a particular sales group. And I did quota setting and some performance reporting, and now I'm a PM for the tools for that group of analysts. And so I don't actually build any reports, zero visual content, only database layers. Tom LaRock (01:28:06): No, I was talking more about the business knowledge that you had touched upon. The person that knows the business and knows the technology enough and can speak to both groups and say, "I know what's possible. I know what you need, and I know what's possible and I know how to get it done. And I can even write something for you so that you'll understand what we're doing and why." And I think that layer has disappeared over a couple of decades as tools make it easier for somebody to just sit down and build shit for themselves. Rob Collie (01:28:35): Let's paint that in, I think, a more positive light. That role, this sort of middle layer, was always inefficient, and that role has just migrated into someone that's embedded in the business. Tom LaRock (01:28:49): They have a different title. They do something else. Rob Collie (01:28:53): Yeah. They're the ambassador, right? Tom LaRock (01:28:56): But there's still a communication gap because there's nobody above them saying, "Hey, these are the three skills that we need to bridge the gap." We need to have people doing that. I don't care what they're called and that doesn't exist, and therefore we still have this gap and the gap is widening. Rob Collie (01:29:15): I disagree. I think the communication gap is dramatically smaller and it's shrinking every day as we get more and more people skilled up on this stuff within the business. I don't think we've lost anything. And when was the last time that someone up the chain from you actually knew more about it than... That doesn't usually happen, right? Tom LaRock (01:29:34): I didn't mean to imply that somebody up the chain knew more. It was somebody up the chain understood the value of the role and said, "I got to make sure I have a team of these people." Rob Collie (01:29:42): Oh, right, right, right. We talked about a little bit earlier, right? In essence, I wasn't properly valued where I was for a number of different reasons, and so I went looking for a place where I was properly valued. Well, this is the story of where P3 employees come from. Tom LaRock (01:29:59): The downtrodden. Rob Collie (01:30:01): I mean, so it's kind of good news, bad news, right? It's good news for P3. The world keeps manufacturing underappreciated, awesome people, right? Bad news, it's like, oh, that's really kind of human sad. I mean, it's important to our business, but it's human sad. Tom LaRock (01:30:18): It's important for [inaudible 01:30:20]. Alex Dupler (01:30:24): Well also I think it's amazing how much, when you learn these tools, how quickly you can go from contributing at one level to contributing at a whole nother level. I'm not really a big fan of the 10X engineer thing, but I definitely think from before I learned Power Query to after I learned Power BI, I can do the same work in a 10th of the time. Now my ability to focus over the course of a year, I'm not 10 times more productive, but in short bursts, definitely that was a 10X. The people that learn those skills, the value that they can create in a single business in that single role, I think can quickly outstrip the sort of scope and business model associated with that role. That was part of why I left the lab. It was like, well, listen, this business model only supports a certain salary and that's not going to cut it so. Rob Collie (01:31:22): There's a quote, it's probably falsely attributed to but it's attributed to Stalin. The Germans were building these incredibly high quality machines, it was like the Mercedes of tanks. And the Soviets were just cranking out just tons and tons and tons of these T-34s that were meant to run for a week before or whatever. And he said, apparently, again who knows, he apparently said, "Quantity has a quality all its own." And they won by the way. You're talking about the 10X thing, right? Alex Dupler (01:31:58): Yeah. Rob Collie (01:31:58): I want to come back to that and say that it might be that 10X is real, even when you account for friction losses and attention span and things like that. Okay. There's a point at which something is so much faster that it's not that you get to the same place 10 times faster, you get to a place you never, ever would have had the ambition to go. Alex Dupler (01:32:21): Yeah. Rob Collie (01:32:22): And in terms of your impact on the business, I believe this stuff is oftentimes more than 10 times faster, honestly. Let's accept it as 10 for the moment. That's ambitious enough. I think the impact on the business, the value of a Power BI empowered human being that's embedded in the business and understands the business domain of the department or whatever is greater than 10 times what the original value of an Excel powered version of that same person. Even if they're not, again, due to friction losses, they're not 10 times as productive in terms of whatever, right? Rob Collie (01:32:56): The quality that comes out of that quantity is different and is far, far, far superior. It reflects a question formulation muscle that is rewarded by being exercised. You ask far better questions than you ever would have dared to formulate before because it would just been discouraging and depressing to formulate those questions that you knew you'd never be able to answer. Now it's practically a lay-up. It's an addictive positive cycle. So I'm going to take your less than 10X and go I disagree. I think it's more than 10X. Alex Dupler (01:33:36): Well, I definitely agree that there's a quantity piece, which is, I think about before scheduled refresh, how many reports could one analyst support? It's like, well, okay, at some point all they do is refresh the same reports. Rob Collie (01:33:49): That's right. Alex Dupler (01:33:50): And sure, there's a point with automatic refresh of reports where all you do is serve as tickets on those reports, but that number is way higher. Rob Collie (01:33:59): Yeah. And let's be clear. It's not just scheduled refresh. It's the fact that everything in a Power BI model and report is built from the beginning to be one click refresh all, right? That's not how Excel reports work. There's no one-click anywhere right? But then you can schedule it to click that button itself. Oh wow, that's cool. There's a certain point at which an Excel analyst is now sunk by the weight of their own spreadsheet portfolio. Just carrying it around with them and turning the crank on it, they're out of time in the week. And so the joke there is, if you're good at Excel, if you're an Excel analyst and you're good at it, what's the worst thing you can ever do for yourself. And everyone always says, "You can tell people you're good at Excel." And I say, "Yeah, but that ship sailed." The worst thing you can do is to build a spreadsheet that people find really valuable. Your success is going to become your luggage. It's become your ball and chain, right? And so you're actually disincentivized from ever creating another spreadsheet because you know better. Alex Powers (01:34:56): I feel like you're speaking to my soul right now, Rob. You're just, I've been here before, but I work in a spec factory too where I wasn't part of the business and it was, "Here are docs, here are specs, just go build it. We don't care." You have no context, and then make sure that you hit that home run with whatever report you're building. You get very good being in those high pressure situations, but I didn't have that connection to the business. I understand that it is valuable, but I don't always think that it is the most important thing in the world if you have someone with the expertise to come in for those last mile efforts because at some point those business analysts, they'll kind of hit that brick wall of capabilities and like, "Oh, we want to scale this. We want to get this to the next level." I think you still need those other people too, accurate or not. And that gap will be closed over time, but that's what I'm still seeing, at least from where we're at today. Rob Collie (01:35:48): We still need people. Alex Dupler (01:35:50): Yeah. Tom LaRock (01:35:50): For sure. Rob Collie (01:35:51): I like that. I wake up every day and check has the general AI come yet? Nope. Okay. So we're still needed. Alex Dupler (01:36:00): So Rob, if you've got some more time, there's one other area I wanted to ask you about, and this is a callback to one of your first episodes, which I listened to not that long ago, but I think it was Brad and Kai, I think were their names. Rob Collie (01:36:12): Yeah. Alex Dupler (01:36:12): But one of the things you talked about was working on building a report package around looking at advertising performance, and one of the project I'm working on right now, traditionally, I focused on internal business performance. What was the change in revenue? But now I'm starting to work on how can we use a template app type experience for our salespeople so they can build on demand, a Power BI app for one customer so they can have conversations with the customer? And I am really curious one, how that venture has gone, two, what are the things that are most important in that session or that project? Rob Collie (01:36:52): The problem with that ambitious project is that we said as a first step, we're going to get really, really, really solidified for our own stuff. We're going to get our own P3 advertising optimized. That kind of put an impossible ever-moving goalpost between us and that project. Some other things changed too, but we're still working with those folks. So we haven't really gotten to that. I'll be perfectly honest. It hasn't even really started. I think the biggest challenge with any effort like that, were we to go down that route, is that I don't think you can templatize it because so much of the important data is going to be on that customer's side of the fence, right? The actual performance of, for example, for us, the actual performance of this advertising is dependent on what happens to the lead or the customer after they start to work with us. Alex Dupler (01:37:46): Yeah. Rob Collie (01:37:46): And only we know that. Our advertising data model unsurprisingly spices together advertiser data, like from AdWords, with our own CRM and accounting data to calculate how profitable our... We're not running web commerce, right? Alex Dupler (01:38:05): Right. Rob Collie (01:38:06): We're not running e-commerce. It's not like we're selling widgets that the customer buys it and then we say, "Oh, well they bought a $500 product and we paid $300 for..." or whatever, right? It's not like that. It's a long running thing. But even if we were doing in e-commerce, even then, right, there's a long-term value of this customer. Maybe it's profitable to only break even on their first purchase. It's not a last mile, it's like a last marathon. And of course, Power BI is really good at this. Splicing silos, that's a Power BI strength for the ages, but you have to do it. I mean, even in our stuff, there's some very complicated SQL that sort of calculates in real time, our best guess as to what these leads will be worth because their long-term value is never known for sure. Alex Dupler (01:38:52): Right. Well, and if you want to optimize a campaign, you can't wait three months to get a signal. Rob Collie (01:38:56): That's right. Yeah. That's exactly right. That's the old knock on BI as a rear view mirror, right? If you only get the information when it's too late to act on it, you're never able to steer. So yeah, every time that that data model refreshes, it's making a statistically historically informed set of estimates as to how those campaigns are performing. Alex Dupler (01:39:17): One of the ideas that we've been tossing around a few years back, there was this Azure solution templates thing where you could click a few buttons and spin up an ADF and a SQL DB and a Power BI report, all sort of in one thing. And so one thing that we could do, because we're designing this initial version based on our internal DBs, and we can't expose that. Also lets us expose some data that we can't put in a public API. A lot of customers want to know what their market share is or their share of voice. And we're not going to tell you the denominator for the total marketplace. That's just not going to happen. We're happy to tell you, "Oh yeah, you're at 10%. And if you bid a little more, you could pass your rival there at 12." We love that sales pitch. Alex Dupler (01:40:04): But one of the things we've played with is we've got this API that you can get data out and it's not a REST API. It's a SOAP API because it's Microsoft, which is one reason why I don't think we can build a custom connector to put on it. I talked to Matt Masson about it and he was like, "I remember that. That's a SOAP API. You don't want to do that." And if we do leave, they said Matt Masson is one of the engineers on Power Query, one of the most senior Power Query engineers. Alex Dupler (01:40:31): So what we were thinking about was, okay, we've got these Python scripts that our support folks wrote to iterate through the API so that they could get data out in bulk. Could we make a version of this that runs in an Azure function or ADF and dumps into a SQL DB that the customer owns and then give them some out-of-the-box reports? Hell, maybe we can integrate Google and Facebook data too because we won't forget to put our data on the first page whereas if you were designing a rational product, you wouldn't necessarily put somebody who's got... Well, we've got 30% share on Desktop, but Desktop isn't the whole ballgame. Rob Collie (01:41:10): It's purely alphabetical. Alex Dupler (01:41:11): Yeah. That's right. Rob Collie (01:41:13): Bing, Facebook, Google. It's defensible. Alex Dupler (01:41:15): Yeah. Yeah. So in any of that, we have thought about maybe we should build some reporting, not to be biased just so we can make sure that we're in the conversation. But automating these APIs and putting them in Azure partially so that we can say, "Oh yeah, 50 bucks a month of Azure, and you'll get all your data in this beautiful format. It's not just API calls. It's a tabular model and here's Power BI. And for another $10 a month, you can have this thing going and share it with everybody. And do you need to build some custom reports and integrate some stuff? Sure. But you're halfway there." Rob Collie (01:41:47): That architecture stuff, I'm the wrong person to ask about. No, but if you would like to hire P3 for a strategic... So we have people for that, there's an app for that. There's definitely people for that. For our stuff we actually use a third party service called Stitch that extracts data from the AdWords API and dumps it into, guess what, Azure data warehouse. For a while there we were advertising on Bing, but we stopped because the Stitch integration with Bing didn't give us the click IDs. Alex Dupler (01:42:17): Yeah. I've heard that before. Rob Collie (01:42:18): So we couldn't track anything with any sort of... And again, this is an instance where, because we couldn't evaluate its effectiveness, we actually stopped doing something. It's not a Bing problem I don't think, it's a Stitch problem. Alex Dupler (01:42:30): No, it's definitely a Bing problem. Rob Collie (01:42:32): Oh, fantastic. So Microsoft should fix that and then Stitch will do it and we'll go back to advertising. But there's an argument to make, right? There are people who are not advertising on that platform simply because they can't report on it. Alex Dupler (01:42:43): Yeah. And we've got some products that would be very attractive to you. We have some LinkedIn-based targeting that for B2B sales make a lot more sense than anything you can get from Facebook. Rob Collie (01:42:54): Oh, we are about to get into some LinkedIn. Don't you worry. We're coming to that. Alex Dupler (01:42:58): So yeah, totally makes sense. And also the other thing that's really big for us, if we go back to when we were talking about the laps, where the business model was commoditized. Well, I don't know if you read Stratechery, but Bing where simultaneously as super aggregator and we have been modularized by Google. You can import your ads from Google into Bing because, to get Mindshare, we need it to be as easy as possible. And so having the reporting be compatible is another part of that. Yeah, people aren't going to just like, " Oh yes. I could get 10% more impressions if I went on to Bing, but if it costs me 30% more time, that's not a good trade, even if the unit economics work." And usually the unit economics are good. Rob Collie (01:43:42): We actually found Bing to be very profitable for us to the extent that we could measure it, but we couldn't get granular with it because of the lack of click IDs. And so we just backed off. Well, fellas, I am so glad that we got this together. I know that we talked about it a little bit in the past and we kind of danced around the idea for a little while and then we did the scheduling dance and there's multiple series of dances, but then we finally danced. Tom LaRock (01:44:05): We danced. Rob Collie (01:44:06): In Thunderdome. Alex Powers (01:44:11): Alexi, it was wonderful to meet you both. Alex Dupler (01:44:14): Thank you. Rob Collie (01:44:14): Two Alex. Tom LaRock (01:44:15): Two Alex, one report. Announcer (01:44:17): Thanks for listening to the Raw Data by P3 Adaptive podcast. Let the experts at P3 Adaptive help your business. Just go to P3adaptive.com. Have a data day!

When Jen Stirrup speaks, she speaks softly.  The meaning of her words, however, speak loudly!  Jen is CEO of Data Relish, a UK-based consultancy that delivers real business value through solving all manner of business challenges.  You don't earn the nickname the Data Whisperer without knowing a great deal about Business Intelligence and AI.  Jen certainly knows not only those topics, she knows SO much more! References in this episode: Data Kind The Art Of War Blade Runner Tears Scene Episode Timeline: 4:30 - The human element of data, Bias in data, implications of Artificial Intelligence and Machine Learning, and COVID data 27:00 - The BI goal is Business Improvement, escalation and taking principled stands, Data-Driven vs Data Inspired 46:00 - Seeing the hidden costs of some business strategies, the value of even small successes, Diversity and Inclusion, and online bullying 1:29:30 - Jen's mugging story (!) Episode Transcript: Rob Collie (00:00:00): Hello friends. Today's guest is Jen Stirrup. Jen and I have had one of those long-running internet friendships that are so common these days, especially in the data world and in certain communities. But we've also had the opportunity to meet in person several times at those things that we used to do called "in-person physical conferences." She's an incredibly well-seasoned veteran of the data world, but if you're expecting us to be talking about things like star schema and DAX Optimization, that's not really what we talked about. You know that our tagline here is "data with the human element," and we definitely leaned into that human element in today's show. Now, we do talk about some of the important human dynamics about data projects. For example, how the business intelligence industry kind of lost its way in the past and forgot that it's all about improvement and how we're as an industry waking back up to that today. Rob Collie (00:00:54): We also talked about the value of having even one signature success in a large organization that other people can look at to become inspired. And she has some very interesting and well-founded semantic opinions about terms like "data-driven" and why maybe, "data-inspired" is better. Similarly, she prefers the term "data fluent" to "data literate", and she explains why. But we also touched repeatedly on the themes of ethics and inclusivity in the world of data. Now, I have a personal idea that I haven't really shared on this show before that I call "radical moderation." It's the idea that you can be polite, you can be reasonable, while at the same time advocating for sharp change. Now, this is personally what I would like to see emerge in our political sphere, for instance, a form of polite radicalism. We need to change, but we need to be nice. Rob Collie (00:01:52): There aren't many readily available examples that I could point to if I wanted to show you "this is what radical moderation looks like." But now if someone asked me for that, I can point them to this conversation we have with Jen. She is soft-spoken, she is polite, she is open-minded, including the open-mindedness that she might not always be correct. And yet, underneath all of that, is a very firm conviction that we need to be better. And I think that's the best introduction I can give this because I don't want to spoil anything upfront. So, let's get into it. Announcer (00:02:28): Ladies and gentleman, may I have your attention please? Announcer (00:02:32): This is the Raw Data By P3 Adaptive Podcast, with your host, Rob Collie, and your co-host, Thomas you know. Find out what the experts at P3 Adaptive can do for your business. Just go to p3adaptive.com. Raw Data By P3 Adaptive is data...with the human element. Rob Collie (00:02:56): Welcome to the show, Jen Stirrup. It is such a pleasure to see you again, virtually, talk to you. I'm really glad we were able to do this So, thrilled to have you here. Jen Stirrup (00:03:06): Thank you so much for having me. I'm glad we made it work in the end. Diaries, schedules, everything else, but I'm really glad to be here and it's great to speak to you. Rob Collie (00:03:15): I know bits and pieces of the Jen Stirrup story and I know bits and pieces of what you're up to. How do you describe yourself on your LinkedIn profile? Jen Stirrup (00:03:23): So I would describe myself as really trying to help people make their data better. I've just finished a post- COVID data strategy for a healthcare organization in the US and in the UK. The reason I'm doing that is to try and have a big impact. I believe in that, I think COVID has brought around a real stress and a lot of technical architectures, and a lot of data architectures as well, and there're all sorts of pressures. So I've just finished that, which has been a nice piece of work. I've been working with a religious organization on their data as well. A lot of people are accessing their services as part of a recovery from COVID. I think it's been a very difficult, challenging time for a lot of people in terms of mental health, and I like to think that by solving these problems you're actually helping people, in a way to contact, some of whom you may never meet, but that's okay. That's really what I like to do, I think, it's a way of connecting, I think. Rob Collie (00:04:22): We subtitled the show 'Data With The Human Element,' you think of the data field is like this cold, analytical, sanitary, and it's not, right? If you're doing it right, you're having an impact in the human plane, and it's a leveraged impact because you can really sort of touch a lot of people's lives via the central hub that is data. And you've got to keep the human beings in mind, even to be successful at the quote-on-quote "cold, calculating data stuff." If you don't keep the humans sort of first and foremost in your mind, you're not going to design, for example, a good data strategy, like what you just finished. Jen Stirrup (00:05:02): That's right. So I believe that the information ladder is quite important. So we start off with data, then we need to turn that into information, but then we need to turn it into knowledge and then wisdom. And I think COVID has taught us many things. I think it's maybe taught us a sense of purpose, it's something that can help drive all of us. Data can be part of that and I think that data in some ways has been replacing some of the bigger-purpose questions that perhaps we should ask ourselves more often as human beings. With artificial intelligence, particularly, I'm finding that people are replacing data with, perhaps, information, knowledge, or wisdom and say "what does the data see?" and that's fine, but we have to have the context to the data as well. Jen Stirrup (00:05:47): I think in some ways with artificial intelligence, what people are trying to do is build a little box of data and it's becoming this oracle that people are going to touch and say: "So, what does the data say?" It's like we are taking this box and we're trying to turn into some sort of God that we can touch, and it's going to give us all the answers, but if we're going to do that, it has to be a God that we are comfortable to live with, and it's one that we can choose, and one that fits in with people's ethics and their sense of purpose. So, I see data as part of fitting something that can make us all better in so many different ways, whether that is healing or bringing people together. Jen Stirrup (00:06:29): So I think if we could solve these problems where people are feeling that they are not interconnected, then we could start to try and look at that and perhaps think about making people feel whole and feel more together. Because I think what COVID has done is really helped us to focus a lot on data but perhaps not about how we could do things better. It seems that we have an opportunity to decide what goes back in to make the new normal or the next normal. And I'm worried I suppose that I don't see that happening as much as I would like. So yeah, data is important. Absolutely. We wouldn't be here without it and the fact people are struggling with it does pay my mortgage. I still would like us to ask ourselves the bigger questions as well as something that's important to me. Rob Collie (00:07:14): Let me check here. Oh yeah yup, it pays my mortgage as well. We're here for a reason that's for sure. I loved you talking about the AI, this box, that we're going to sort of elevate to the status of a God or that's how a lot of people are viewing it subconsciously. Of course, it's a box that we built. Jen Stirrup (00:07:33): Yeah. Rob Collie (00:07:33): We fed it with our context. It got fed with our assumptions and also our blind spots and now if it makes decisions, that thing starts making judgments and decisions that impact people's lives. It's a tricky proposition, it's one that's best approached very carefully. Jen Stirrup (00:07:55): I agree and I think that's why the bigger questions are important. So say for example, you may have seen the Netflix information series. It was called 'The Social Hack' or something like that. I've forgotten the name, but it was talking about the role of bias in data. One of the researchers found that their facial recognition algorithm didn't recognize a face. And the reason for that was that she's black and for me, I just thought, that's such a preventable issue and how much time do you spend looking at preventable issues? And perhaps not very much. I still see the magpie problem a lot in technology. Companies are happier buying a new technology that they see that's going to solve all their problems, but actually it's not doing that. It's maybe replacing as a bad answer to a different question. We can't see that right now in artificial intelligence. Jen Stirrup (00:08:48): There's some research going on, which will decrease the size of data sets that AI needs in order to create its algorithms and that sounds fine. It's a good piece of research, but what I'd like to see is more researches on collating datasets which are less biased, so that we can think about focusing and trying to make the algorithms fewer rather than focusing on making them smaller. Jen Stirrup (00:09:13): I know a few years ago, you probably remember, everyone talked about big data. Big data was the thing but we didn't ask ourselves if this was the right data. It might be big, but if it's missing out large sections of the population, then that's building an inequality before we get started. I think, even if you don't have the answers, asking these questions is a good thing. I don't have all the answers. There's people working in this field much much smarter than me and they all live and breathe this stuff and I read it, the things that they're doing and talking about, and I think this is such an important part of what we do every day. I think it's really important. I don't know what you think, but there's so much going on in the world of data at the moment that it feels hard to keep up sometimes. Thomas Larock (00:09:58): So first I want you both to remember in case you've forgotten, but you can purchase the Azure Data Box, that does exist. Rob Collie (00:10:07): We will just call it God in a box. Thomas Larock (00:10:09): Azure Data Box, it's actually for shipping storage to an Azure data center, but that's what they chose to call it and I said: "You put your data in the box or it gets the hose again." Right? So- Rob Collie (00:10:20): No no, Tom, it's one: "Put your data in the box." Thomas Larock (00:10:26): So, I mean, that does exist. The first point I wanted to make that you danced around, like Rob you were talking about how we're building this thing and it comes with all of our failings. And I know Jen, she leads discussions on diversity, inclusion, equality and I try to emphasize why that's so much more important and especially seeing the rise and I saw the Netflix special as well, and the Data Justice League. The idea is we need to have those programs in order to have better models. We have to be aware of the bias inherent in the stuff that has already been built. And I think there's a lot more awareness over the last 18 months regarding the products that are on the market that are already failing us because they were built with these biases. And that's a difficult thing to overcome now that you have police departments or governments deploying this technology, thinking, as Jen said, it's this God that is just going to give you all the answers. Thomas Larock (00:11:35): Jen, you also hinted on the thing about the question. So, you're replacing one problem with another, and that made me think of how vital it is that you understand the question you need answered and a lot of times that gets kind of shifted, it's fluid almost. It's like: "Oh, well we were doing this thing we think this next thing we'll solve for it." But the next thing you're getting is actually answering a completely different question than what you thought you were doing and it leads to a huge, huge disconnect. And I think the last thing I would say Jen, I've seen that research about the data sets. I'm encouraged by the idea that we could get people to understand that it's not the volume of data that makes a better model. It's the data that was chosen to be collected in the manner in which it's collected. Thomas Larock (00:12:30): So I know the research on building these models and they're saying: "Yeah, you don't need a billion rows. The accuracy tails off at some point after, say, a million rows." At some point more data doesn't make this model any more accurate but the inherent problem is how was it collected? What were the biases and how was it collected? What was missing? Was it missing at random? Was it missing not at random? The analysis necessary to conduct that research, I think is where we are sorely lacking in business. I know it exists in academia, but those people, they don't scale. There's only so many of those, and there's a lot more businesses trying to get the job done so I think that's fairly important. Jen Stirrup (00:13:13): There is a huge gap between academia and business. I guess there always has been, I do speak to academic institutions from time to time and it's clear that they are doing so much work. They really are, but how that is getting out? I am not sure. Maybe that's why they asked me to come and talk to them so I can talk to other people about what they're doing and I don't mind doing that. I think there needs to be more of that, because I think these scientists, these academics are working in this, have to get access to each other as well and the multidisciplinary aspect of it is really interesting. I did a Postgraduate in Cognitive Science about 20 years ago, and suddenly it's back round again, and it's about philosophy, linguistics, psychology, AI. And why did that go away? Jen Stirrup (00:14:03): It should never have really gone away. I think we got as an industry perhaps Goldstone and such technologies which these things were re-badged as, and we got derailed by the marketing efforts. But I think that there's real room for doing these things in a better way. I don't know if you see this, but I see, or maybe it's my age now, I've been around in the industry for a long time, but I see that people are doing and making mistakes that I first saw 20 years ago, data collection, which you rarely mentioned, Tom, that's been there for a long time and then it seemed to go away. Jen Stirrup (00:14:36): I think that's why academia does help because it gives us maybe more of that consistent backgrounds than perhaps we get from marketing noise, which was goes round in cycles and trends as people are under pressure to purchase these licenses or whatever it happens to be. I wish I had better answers for all of this, I think sometimes it's about just asking these questions, blogging, talking about them, putting them on social media so that when people are thinking, "what do I do about data strategy?" That these things are part of this. I saw a study recently saying that companies are decreasingly likely to include ethics and these questions and bigger societal questions as part of the data strategies as you're trying to get the link. But it disheartens me because I thought I could see that the voices are getting squeezed out. Rob Collie (00:15:25): Decreasingly likely, like we're trending- Jen Stirrup (00:15:28): Trending down. Rob Collie (00:15:28): You know, it'd be one thing to be flat, right? I mean that would also be disheartening, but to be decreasing, decreasingly likely to be factoring in ethics into a data strategy. Now we've been talking a lot and I think it's a good thing to continue to talk about the implications of AI and machine learning in this space, the business intelligence industry isn't particularly fraught with this kind of problem, right. Transactions happened, or they didn't, you know, and it was the number of six or a seven. I mean like, you can get it wrong, you can have bugs, right. But there isn't any like objective debate about what, there shouldn't be any way about what actually has happened. But the decider systems, are a completely different game, like where should we route this patient? This is going to have a huge impact on their life. Rob Collie (00:16:21): That's a very, very, very different game and we've been talking about sort of like, the completeness of the data that is used to train these systems, but I think it's really instructive just to stop for a moment and go, you know what, even if we were able to feed these systems a 100% comprehensive picture of today's world, we still have to accept the fact that we're telling it that today's world is what we want. Right. And maybe we don't, you know and there's always a judgment in training these systems, we tell it what is a success and what isn't a success. Our unintentional biases can leak into this stuff in a million different places, even if you suddenly had God-like comprehensive powers to feed it, quote-on-quote, all the data, right. It's still leaky. It's still fraught. Jen Stirrup (00:17:13): Yeah and actually, I think it's an extension of their problem that we see just when we're building a data warehouse. Sometimes I'll go into a customer and they'll say, "you know, we want to see our data and see our latest vendor here," and then I'll say, "well, is it preserving the data or is it just, you know, been reamed out the other end, what you're doing with it? Where you're storing it?" And then the argument against the data warehouse as well. It's not going to capture everything in the possible universe of possibilities in my business, so I don't want to do it. And I find the argument goes something like, "there's an edge case that it won't cover." Others, "this edge case, it won't cover here." And then you have to say, "well, you know okay. So it's not going to cover all the possible edge cases, but it will cover 80% of what you need, and the rest, can go to shadow IT or shadow data systems or wherever they happen to be." Jen Stirrup (00:18:03): And I think we're still trying as it's a bigger picture perhaps trying to control everything that happens around our business, but we have to be flexible enough to cater for these scenarios. We haven't seen this before. I think that's what makes the AI so difficult actually, as we have more than one type of AI, we have a general artificial intelligence, which is more like Terminator, you know, these kinds of things. Rob Collie (00:18:29): Innocuous stuff like that. Thomas Larock (00:18:30): Harmless. What's the worst that could happen. Rob Collie (00:18:32): Yeah. I mean. Jen Stirrup (00:18:35): Well, I think as humans, we do enough damage to ourselves, most of the time we don't need a Skynet. Thomas Larock (00:18:38): That's true. I agree. That's often my reaction to, well you know, like self-driving cars, like what if it makes this mistake? Okay yeah but the human being track record behind the wheel, we're not trying to be perfect, we're just trying to be better than people, which is a little bit more achievable perhaps. Jen Stirrup (00:18:56): Exactly and it's all a bit context, which is how to program. You probably remember a few years ago, at SQLBits say Tom, Steve Wozniak visited. I don't know if you were there for that SQLBits but Steve Wozniak is one of the team that founded apple. You must know who he is, but he's talked to us about the Wozniak test for AI, the testers will have an artificial intelligence sought of robot come into your house and make you a coffee from scratch. Now that involves a lot of contextual knowledge. They have to find your kitchen, they have to get your ingredients and get a cup, you know all that kind of thing and that requires context. And that's more general AI, that's more difficult to program. But if we're to think with CEI being more successful for businesses automation productivity, and it's just trying to do something, one thing really, really well, something that will help a human to make better decisions faster. Jen Stirrup (00:19:51): Such as perhaps parceling out x-rays, which don't show any presence of a tumor as an example, but we then get the 10% of x-rays that makes sure something and passing those onto a human to look at. So there's plenty of rooms for defining what success looks like for us for artificial intelligence I think. With business intelligence, your right, we should have one version of the truth. People are still living so much in Excel and Google sheets and things of empires away, and that are sitting in their laptop. How do you move that to the cloud? So you move them perhaps to office 365 or a Google work space, and then you're trying to encourage people to rethink the processes about, Hey why do we save stuff in the cloud? Or why do we make our decision making more apparent? And it seems a bit difficult to ask AI to make its decision-making more apparent, when actually a lot of people spend time hiding or umpiring the knowledge anyway. Jen Stirrup (00:20:49): I don't know if you think this, but I often think business intelligence problems are change management problems in disguise. It just happens to be showing up in the data that there's a problem. Thomas Larock (00:20:59): Yeah. Rob Collie (00:20:59): Ultimately it's not about knowing, it's about improving. Knowing that there's a problem and even knowing what's causing it is really just the beginning. Very often it's like okay, now what? This is going to be a really difficult problem to address operationally. Jen Stirrup (00:21:16): I think we forget the process of optimization and business intelligence. And I wonder if that's the reason why AI is becoming so prevalent at the moment, because it is much more clearly talking about optimizing and improving processes and automating. I think in business intelligence, we have almost stopped talking about optimizing business processes. I don't see it quite as much, I wonder if we get sort of caught up in data visualization, you know Tableau came along and then power BI and everyone started chasing after that. We're perhaps forgetting that actually we're doing all that for a purpose, which is to make something better somewhere. I don't know if you find this but, I obviously run [inaudible 00:21:54] business and it's very hard to get customers to agree to a case study because they don't want to show that actually they were in a bad place and they don't want to show the competitors that they were in a bad place. Everyone's ashamed of the data. So it's really tough. Rob Collie (00:22:07): I've seen sort of multiple facets of that. So first of all, yes, everyone thinks that they are uniquely broken, everyone's organization that they feel a level of sort of like discomfort and shame about where they're at today or where they were yesterday. They feel like they're the only ones, but we see so many organizations per year, especially the kinds of projects and the pace at which we move the world is very much uniformly broken. No one's really behind, everyone's way behind of where you'd sort of like as a dispassionate observer, you'd expect people to be a lot further ahead than they are, but no, no, really the basics are still not sorted out universally. We're still kind of in a dark age, in a way. Jen Stirrup (00:22:51): Yeah. Something, I see really basic issues of one customer example of talking about where they were calculating the mean incorrectly for two years. And then two years before that, for another two years, they were calculating the median incorrectly in Excel. What they were doing was it were taking the middle value of a column. So of course, if you sorted the column next to it, the value changed. And they said that that was the median. And I said, "okay, so you've got a column of 20 items. Are you telling me that whatever's a number 10 is the mean?" And they said, "well, yes, that's in column B." What happens if you change the order in column E from perhaps alphabetical order to reverse alphabet order, the values can be changed, right? And they looked at me and I said, "why did you calculate it like that?" Jen Stirrup (00:23:41): And they said, cause we can calculate the mean using Excel formula. So eventually I said, "why are you using the mean," because it's quite sensitive to outliers the median's better. and then they said, "well we've tried that but we couldn't calculate the median either." I said, so okay "for four years you've been trying to calculate the mean and the median incorrectly in this one spreadsheet. Can you tell me about the rest of your spreadsheets? How often are you trying to use the median or the mean all of it incorrectly?" And I think it's probably the only time in my 20 plus year career, I've seen a customer actually punch himself in the face and it was just absolutely stunning. And he said, "I'll go and speak to the statisticians." And I thought, you've got statisticians working here. I'd love to meet them. Jen Stirrup (00:24:26): I wonder what they're telling you. And that was my second deal in sight, I was on the on and off for six months. And that was just the first problem I found. So I know we talked about data literacy. I'm not a fan of that phrase. I prefer fluency or something along those lines. So I don't want to assume people are data illiterate. Because I don't think that they are, I think we're born naturally within us an innate sense of numbers in a way, we can tell more from less, right? My dog can do it, right. So if I got five treats in my hand, he knows I've got others. If I just give them one, he's not stupid, he has a sense of quantity. And I think it's about, we need to get better in industry, perhaps explaining results, findings, conclusions, and context to people instead of just throwing dashboards at people and expecting them to understand it. Jen Stirrup (00:25:16): If somebody recently sent me a scientific article which was all about COVID and some testing that they did in mice, and I could read it, but I couldn't understand it because I don't have a background in medicine. I read the abstracts and I read the last paragraph and the first paragraph, but I didn't read the rest of it because I thought this is way beyond me. I don't understand what they're trying to say. But I think for me that highlighted a problem with data literacy, I could read it, I couldn't understand it, and I certainly couldn't act on it. And I don't want to give other people who are trying to consume business intelligence products in some way, whether they're dashboards or even dumps from Excel, that they just don't understand what they're getting. How we do that, I think is perhaps focusing in data translation. Jen Stirrup (00:26:03): How we do that, I think, is perhaps focusing in data translation. I had a woman who worked for me, she actually was a qualified librarian. So, her insights about information retrieval were very interesting. I learned a lot from her, because that was a little bit the data. And she would say things like, "Jennifer, Google is not the only search facility in the world. We can use so much more," because she's accessed all their library systems around the world. And there's so much information we don't access because we can't, usually. But the point being that what I learned from her was about translating things, where they were easier to understand for other people. And I think it's an incredibly valuable lesson, and the world needs more librarians. Rob Collie (00:26:43): There's a lot here, right? Business intelligence was always a means to an end, but because it was so difficult, it was just so incredibly difficult to even get a halfway-competent system instilled, built, configured. When something is that hard for that long, it becomes its own goal after a while. It's easy to habituate to the idea that this is the goal, intelligence is the goal, knowledge is the goal. No, no, no. Improvement was always the goal. What's really been fascinating for us is, when we see our clients, the people we work with, when we see them start to get the BI problem under control for the first time ever, their gaze immediately sort of zooms back and they start thinking completely unbidden by us. We don't have to seed this conversation. It just happens. They start looking at the bigger picture now and going, "Oh, okay. So, now this information needs to feed into better decision loops and optimization and things like that. And how do we facilitate that?" Rob Collie (00:27:53): And from the beginning, we try to counsel everything being built around that "taking action" thing. You can build an incredibly informative dashboard that is intelligent, it's a work of art in many ways, on many levels, and it can be useless. It can be factual, it can be impressive, and it can be useless because you can't use it to make any improved decisions. I've been guilty of this. I have built things like this, like, "Ta-da." And the client doesn't even have the language to push back. Jen Stirrup (00:28:30): It's something I've tried to keep in mind now is the utility of what I'm actually doing, because people just want data for the sake of data, and they get that. I think, sometimes, they don't know what to ask for, so they take something because it's better than nothing. And they'll say things like, "Right, I want the last five years of data and 191 columns, I want it all on the same page, and I want to be able to print it." And then you have to say, "Well, let's think about how feasible that is. You'll get five years of data, it's not going to fit in one page. 191 columns is going to be really small. So, let's have a..." People ask that because they don't know what they want. Jen Stirrup (00:29:06): About a dashboard recently, a health and safety dashboard, it was using power apps as well. So, the company, if they saw a health and safety priority issue, they could use the app, if they were health and safety professionals, and the app would record data, you could upload a photograph, and then that would go into a system which you could then see in Power BI. And the nice thing about that was you could see improvements over time because people could get their health and safety issues resolved more quickly, so things like boxes stacked against fire exits, slip and trip hazards. Jen Stirrup (00:29:43): Now, it may not seem very interesting, but actually, the reason that project had happened was because someone that had been in a health and safety incident and it had not been tracked properly, and the idea being that they were trying to improve the process. But sometimes, I think data problems and data solutions happen because of two things. One is you need an executive sponsor, and the second thing is a crisis. And together, the executive sponsor and the crisis will engender change somewhere. And that change management process so often turns into a business intelligence solution. And nothing is an industry. It's something I'm personally trying to always keep in mind is: what's the purpose? What's the optimization? What problem am I trying to solve? Rob Collie (00:30:30): Yeah, one time, I was asked by a client to help debug a report that was really slow. So, this is great because this is an example of a report that I didn't build, right? I can use an example that wasn't one of my own families, but I'll tell my own as well if you want. But I go, "Okay, I'll take a look at it." I'm expecting some sort of DAX or data modeling problem or something like that. And they show me the report, and it is a 100,000-row pivot table. The pivot table has a 100,000 rows in it. There's DAX behind it. It's a DAX data model behind the scenes, but the report itself, the output is 100,000 rows. And before I even engage, I just turn and look at them and say, "Oh, my God, who was using this? You don't have a performance problem. It's..." And they're very insistent. "No, no, no, no, no. This is the thing. We need this." I'm like, "All right." Rob Collie (00:31:21): So, I start looking at it, and it's crazy how many columns there are. And it was a list of every employee and every location that they have in the country, which was hundreds of locations and thousands of employees. And for each employee, their scheduled time-in and their scheduled time-out, and their actual time clocked in and actual time clocked out. I turned back at him again and I go, "Okay, really? What are we doing here?" And they're like, "Okay. So, we have all these regional managers that are looking at this multiple times a day, probably eight times a day or more, to try to figure out if any of their stores are empty, aren't staffed because people didn't show up." And I just smacked my forehead and I go, "You don't need the timecard report," which is what they called this thing, the timecard report, "You need the empty store detector." Rob Collie (00:32:18): And I mean, there was no way to make this thing faster. I mean, this thing was such a gross misuse of technology. I just went to the whiteboard and I sketched what the empty store detector could look like, and they're like, "Oh, that's great. We'll never get our managers to switch over to using it, so let's just go back to fixing this other piece of junk." Jen Stirrup (00:32:37): Yeah, because something that I struggle with, personally, is the idea of surveyance reports. It's something that really bothers me. I've pushed back on a few customers to see, "Are you micromanaging or are you surveying? What is it you're trying to do?" On occasions, I have escalated it to say, "Look, this report is probably been used to hit people for the head, and I'm not comfortable with this because I think this has gone beyond micromanaging." And we had set the scope of the project of the thing we were supposed to deliver. So, I'm going to escalate this because I want to understand better the purpose. And if I'm wrong, we will deliver it." Jen Stirrup (00:33:12): And normally, when I go back and see that, even in that particular instance, I showed the senior management and I said, "Your middle management want to do this." And they said, "No. We are not spending time doing that. We need to understand the wider context. If there is any issues going on with staffing, then this is probably a symptom rather than the cause of the issues, if people are being watched like that." So, I think some teams escalating, as much as I don't like to do it, sometimes is the best way forward. Rob Collie (00:33:44): It takes a lot of professional courage to do something like that. For example, have you ever taken one of those principled stands and ended up no longer working for that client because they basically fire you for not staying in your lane? That's a risk, right? Jen Stirrup (00:34:01): Yeah. It is. I've never been fired for that, but I have said, "Uncomfortable, and I'm we going to stop delivering services, and we need to decide on an exit strategy." There's different ways you can do that, right? So, you deal with the current project. You then say that you're busy for the next century when we come back to you for other work. I don't like doing that because I often feel like you should give them an alternative to say, "Well, here. I can't deliver it, but I know someone who can." And then I recommend one to my network. But the thing is, when I make these quite principal stands, people back down often, or they back down and they just asked me to do it. But when I've gone back to people like that customer, who come back to me for extra work, I've done some investigating work and I've found that they have not implemented a thing that I've been worried about or concerned about. Jen Stirrup (00:34:49): So, I think, sometimes, if you do speak up, people are maybe surprised by it. It's maybe different who it comes from. And I think, perhaps, even a soft Scottish accent, smiling sweetly at them and saying, "Can you explain to me a bit more about the reasoning behind this? Because your team want to do this thing, but I have some discomfort because it's outside scope." And they're not telling them, and they're very direct. Wait at first, but they start to get their message. Jen Stirrup (00:35:16): A former boss of mine years ago, he said I had a soft rein approach. I actually think that's a nice way of putting it, where, as much as I might be tempted to go in all guns blazing, I'm trying to gently bring it up and then bring it up again a bit more firmly, and then, suddenly, people are starting to understand better. But that's me having to probably, sometimes, exert a huge amount of self-control as well. But I think that's part of the consulting game. It's very tough. But I think seeing something like that happen, I think the reason it happens is because people aren't thinking about it longer-term. And me as a consultant, it's easier, perhaps, for me to think about it long-term and also a bit more closely as well, because you are thinking about the consequences of what you're trying to do, the purpose. Rob Collie (00:36:04): Yeah. If you're good at data and you're experience with it, you spend a lot of time with it, that allows you to put some of those things a little further down in the subconscious, and the rest of your human faculties can resume working, whereas, I think, for people who data is still this arcane thing, it's not the thing that they've spent their lives with, it's just really easy to get target-fixated on the data, data, data, data, right? "It's not about the people, we're trying to figure out the data," right? "And inform me," and all of that. Rob Collie (00:36:33): And I think it's like when you're first learning to drive, I couldn't have the radio on. The radio was really distracting. And you certainly couldn't have a conversation with someone next to you. So, all you can do just to make sure that you're turning the wheel the right amount and all this kind of stuff. It's just overwhelming. But once you internalize all that stuff and you build the muscle memory and all those sorts of things, now your brain is free to do some other things. Like this data fluency thing we were talking about, it's neat how, as you climb that slope, you're never there, it's a perpetual journey, the other parts of the equation like the human things, right? They can come back. Rob Collie (00:37:12): An example, even just from our own business, we do a lot of internet advertising. And sometimes, when people at our company are thinking about this, now the wrong way to do it is to go and like, "Oh, let's go look at the ad words API and let's get fascinated by the tech around this." And I'm always trying to remind people that, no, no, no, we're trying to scale a human interaction. That's what we're trying to do. We're trying to reach people with our humanity- Jen Stirrup (00:37:43): I think that's so true. Rob Collie (00:37:43): ... and we're using a technological system to do that. It's a tool for the other thing. Jen Stirrup (00:37:50): You're so right. I think we should be using technology empower and enable. And I think my personal mission is about helping people. I find that rewarding, personally. I like things with a purpose, so that's why I do charity work with organizations like DataKind, because when you get someone crying because you've solved a problem for them and you've helped them, you know how incredibly grateful they are. But I think, for me, that's why diversity and inclusion, equality, and intersectionality more recently has become really important to me. Jen Stirrup (00:38:21): I'll just give you a few examples that's in my head. I did a project recently, and there was a woman of color in my team, and I felt that she was being talked over. I'm used to being talk over, softly spoken. But I could see it with her. And I just made a conscious effort to say, "I'm sorry, but I don't think she's had the opportunity to speak, and I can see she's tried to have some input." So, some of it's a bit like that. But some of it is directly saying, "What do you think? Sorry, we haven't heard from you," and pulling people out. And you know what? She was and is still incredibly insightful. And sometimes, the best data scientists I work with are people who can't code. And I think about her and I think about another woman of color as well that I work beside. Jen Stirrup (00:39:06): Fantastic data scientists, they both know Excel, but they can't write a line of code. And the reason they're so good is because they are such fantastic questions. That means the rest of us who can code have to then go and get the answers. And I think the knack of asking the right questions is such a gift, it's such a skill, and it's something that I am consciously trying to improve myself on. And I think diversity, inclusion, and equality is really important, but we wouldn't get anywhere with any of that if we're not allowing people space either to talk or we're not able to give them the space to ask the right questions. Jen Stirrup (00:39:42): Now, I am constantly learning every day. And to do that, I'm having to learn to get better at asking questions. And it is a skill to ask, but I think, when we're dealing with data, it's about helping people not to feel stupid if they're asking questions, because I think, with these particular cases, it's very easy to feel diminished in a conversation where other people are understand the technology, they can code, you can't, but you've got an insight. I know we talk about data-driven, but I like the term "insights-inspired," and I wish we had more of that because that, I think, gives us room for other people who perhaps don't understand the technology but do have business insights that I would never get, because they help me interpret the code or the data to make it better. Thomas Larock (00:40:28): So, you said data-driven, but you prefer insights-inspired. I think those are still two different things because, when I think of data-driven, I actually think of that in terms of, "I'm going to make a decision based upon what the data's telling me, not upon my feelings." The insights-inspired, to me, is how I get to the question I want answered, right? But I'm still data-driven. I think there's some overlap, but I also think there's a lot of space there where they are distinct, because I do believe in data-driven because I've been in those meetings where somebody's like, "Yeah, I don't really care. We're going to do what I think is right." "But the data says something completely opposite." "Yeah. That doesn't matter to me." And lots of those cultures exist. I love insights-inspired, and I'm going to steal that. Jen Stirrup (00:41:16): That's fine. I think we need both, actually. I'm sorry if I wasn't clear. But you're right, there is a good impetus for people to think, "What does the data say?" And I like that. I think the "insights-inspired" piece will help us to understand if the data's right. And I'll give you an example of something that I did. So, I was doing some work for the national health service and there's some data missing for a hospital, and it was not an insignificant amount of data. It was for about five years, the data. And I searched for it all morning, and I was just about to ,arch down the corridor to go and corral a DBA to ask him, Have we lost any data? Because I cannot find this." Jen Stirrup (00:41:55): And then, when [inaudible 00:41:56] was passing, she said, "How are you doing?" I said, "Oh, have you ever worked at this hospital?" I won't mention which one it is. And she said, "Oh, I was there until it closed for five years and it merged with another hospital." And I thought, "Oh, you've just answered my question. Right." Because I was sweating beads because I thought, "We've lost five years' worth of data." And I thought, "We've done that. We are in so much trouble," because it's a lot of data. It's a lot of patient data. No, no, no, no. They went somewhere else. And there was a very good explanation that I would never have got by the data. I could have hugged her. Jen Stirrup (00:42:31): And to this day, I still feel the palpable relief, because I was walking in the hospital, thinking we need a really good explanation for this. But according to the data, it was not there. So, I think, when I look at data-driven, I think they're two sides of the same coin, because insights will tell you what the nurse said, "Well, actually, it's like this," and they will add to the interpretation. Jen Stirrup (00:42:54): I just sat in a meeting once where one of the leaders said, "All right. So, we've got the data now?" I said, "Yes, everything's fine." And in front of four of his team members, he said, "So, we can get rid of the business analysts then, because we've got the data now." And even when I mention this, I still, at this point, feel my blood pressure rising, which is not good for me. I am well over the age of 40. And actually, I was stunned. I said, "How are you going to understand the data if you don't have your business analysts. Who's going to tell you what it means? "Oh." I said, "Are you really thinking that you can just throw your data at a wall, see what sticks, see what's left, and that's going to drive a business? Because, pretty much, that's what you're doing, if you are not involving the people who understand the business." Jen Stirrup (00:43:43): And after the meeting, I mean, some of them were crying, saying, "He was talking about me losing my job." And the people impact was terrible. So, this is where I've got my principals coming in. So, I went and I escalated that afternoon, and he was taken off the project the next day. That was due to happen. That was just outrageous. And if any of you who are listening and this is you, I love that team, their insights were incredible and I learned so much from them. And to the leader in that organization, please listen to your team members. You will get so many many great insights. Rob Collie (00:44:23): Wow. Jen Stirrup (00:44:24): Sorry, this is very cathartic for me. I'm glad you've brought me on today. Rob Collie (00:44:33): I mean, just watching your face as you told that story, I can see the emotions that you're feeling, right? Jen Stirrup (00:44:37): He's going to get this. Rob Collie (00:44:38): And it's a mix, right? It's a mix of the beauty of some of these people that you worked with, right? Contrasting with like this horrible, horrible attitude, at the same time, from this one individual. When you have all those feelings at the same time, it's like you need a new name for it. It's like, "What is this feeling?" Jen Stirrup (00:44:56): And I think the industry is like a pendulum, so we go towards data-driven. And for some organizations, they need good data-driven, so Tom's given a great example. But sometimes, it goes too far and they say, "Yeah, I read that buzzword. I'm going to do that." And then, there's an expense, something has to give. And that, unfortunately, was his team. Like you said earlier, Rob, it's about the people. We should be there to help people by helping people do their jobs better, not necessarily replacing them. That was not ever on the menu. Rob Collie (00:45:29): Yeah. It's counterintuitive. Sometimes, when your data system gets better, the right move is to have more analysts because there's more ROI in having them. Even just hiring a data professional services firm such as yourself, the reason to do it is because the ROI can be massive. Jen Stirrup (00:45:51): Yes. There's lots of unseen costs. I worked with an accountant last year who spent four out of five days a week merging Excel together. And I sat with her, I got to know her pretty well, I mean, remotely because of COVID. And eventually, she said, "Oh, I'm looking for a new job." And I said, "Oh, really?" And she said, "I did not incur a graduate debt to sit and do something that I could have done without my degree." She'd put a lot of effort and, same in the US, lots of student loans to do a degree. And she said, "Technically, my job title is accountant, but I'm not accounting. I am munging data around in Excel." And one of the projects I had recommended was data integration, right? And they wouldn't go forward it. They kept saying, "No, no, no. We've always done it this way. So-and-so om accounts does all that." But they never asked her what she wanted. Jen Stirrup (00:46:43): So, she left, and I was not a bit surprised because she said, "I want to be an accountant. I want to account." And I know that it's not my personal lifestyle. It wouldn't be my choice of a job, but for her, she just loved that, and she wasn't getting to do. So, sometimes, the causes are quite unseen if you're not looking after the processes or the data, because that incurs hiring costs, then, on staff onboarding costs that don't get included often as part of these business strategy projects. When I'm doing a data strategy, I try to include them, to say, "But what happens if you change? But what happens if you don't?" And you're going to lose people because your people, very often, want to be skilled in the later technology. Jen Stirrup (00:47:25): And I'll give you an example. One customer I worked with said to me, "We need your help with reporting services, SQL server." So, "Okay, good. I like reporting services." Then, they talked to me and I said, "What version are you using?" And they said, "2005." And I said, "Why?" "Because the application that's using it requires SQL server 2005 and we can't upgrade." Said, "So, what was the application written in?" "VB6," which you may have heard of that technology. It was around in 1999. It was last century. So, the data state was antique. I had no idea that it was that bad. But then, the application came up, and Microsoft still do a version of a Visual Basic. You can go to the site, the latest version... But the point being that the staff and that place had settled for VB6, they'd settled for 2005. That doesn't mean that you're getting the best team members. And when we worked, it was recommended an architecture. Said it was not touching it with our [inaudible 00:48:30]. Rob Collie (00:48:30): I'm still very fluent in VBA6, so maybe after we finished this show, can you give me the information of this organization? I might go apply. The last place on earth that VBA6 fluency is... Actually, that's not true. It's still being used everywhere. It's just not being used centrally. Jen Stirrup (00:48:53): Yes. I did say to them, "I am not touching any software that was not built in this century. So, if it's in the last century, you've no chance." So, re-architected, actually, we're using the Azure Cosmos... Thomas Larock (00:49:04): It's a good rule. Jen Stirrup (00:49:05): ... and dot... Yeah, it's a good rule. It's a rule to live by, you can quote me on that. I use no software built in the last century. In fact, I'm going to make that my new company advertising strapline. That's great. I like that. So, they're happily in Cosmos and .NET. And we used that because the developer said, "Hey, does that mean we get to modernize?" I said, "Yes. And you will either modernize or I will leave. Your bosses are going to have to modernize." So, they did. But again, that soft Scottish accent comes up. "Well, why don't we use software that's built in this century?" Rob Collie (00:49:42): It's a devastating maneuver. If we were making a card for you in a trading card game, that would be one of your two power moves, right? Soft Scottish accent. And the description of the power is something like, "Removes all defensive screen cards from opponent." Thomas Larock (00:50:07): Disarming. Jen Stirrup (00:50:10): Absolutely. Yeah. It's just funny how the data problems are really throwing up what's wrong with the organization. Obviously, they did that, but two years ago, I went to visit them again, just before COVID last year. They'd implemented a data science team and they just wanted some strategic consulting. And I was really pleased with how they turned around. So, sometimes, if you just find a problem like that, a small success, building those small successes, and they were allowed to up. I don't know if you see this, but big thing of what I'm doing when I'm in organizations is change management, but also a lot of that's people. And people tend to align themselves with success. So, if you can just show one small success, people get on board with it. Rob Collie (00:50:53): Yeah. I mean, it's everywhere in humanity, right? We're fundamentally pattern-matchers. And if you haven't given a population any positive patterns to match, no examples, it's amazing how stuck you can be. But one success, right? We have an infinite percentage increase in our population of successful examples. We went from zero to one. Like you say, the dog knows that there's five treats in your hand, right? We're not dumb. If there can be one success, there can be more. But if there's zero successes, that's powerful. Jen Stirrup (00:51:25): Yeah. And I don't know if you see this problem, but it's something I see a lot is people think maybe Tableau or Power BI, they buy this, it's going to give them a success. And it does, until the data starts to get hard. And then they either have to scale up in DAX, which is fine, but sometimes they don't have room or bandwidth to do that, so they get almost a bit depleted because they realize, actually, data's hard. We've never really nailed data as the human race. Rob Collie (00:51:55): It's always hard. Unfortunately, to sell software, to a certain extent, you have to sell the lie. If you're a software vendor, you have to se... Rob Collie (00:52:03): ... have to sell the lie. If you're a software vendor, you have to sell the lie that this tool is the magic fix, that it's going to make data easy. And I do actually, in a weird way, I kind of like blame Tableau for making this worse, but while at the same time, being very grateful to Tableau that they made interactivity a must have. Jen Stirrup (00:52:24): Yes. Rob Collie (00:52:24): I think they were actually, more than any one entity, responsible for us breaking this notion that reporting services and similar tools were it. Jen Stirrup (00:52:34): Yes. I remember the first time I saw Tableau. I had been hired as a developer for SQL server [inaudible 00:52:40] services and my boss said, "I think this is a future, this stuff, Tableau. Here's the download link. Tell me what you think". 10 minutes I was completely hooked and it changed my career because otherwise I would have probably stayed in the database reporting world and I suddenly thought there's a whole world here with stuff. So I love what they did. I really, really think it was groundbreaking. Thomas Larock (00:53:01): At what point did a report just become synonymous with the word "Tableau"? I have a limited experience and maybe it's an outlier, but to me, I always hear people say, "I'm going to run a Tableau report". I mean, it's just a report. I worked with Crystal and BusinessObjects, same thing I guess. And do people always qualify the type of report they're running as if that makes it more special or do people always say, "I'm going to run a power BI report"? Why is it always a qualifier? And in my case, I always hear, "I'm going to go run the Tableau report". I'm like, "It's just a report. It doesn't really matter what's the software that's doing it. It's just data. It's just a report". But I hear that a lot. I just figured I'd ask you two if that's the same experience? Jen Stirrup (00:53:43): Yeah. I think I'm hearing that more and more and I actually think it's almost going the other way, where people are only wanting interactivity, they're only wanting things they can click and tick. And what they're not wanting as much is a SQL server, mahogany red, forest green, slate gray, corporate template, because that was the what, about four templates you got with reporting services. So I see that more and more apart from the finance world. They still very much want it. But what I'd still see is a big need for tables. People still want to export to Excel. And I think it was you, Rob, who actually said this years ago, that the third most common button in Tableau is something like "export to CSV". Thomas Larock (00:54:26): Yeah. Rob Collie (00:54:28): Yeah. The third most common button in any data application is "export to Excel". Thomas Larock (00:54:32): Yeah. Rob Collie (00:54:32): Behind "OK" and "Cancel". That's the joke. And what it is, is an acknowledgement of, again, the human plane that this report, this app, does not meet your needs. It's in a way like if you could instrument your organization and find all of the "export to Excel" buttons that are being worn out, those are like the hotspots for you to go and improve things. That button being, click, click, click, click, click, click, click all day long, is telling you that there's a tremendous opportunity for improvement here, both in terms of time saved, but also quality of result. Quality of question that's even formulated. You mentioned questions earlier, asking good questions. Here's the problem. The ability to execute on answers and the inability to execute on answers, the friction, the inertia, that works its way upstream into the question- forming muscles. The question-forming muscles atrophy to a level where they fit the ability to execute on the questions. And so when you suddenly expand the ability to answer questions, it actually... You've got to go back and re-expand your question-asking muscles to be more aggressive, to be more ambitious. Jen Stirrup (00:55:52): Yes. I think sometimes the data-driven piece is trying to, in a way, subtly bring that back into play. It's okay to admit that we don't have all the answers and it's okay to admit that we need to ask questions. I think there should be more of that. Something that, certainly earlier in my career, asking questions was discouraged. It meant you didn't know it. It meant that you were vulnerable in some way. And I think as an industry, we need to encourage people to ask questions. I think with the diversity inclusion piece, try and make a conscious effort. If I think someone in the meeting is being quiet, regardless of the background, but at least I'm trying to watch out for that now, whereas maybe 20 years ago, I wouldn't have realized it, but sometimes people do sometimes need that extra help to speak up and speak out. They often don't know what to say or how to beckon to a meeting and say something. It's quite difficult. Jen Stirrup (00:56:51): Especially if you were being measured in your performance. I think sometimes people see things very confidently. And actually when you start to pick it apart, you think, "I need to as a person, stop believe in confidence and maybe thinking is that right, not how it's being delivered". I think they're stolen for quiet voices, hopefully like mine, who are trying to say things but I do find that harder to get heard. I think it's good that you do podcasts like this because I think it gives people the opportunity to talk about different ideas and how they impact people because that is important. There's loads of vendor podcasts that will talk all about the technology but we need to know better how to apply it. Rob Collie (00:57:31): When we were talking about starting this show, it was pretty clear we did not need another tech show. People who are working in tech, but are human beings, like yourself, and who are focused on helping other human beings. We weren't sure if it was going to work. It was one of those like, "Are people are going to listen?". Thomas Larock (00:57:45): We're still not sure. Rob Collie (00:57:50): We knew that we were going to like it, but yeah, it's building an audience. I've enjoyed it. And plus, it's an excuse to get together and talk with people such as yourself. If we just pinged you out of the blue and said, "Hey, you want to get on a two hour Zoom call with us and just catch up?". That's going to get pushed and pushed and pushed and pushed, but, "Oh a podcast? Oh, well, yeah. That's exciting". Jen Stirrup (00:58:14): Yeah. I know what you mean. It's good to, I think, to try and translate data and technology into something people feel is within their reach because I think there is still an element of people who are almost being scared of working with data. I deal a lot with CTO's, CIO. I was busy CTO and some way reports sent to their CFO because their CFO is over all of it, keeping costs down. The CTO has to work really hard to justify them. And I think what they want, ultimately, is not to appear stupid or not to know what they're doing. So some of these leadership conversations I have are about people saying, "Explain these terms to me. I don't know what a data lakehouse is. Do I need one? How's it different from a data lake? What about the warehouse? Is that going away or is that rebranded as well?". I know Microsoft talked about data hubs recently. If you're a data vault person, a data hub means something quite specific. It's been a term around for 30 years to mean something else. But I think sometimes people get very confused with the terms. Rob Collie (00:59:16): Like for example, the noun "dashboard" in Power BI, right? It's just a head clutching frustrating mistake. I mean a Power BI report is probably best described as a dashboard. The multi-visual, interactive experience, lowercase D dashboard is what I always want to describe it as, but no, no, no, no. We repurposed that word. Jen Stirrup (00:59:41): I know, and customers don't always understand it because they say, "Well, actually my report looks exactly like the dashboard. So I don't understand this publishing thing". So I have to try and explain that actually, we can take data from [inaudible 00:59:55] here and you can extra things. I'm interested to know actually, how much Power BI users spend actually making dashboards as opposed to making reports. And I just wish we'd ever the answer to that because sometimes you just want to get reports that they can run in their desktop or not always sometimes use a browser and just have the reports and have them open on the actual dashboards higher up. So I feel that's a bit of a separation that maybe wasn't required to have. But Tableau does something similar, doesn't it in a way? But I think with Tableau, it's a bit more clear that you're putting these things together. Rob Collie (01:00:29): Well, we were talking at the beginning about the importance of comprehensive training sets. Well, let me just tell you, we only need one data point here. I, as a Power BI user, have never once created an actual Power BI dashboard. So let's just conclude that that's it. No one uses them. But yeah, I've never felt compelled to need one. I tend to put together, what I need in the report. Jen Stirrup (01:00:56): Yes. And that's what I do because I'm trying to get the customer from A to B. I'm trying to do it quickly and I can see that they've reached on that tool ceiling of where they want to go and then they've got this other thing they need to do and they don't understand why. So sometimes it's a battle I just don't have because I just think, "You know what? These often been through so much to get to that point in the first place, cleaning data and getting access to the data and all the things that are hard and even understanding what they want in the first place". I try and work out where the fatigue is. Rob Collie (01:01:28): Yeah. I think there's a certain hubris just in the idea that a user will go around and then harvest little chunks out of other reports and take them completely out of context. Anyway, we didn't come here for cynicism today but- Jen Stirrup (01:01:43): I have plenty of that. Rob Collie (01:01:43): But it's still there. We can't really help it. So it's come up a few times and I want to make sure we actually make some time to talk about it specifically. So you've mentioned a number of times, inclusion and diversity and already a few anecdotes within your own professional organization, within your own firm. Outside of your own data relish organization, what are you up to in this space around the diversity and inclusion as a cause? You're very active in the community in this regard. Can you summarize for us what all you're up to? Jen Stirrup (01:02:15): Yeah. I've started there to talk more about intersectionality. There's a lot of data, which I don't have to hand, which is terrible, that shows that it's the intersection of people's lived experiences that can sometimes work together against that person. So for example, we know that women are paid less than men, and regardless of the stats that we use, that's the number that comes up. There is some data that talks about how for black women, it's even less and for Latin women, it's even less again. So the idea being that, for people of certain ethnicities and backgrounds, it's interacting with the fact that their background, their ethnicity, their race is interacting with the fact that they're female, and both of these characteristics together are interacting to produce an adverse outcome for the individual. Jen Stirrup (01:03:06): Now this is quite an interesting area, is something that's been part of academic research for about 30 years. And there's some great researchers out there that talk about this. I want to say [Christy Reynolds 01:03:18], but I need to double check that surname because I'm not very good with names. It's age. Which is another characteristic as well. I was called an interfering old bag by somebody. Rob Collie (01:03:29): What? Jen Stirrup (01:03:30): I thought was really funny. There was a community person. I don't know if they realized it would get back to me. And I said, "Okay, so diversity and inclusion, obviously it's there with the bag aspects, but old as well. Okay. Thank you very much". Rob Collie (01:03:41): Great. Jen Stirrup (01:03:43): And yes, I am interfering. You know I am. Rob Collie (01:03:46): Triple word score. Jen Stirrup (01:03:50): Exactly, and it's... For me, I just thought, "Okay, so it fits". It was actually somebody I know that said that behind my back and they told somebody else and they said, "So-and-so has said this about you'll and I said, " Okay, that's fine. I am interfering, absolutely. If I see something I don't like, I can realize that being a staunch supporter of certain initiatives, of causes, can make me appear interfering". If I feel strongly about something, I will speak out and that can be cast fairly quickly into, "She's interfering again. Who is she? She's an old bag". So I think that phrase for me, made think about those two things. One is my age. I'm 48. Jen Stirrup (01:04:28): So there's that and female as well. I thought whatever I was interfering about, does it matter if I was right or not? Because whatever I was interfering about has got lost somewhere and it's got lost in the fact that my other characteristics were brought up as well, rather than seeing, "Well, actually Jennifer is wrong about something", which I could have accepted, I think. If someone had said, "Actually, you've got that wrong", I would rather know that, because then I can rethink or change my mind or perhaps give an explanation and then it turns into an adult conversation then, rather than back and forth. Which I don't want to do, I am not interested. So I didn't engage with the individual. I thought, "I can't change your mind, because if you're just going to bring up my age and my gender, I'm not getting a good starting point. Maybe you need to listen to someone else and see what they think". Rob Collie (01:05:22): How do you achieve that peace? I'm sorry to interrupt, but that's just killing me. How have you reached the point or have you always been like this? Teach me. Can you teach me that peace? Jen Stirrup (01:05:35): I think I got to the point of fatigue actually, where I realized that no matter what I said or did, there was always going to be haters who would always cast whatever I said or did in a bad light, and it doesn't matter what I do or say, that will be twisted. Someone wrote me a piece of hate mail recently on Instagram, which said about, "You said something about this blog post, about this person", and I said, "No, I didn't". And then I thought, "Should I go back and ask them what was I supposed to have said and who about?". Jen Stirrup (01:06:05): In the end I just wrote, "Thank you very much for getting in touch. I appreciate it and I think we need to close the conversation here". And I just left it because I thought, "Actually, I don't know what's going on, but whatever it is, I'm going to lose". And I think maybe I did start to lose in some ways, because I thought I am just going to lose and I can keep being dragged around by other people and it's spending energy. I don't know if you've ever read The Art of War? Rob Collie (01:06:31): I have. Summaries of it anyway. I'm not sure that I've read the whole thing, but it's not very long, is it? Jen Stirrup (01:06:36): It's not very long and there's a bit in it that really speaks to me. It's the bit about... So obviously about war, but it talks about strategy and it says that you should regard your enemies... stand back from them and regard them as a boulder rolling down a hill, and if you stand back, they will eventually run out of energy because there's going to keep rolling. And the whole point about you as that strategy person or a tactician, is that it's all about timing and it's spending energy. And I realized actually, the best thing I can do for myself is be very careful in how I spend my energy and be very careful about my timing. So I think I've been trying to ignore stuff online for quite some time. I'm not going to pretend it hasn't been challenging or that it hasn't been a hurtful. It's been both of those things. Jen Stirrup (01:07:23): And I think that, because it would speak out a refute to do anything, people tend to believe bad things and they just think, "Where did all of this come from?". I've come to the conclusion as well that I think we talk a bit about mental health in the tech community. I think that people are struggling and have sort of come to the conclusion that some people who are throwing stones and things are actually not in a good place, because if you're a happy person you're not behaving like that. Jen Stirrup (01:07:50): I'm not seeing people are mentally ill, and even if it was, it's not a bad thing to be a mentally ill or have mental health issues. That's not what I think, it's not what I mean. I just think it's coming back to asking questions again and timing to say, "Actually, I'm going to stop behaving like that because that's not a normal or proportionate way to behave, and I have to perhaps seek some help from the right person. Whether that's a friend who's going to give me a very honest answer or perhaps stepping outside my echo chamber or my [inaudible 01:08:22] chorus, so that I get different perspective". Jen Stirrup (01:08:25): And as humans we like to flock together in terms of who we like, and also we go off into groups, perhaps through a shared interest and that for a community, and that's a good thing. But sometimes it's a bad thing. It's something I should probably mention before. I've been speaking to the police for a long time, telling them about online harassment, because some of it is really unpleasant and some of it I have had to report to the police. I've been dealing with the London Metropolitan Police and [inaudible 01:08:52]. We all have. I've handed them access to things like my Facebook account so that they can see some of the stuff I am being sent. I don't mean people seeing my Facebook messages. They are deadly dull and boring. I do not live an interesting life. It tends to be things like, "What would you like me to bring you back from the shops?". Jen Stirrup (01:09:12): And what I have found is that there's much more sympathy for victims of online bullying than there used to be, and it's too much to the point. I've speaking much more closely and going through a sort of community resolution process at the moment with someone. I don't know who they are due to protection. Isn't telling you that. But I have been speaking to the police where someone has been caught, questioned, and they're going to write me a letter of apology. And I think that is going to be tremendously huge for me, because all I want is an apology that it's just not going to happen again to me or to anyone else. So sometimes speaking up and speaking out, you don't see results right away, but sometimes if you stick at it, collect the data, collect the evidence, speak to the right people, sometimes you can get a result. Jen Stirrup (01:09:59): In the minute I feel full of nervous energy about it. I'm not pleased yet because I think it's going to take some time to uncoil from all of that. But I think the point I'd like to make is, when people are seeing things by other people online, there are consequences for the victim, but to also see that for them personally, there can be consequences as well. You know so somebody has now to spend time in the police station, which is taking them away from their work, they have to explain to the family, that kind of thing. So I think what I'd like to see in the tech community is more proportion, a sense of proportion. If I have upset somebody, they need to talk to me directly. I will hear them out and if an apology's due then yes, absolutely I would give someone an apology. Jen Stirrup (01:10:44): I had to apologize to someone a few weeks ago. I said something that inadvertently offended somebody. I had no idea. I was very upset about it and I learned how that had been taken and wrote them an apology in Facebook actually, which they accepted. But I wanted to give them the opportunity to say their peace and I talk a lot about people speaking up and speaking out, and I need to take it when other people take the time to speak up and speak out against me as well. And I did the right thing and I learned something about how something came across. I'm really actually grateful that they took the time to do that. Rob Collie (01:11:17): Just hearing you say these things live, it brings it home viscerally in a way that really no other medium does it. It's the cliche now in this increasingly hyperpolarized world, there are people quote unquote, on your side of these issues who are incredibly combative, unfriendly, non listening, right? There's people on both sides, right? That are like this. I'm not saying like, "Oh, it's the inclusion and diversity people that are so terrible". We've got those people everywhere, but it's just so hard to imagine talking to you, anyone characterizing you as a villain. It's just jaw-dropping. If I disagreed with you on every single thing in the world, I still... there's no way that I couldn't get along with you. Jen Stirrup (01:12:07): Thank you. Rob Collie (01:12:11): I really, really, really don't get it. Jen Stirrup (01:12:13): I think some of it is perception and some of its proportion. I think, as I've got older and I'm divorced... I've been through a bad divorce, and I think it had been true so much in some ways that I've suddenly developed a lot more empathy. And I think maybe you develop more empathy as you get older. So maybe it's that. Maybe if you spoke to 22 year old me, you would find a very different person because I am opinionated about lots of things and I do interfere, but I think maybe I've just got to learn a better balance between knowing when people don't want to hear it anymore. And me realizing that actually, you do need to speak up and speak out but when does that stop? I've been thinking more about this in the past few days actually. You can speak up and speak out, but maybe at some point I have to get better at understanding some people are just not going to listen. Jen Stirrup (01:13:05): And then that comes back to your previous question. How do you walk away from that? And I think you kind of have to, to protect yourself and maybe think about family members who are affected by seeing you upset. I think the community needs a lot of healing. I think past is appearing is not below any healing and I think [inaudible 01:13:24] over topics which are maybe not constructive and going to help anyone, is going to help. But I'm happy to talk about things like diversity inclusion, intersection and equality and inequality where people feel that perhaps there's something they could apply or maybe help them to think. And I should add that I am learning about these things all the time, and I can and I do get it wrong because I talk and ask questions with people because I am learning. And I'm very fortunate that people have been quite patient with me I think. Rob Collie (01:13:55): I just want to, almost sarcastically, go, "Wait, wait, wait, wait, wait, wait, wait. We're on the internet now. What is this humility and 'maybe I don't have all the answers'-stance? Like you can't do that. You've got to go out there and just bluff that you know everything and you don't understand why everyone else can't figure out something so simple". It's so easy on the internet to curate one's own profile. When you're up close and personal with someone, it's hard to hold it together. The human flaws that we all carry, they just kind of leak out if you're in the vicinity of each other and you're watching closely and if you're like working together in person or whatever on a daily basis. But if you're an internet personality or just really honestly, like everyone is becoming an internet personality in some way, right? Rob Collie (01:14:45): If you have a social media account, you have the opportunity to start broadcasting to the world a curated picture of your life. That's very, very, very seductive. Like, "Ooh, I can put out there only what I want people to see". You don't even need to consciously think about it. And then you get all these examples of other people who are doing it. It's almost like the prisoner's dilemma, right? If everyone else is doing it, but the thing is, you don't know that they're doing it, right? You see all these people that appear to be so together and so you need to go out there. The pressure to go out there and be the same, pretending that you have it all figured out, is super high. It takes a lot of courage to go out there and say, "Yeah, I'm figuring this out. Work in progress". Jen Stirrup (01:15:28): Yeah. I'd rather give an answer that's got integrity than pretended expertise I don't really have. I wish I had better answers. And maybe if you ask me in a year, I might have changed my mind. I think people are in a bad place in many ways and I've tried to be more thoughtful about actually maybe people are not accessing or getting the support and help that they need and that is coming out online. And maybe you just need a bigger sense of proportion that I don't think we get online on social media. Rob Collie (01:15:55): When there's an audience, everything gets orders of magnitude more toxic. For a while there, at a point in my life, operating my blog was a big part of my professional existence. I would get trolls that would come and attack out of left field for assuming just sometimes like incredibly bizarre reasons that you wouldn't even understand. I had a guy emailing my manager. They had tracked down who my manager was at Microsoft because I was still working there at the time and was emailing him that I had done something like I had not done at all. Like, it was crazy. He thought I hacked his PayPal or something. It was just totally out of the blue. He wasn't part of this tech community at all, right? That's the most egregious example, but I had an amazing, in the end, like an amazing track record turning these trolls. Rob Collie (01:16:39): Engage with them one-on-one privately behind the scenes and don't lead off with a punch. Lead off with, "Hey, hey, hey. What's what's up here?". And it was amazing. My expectation going into all these interactions was that this person is just unhinged, you know? And they turned into great people. I think like 9 out of 10... I think there was like one that didn't out of all the top 10 trolls from my history of operating the blog, nine of them turned into people that if I'm in their city, I'm going to say hi to them. Jen Stirrup (01:17:09): That's nice. I think you can try and give people an opportunity, but I think the other side of it is me thinking I have to look after my own mental health too. And I think it's better just to say, "Hey. My email address is online. I'm here on LinkedIn. There's always lot's of ways to get in touch with me. You can come and talk to me, I know". And I think maybe as you get older, you're better at picking your battles. Maybe it's a bit of that. Rob Collie (01:17:34): Yeah. Jen Stirrup (01:17:34): So I wish a [inaudible 01:17:37] face for people who are going through that. I've just decided just to screenshot, block and move on. And I feel that every blog post right now, I have to preface it with "This is not about aimed at any individual. This is just some observations about intersectionality". So I've got a blog that I need to publish about that and the last paragraph is about, " I'm not picking on anyone. I'm just trying to highlight an issue that-" Jen Stirrup (01:18:03): I'm not picking on anyone. I'm just trying to highlight an issue that is our various characteristics interact, and it's something that for me talk about diversity and inclusion, we focus on women in tech. And not that that's not a normal thing, because as it tends focus on white women in tech, and I'd like to see a mixture, women of color included as well. And a woman of color wrote to me last week, she said, "I'd like to share some of my experiences with you because women in tech is not including women of color." And I've actually got a meeting with her next week. I don't know what I can do or say, but I'm going to hopefully use it as a learning experience. It's not that I've said anything bad. She just said, "We're not talking about enough about women of color." And I feel it's probably fair, and I think it's important message. Thomas Larock (01:18:52): So I want to make sure, Jen, that you understand that I want to thank you for interfering. That's an important role that we all have to play at times. It's one thing if you're just always interfering for the sake of interfering, but when you see that interference is necessary to advance, to make something better for everyone, at the end of the day, we should all be looking to just simply do good things for ourselves, but for each other. So sometimes that interference is necessary. And I know you and I have interfered with things and tried to get things to a better place, and it's necessary and I don't want you to feel that you should stop or that you're not supported. If you ever need a pick me up, just call me and Rob, we'll spend a couple hours telling you how great you are. Jen Stirrup (01:19:41): Thank you. Thomas Larock (01:19:41): But interfering is necessary. The thing about the online, like Rob was talking about, I wanted to say to Rob, I think part of it is that when you're online, there's a tendency to feel seen. And what I mean is if I make a comment, and I've learned this over specifically the last five to six years, if I just make some comments about a behavior that I've witnessed that I think is bad, inevitably somebody online thinks I am talking about them, specifically. Like you probably think this tweet is about you, don't you? And it's true. I can track things out. You've had those trolls and I've had my share of trolls and abusers. And inevitably it always comes down. I feel you're speaking to me about this and therefore I'm going to stand up for myself. And I'm like, "How did we get to that point where you see something online and you think that person must be talking about you?" If that's the case, then you need to do some inward reflection about what you're doing instead of attacking the person for calling out a bad behavior. But anyway ... Rob Collie (01:20:51): It's like the old joke, if you go to work every day and there's like one or two assholes you run into, that's just normal. But if you go to work every and everybody's an asshole, you're the asshole. Jen Stirrup (01:21:00): I think it's a mix of confirmation biases. We've got these sort of bases in our heads. One is I want to say Dunning-Kruger, but I don't think that's right. And where we think we know more about something that we do. And the think there's a related bias, which I've forgotten the name of that, which is if we don't understand something, we think it's going to harm us. And I think that the bias is there if I've maybe put up a tweet or a blog post, they haven't understood what I'm saying. So then some way of thinking, she's out to harm me. And the two things together, I know a lot about this and I don't understand what she said, she must be able to harm me. And I think people are jumping and mixing biases. I've got blog posts in my head and it's really something I try and practice doing is, what do I see if I see something that I'm uncomfortable with? I've got better over the years. It's saying, excuse me. Jen Stirrup (01:21:59): And then, this is a Scottish thing, and I know people on the podcast can't see it, but pointing your finger is actually very Scottish thing. So I remember it being at [inaudible 01:22:08] a few years ago, when some of their guys were making comments with one of the female presenters and I pointed my finger at them and I said, "I'm speaking to you, lot. What do you think you're saying? Stop it." Now, I'm getting better at calling people out as I see it, then I used to be. And I think maybe it's partly because I'm more aware of it maybe but also I think maybe I'm seeing more of it as well, so maybe there's that. And then it's just confidence to go up and point fingers at people. Jen Stirrup (01:22:37): But I think something, a line I cultivate is, are you kidding me? And that works everywhere. So anything that you've seen ever. And I want to sort of blow post an axe at something I'm trying to get better at. And the reason for that is I was in [inaudible 01:22:54] a while ago and somebody said something against the Jewish person. I live in an area with a lot of synagogues. And they said something which I will not repeat and asked him if he was okay afterwards. But I just didn't know what to say. And I thought, I now have to practice situations where it will prepare me better for speaking out when I see something which makes me uncomfortable. And it's a sort of indicator of the world that we live in, that we have to practice and have a stock series of phrases. And I love to blog about this. And the reason I mention it is just because I'm still figuring it all out. And I wish I knew a better way forward, but I think that reacting to trolls, reacting to people online, reacting when I see something, I think if you're a nice person, you're not expecting something. Rob Collie (01:23:42): Yeah, it's surprising. Jen Stirrup (01:23:43): And it surprises you and it catches you off guard because you think, "Well I never think like that. Where did that come from?" Rob Collie (01:23:48): Just as a humorous aside, you mentioned that are you kidding me is proven to be very effective. I have not found this to be very effective in my marriage. It is exactly the wrong thing for me to say, even though it is a go-to. I keep thinking that this will be the time that this sentence works. No, it doesn't. Jen Stirrup (01:24:08): No I wouldn't recommend it to a spouse. I'm divorced. Rob Collie (01:24:13): So am I, so am I. This is marriage number two. I want this one to work. It doesn't mean that are you kidding me is the move. Don't do that. Jen Stirrup (01:24:25): Just if I see something. So if I saw that incident again on the train where that man said something to that gentleman, it's pier capping things on, I would be best to prepared now. Thomas Larock (01:24:30): I'll try it with Suzanne to get some more empirical evidence. We'll see how it goes. I'll report back next podcast. Jen Stirrup (01:24:37): Suzanne is lovely. I don't think you'd ever have to say that to. I remember meeting her and we discussed Scotland because she had favorite TV programs. Thomas Larock (01:24:45): Yep. Rob Collie (01:24:46): Oh, I don't have to say it, either. It's not like it's justified. I'm just, anyway, I think it's fair to acknowledge that my nine out of 10 trolls story. All 10 of these trolls were male, shocking, I know. I definitely benefited from being a guy while handling them. There's a guy language and a guy code, and there's a lot of subtext going on in an interaction between two men that is very different. Whether we like it or not, it's very different when a woman is the target of it, and the way that you deal with it. It's weird. Even over the internet, there's this almost primal behavior going on. There's this threat that's been signaled, and then if there's a man on the other end, he turns around with some version of, "Oh yeah." Jen Stirrup (01:25:37): Yeah. Rob Collie (01:25:38): And there's sort of a deescalation that happens at that moment. And I don't really think that those tools are available to women in the same way that it's available when it's a guy to guy interaction. That's tricky. Jen Stirrup (01:25:54): You're taught to be fair and move. Rob Collie (01:25:55): Yeah. If you use exactly the same words that I do in an email response, it's not going to come across the same way. It's not going to get you the same result that it gets me. Jen Stirrup (01:26:08): Yeah. Rob Collie (01:26:08): I just basically said some version of, "Oh, okay. Oh, come on. There's got to be some sort of ..." There's no doubt that your success rate with those same words would be different than mine. Jen Stirrup (01:26:17): Yeah, it can work either way, I think. Sometimes it's just like the old bag is speaking. I'm not going to listen to her. But the other side of it is, "Hang on a minute. She actually said something to me." And that can cause a reaction. So sometimes it can work either way. It's quite hard to know. And I think trying to codify that into a checklist is quite hard. But I just thought, how can I best get better at dealing with these situations? Because I think as we come back to the world after COVID, we'll probably see more of this similar misunderstandings. And a few years ago, I tried to set up a decency charter and a code of conduct in [inaudible 01:26:55] . And I was surprised because not that many people adopted it. I think only two organizations did. And I thought with a decency chart or say something like, "We will be inclusive. We welcome everyone regardless of color and age and everything else." I put that together. And people wouldn't sign up to it. And I thought, "How can you not? How can you not" I don't know. Rob Collie (01:27:17): I want to go read it. And I want to go find all of the incredibly controversial content that I object to in this charter. I'm just kidding. Jen Stirrup (01:27:29): I know. Rob Collie (01:27:29): I'm expecting it to be a hundred percent obvious innocuous. I can't wait to say, "Oh hell no, we're not adopting this." Jen Stirrup (01:27:44): I'm going to throw it out. That's getting a red pen through it. Thomas Larock (01:27:47): Rob, our blog posts will just be, "Are you kidding me? Are you kidding me?" Rob Collie (01:27:54): I've even, by the way, on Facebook a long time ago, I made a fake dashboard of all of the buttons and levers available to me in a conversation in my marriage and giving them all labels. But there's this one giant button in the middle of the dashboard that's, are you kidding me? It sort of represents me just failing over and over again. Thomas Larock (01:28:21): Put that in the show. Rob Collie (01:28:22): The completely stupid rob dashboard for ... anyway. Jen Stirrup (01:28:31): Yeah, I was just trying to find stock phrases that I could just have in my head to respond with really quickly. Where I could just, if I see something. There's that and another one is which I find works if I'm getting harassed in the street or something, is I just say, "Oh, grow up." No one can say anything when they're told to grow up. I've never, ever had a good answer. So say I'm just walking around, minding my own business, walking with a laptop and going back to my car. And somebody wolf whistles. I'll just say, "Oh grow up." And I've never had a good answer to that, ever. What I'd like to do is find these phrases. Where if I see something, it makes people stop and think. I don't know, I wish I had a better answer. I'm still trying to figure so much out and I think I'll figure it all out and then I'll day the next day or something. There's so much to learn. Rob Collie (01:29:17): I know, and all this acquired wisdom, you start to like really cherish it. Like I badly, badly, badly want to share as much of it as I possibly can with my children. It seems like such a waste to develop it and then it's all lost. Like Roy Batty said like tears in rain at the end of Blade Runner. You mentioned sort of having these stock phrases on speed dial. I read one time about this guy who had been in prison for a long time. And he says 20 years later after being out of prison, he still has this five punch combination that he memorized for himself on speed dial. At any moment's time, he remembers exactly what it is. He had to have that to survive. Jen Stirrup (01:29:56): Yeah. Rob Collie (01:29:57): Not a good sign. Jen Stirrup (01:29:58): No, it's not. Rob Collie (01:29:59): When you need these things. Jen Stirrup (01:30:00): Yeah, and we shouldn't need them, but it's unfortunate that we do. And [inaudible 01:30:05] do self-defense classes. They're so helpful. Rob Collie (01:30:13): So you're up to like a two to three punch combination now. Takes a little while to get to five, I suppose. Jen Stirrup (01:30:18): Someone tried to mock me in my local village. I'd got some sausage rolls for my son and had them in a little bag, in a baker's bag. And this guy came towards me and I thought, [inaudible 01:30:30] with his fingers. And I thought, "He's going to mug me to get my thing." I got nervous. And then I remembered all the self-defense training. One of my friends is a former Olympian for the UK in karate. And she's very much about, you need to be on your guard all the time. I attend her lessons and I'm not very graceful at any of it. But he tried to grab my bag. And she taught me, use your elbow. And all that stuff because it's muscle memory because I practice every week. I'm trying to lose weight. So I hit someone in the nose with my elbow and he jumped back because it didn't expect it. So he's twice my height and half my age and it's the last thing he expected me to do. And as he came forward, invited himself, I punched him in the temple and the side of the face. And I'm laughing now, but at the time I was absolutely terrified. And it wasn't worth it over the sausage rolls. I should've just given them up, but just the fact that we have to be on our guard all the time. Jen Stirrup (01:31:28): And if he'd said to me, "I'm hungry and homeless and penniless," I would have given him it. But just the fact that he didn't look any of those things. I think he just thought, perception, older female, I'm going to take whatever she's got in her bag. So when I punched him and he fell to the ground, actually, and I just walked away. And I guess that I never thought I would ever need self-defense, but I think these things that we see sometimes in our daily life are showing us that we do need some form of self defense against seeing things and empowering ourselves to see something when we do. BEcause I think that's one of the hardest things is speaking up and speaking out. And I know I talk about it a lot, but I find it tough. Jen Stirrup (01:32:14): And I guess my lesson here is can we develop some sort of self-defense when you look at community events where we have got a code of conduct and my decency charter. It doesn't have to have my name on it. I don't care if it does or it doesn't. I just want people to feel welcome and that we are doing self-defense and other defense if we see something. Because it's a shame because as Tom and I know, we spend hours if not days that pass talking about anti harassment. And we developed, I think, a great piece of work on that. I think that's something, a past legacy that we should be incredibly proud of actually, because we put so much work into it. Was it perfect? No, but we're back to 80/20 again. It covered 80% of cases, edge cases sometimes not so much, but that's why the red cases, because there was the things that you maybe, or I didn't think of, maybe tried to be as prepared as we could. And I think it's almost like we need one of those for everyday life. Rob Collie (01:33:11): You got in a physical altercation. Jen Stirrup (01:33:15): Yes, I did. Rob Collie (01:33:17): It sounds like you're knocked him out. Down he went. Thomas Larock (01:33:21): What I took away was she left him for dead. Jen Stirrup (01:33:26): Yeah. Rob Collie (01:33:28): She walked away to go after his family. Jen Stirrup (01:33:34): Walked away because it was me scooting away, because I thought I'm just going to scoot away because I've done this and I'm embarrassed. And I was full of adrenaline and I didn't want to run because then you look like you're guilty. Whereas he was coming after me. So I turned back about halfway down the street and he was dusting himself off. And actually a few weeks later I was in High Street and someone stopped me and they said, "I saw what happened with that guy. He's been doing that for some time and somebody needed to teach him a lesson." Rob Collie (01:33:59): So you interfered. Jen Stirrup (01:34:00): I interfered. Rob Collie (01:34:03): You interfered with the normal order of events, which is him harassing people and taking their stuff. Jen Stirrup (01:34:09): I mean, change not too much. What's your bio, actually. I think I've now made a life objective to be an interfering old bag. Rob Collie (01:34:17): So are there any sort of pithy, short tips that you would provide to organizations, sort of like easy things to remember, easy things to do, new habits to develop or old habits to discard that can advance the cause of inclusivity? Jen Stirrup (01:34:35): Be kind. The reason I say that is because I was on customer set a few years ago and I witnessed something and I'd rather not say what it was, but there was an incident I was unhappy with and nobody stuck up for that person. And it's just sometimes in technology companies, there isn't that culture of speaking out. And I thought, "Well, I'm going to speak out on behalf of that person." And I wrote a report, actually. They asked me to write up a report about it, which I did. And I said that, I should try and find it, but wrote this phrase about acting with kindness. Jen Stirrup (01:35:06): I think the other thing I think is something that Mark Sousa used to say a lot is that destiny that joins the passport and it has helped me so much is assume good intentions. So for me and other interfering old bags, that I need to remember to look for the good intentions. Because I think I can be quite a negative person and I'm not always looking for those. And I think that was an incredibly wise thing. Yeah, I've forgotten the actual phrase which I gave to that customer. But I did say something along the lines of, and I actually suggested even ambassadors of kindness because the culture was really unpleasant. I shouldn't have to tell companies this, right? Rob Collie (01:35:47): No. Jen Stirrup (01:35:48): Anyway, I left and I don't know if he did anything. Rob Collie (01:35:51): So many table stakes. Here's one, I haven't taught a class in a long time, but I used to teach all the time. And one of my observations from teaching classes in DAX and data modeling, power BI, whatever, is that the population of students in the room was always at least 50% female. On average, over time, it was slightly higher than 50% female. So the quantity was 50/50. And then quality-wise, my guess as to who the best student was in the class at the end of a couple of days was again, coin flippy, whether the person who I thought was probably the most promising half the time male, half the time female. And yet our company, the people who apply for jobs with us, overwhelmingly male. Rob Collie (01:36:36): I feel a number of things about this. First of all, I just feel that it's a terrible shame. There's something wrong here. You talking about women in tech and all of that, usually you don't start from a baseline like 50/50. We're in this place where we have an amazing baseline. We're already in a great spot. You mentioned things like asking the right questions and everything. I actually think that, might be unpopular to say it, but I actually think women are probably better on average at formulating questions than men. The talent base is there. Where are we failing? Jen Stirrup (01:37:05): Yeah, I think there's a few things happening. I tend to get lots of job applications of people speculatively finding CVs, and the sense you get a lot of women and people from LGTB backgrounds as well applying. And I think maybe some of it is about positioning the company. So if you position it as, "Yeah, we're all about the data. We're all about the technology. But there's our mission in there." Because I do work for DataKind who are a data science charity. They always get 50% new female inclusion and I don't really want to see 50/50 because some people are non binary as well. So there's a healthy representation there, I think. I'm not non binary, but we are attracting people from at least three genders and that's good thing. But I think the reason for that is I get some tacitly, technically astonishing women are doing fantastic things. And I think it's about the way that it's maybe marketed as being inclusive and secondly helping people and having a really good impact. Jen Stirrup (01:38:09): I think people tend to be incentivized in the workplace is that they're having a good impact or perhaps being salary motivated. And if you go in more with the mission and the purpose, then that's a good thing. I mean, that's just I'd have to look at your company website or any company website. But I think some of it is what do you blog about? Are you just maybe talking about technology or are you talking about making things better somewhere for somebody. And maybe that's quite general, but when I go into Datakinds data dive and I'm sat with at least 50% women, they are harnessing people of all different backgrounds. And I think it's about that we have a mission. We and our skills and technology expertise are part of us, but we're here all with a common purpose. And I think the tech community needs that as well. Jen Stirrup (01:38:57): I've often said I'd love to see Microsoft Ignite, for example, doing a Data Hackathon Day. And I think they've tried something like this sped would think it was dialed up enough. I would love to see them partner with Datakind as an example and do something really good over the course of a conference like that. I think I had the idea sort of after we left Pass. So don't think it was something I did as part of Pass. But the thing is, people will build a sense of community and the get better technical skills because they're interacting with people of all sorts of backgrounds. Jen Stirrup (01:39:29): One of the guys on my team is at Cambridge, University of Cambridge PhD doing a post-doc in genomics research. And he was doing location mapping for one of the charities that we were working for. And he was just phenomenally brilliant. I think when you deal with people like that donating their time for a cause, they tend to be nice people because they want to help. And I think technical community, speaking generally, it doesn't always have that. It's like, who's this serving? Is it serving our company? Is it serving the individual because they're building their career? I think when you talk with tech community, it's about, "Yay, come present and you'll increase your career skills and your technical skills." What about saying something like, "Hey, come along and you can help people. You can have a charity. You can have an impact in people you'll never meet." I think I'd like to see more collaborations. Rob Collie (01:40:19): Jen, thank you so much. I've really enjoyed, but I've also just really appreciated this conversation. Jen Stirrup (01:40:25): Thanks so much. Announcer (01:40:26): Thanks for listening to the Raw Data by P3 Adaptive podcast. Let the experts at P3 Adaptive help your business. Just go to P3adaptive.com. Have a data day!

We were absolutely thrilled to be able to sit down with Adam Harstad, who is basically Rob's secret weapon to dominate every year in our office Fantasy Football league! Fantasy sports is just one avenue that allows Adam to display his incredible art of storytelling through data. References in this Episode: Football Guys Forums Football Outsiders Imperial March Major Key Version The Map Is Not The Territory The Treachery of Images  Reactive Armor Zidane Headbutt Joey Harrington Article Blotto Game Ted Lasso Dart Scene(FOUL LANGUAGE) Episode Timeline: 3:30 - Adam is a HUGE fan of path dependency, SSOG and The Crying Elephant, the ups and downs of being a sports fan 17:05 - Data and Storytelling, Business data vs. Sports data analytics, 46:40 - Risk assessment, The NFL Draft, Sunk Cost examples, Adam finds his niche with Fantasy Football 1:08:10 - Single trial experiments, Calibrating predictions and models, and Super Forecasters Episode Transcript: Rob Collie (00:00:00): Hello friends. There was an old Monty Python movie titled And Now for Something Completely Different. I think that's an excellent tagline for this show, and today's guest, Adam Harstad. Nominally, Adam, is a fantasy football writer and analyst, but I think you'll see that that title or really any succinct title really struggles to describe and contain the human being that is Adam. A better term for him might be something like curious adventurer, because that really seems to be the way he approaches everything. Rob Collie (00:00:34): We went in a lot of weird and nerdy directions, as you might expect. Yes, of course, we talked about football. So if you're not into football, well, you can tune those parts out. But even there, when we're talking about football, we're really talking about the fundamentals of how to think and specifically how to think in the face of uncertainty. We also talked about the differences between sports analytics and business analytics, which of course are significant. And we revisited an old favorite topic that of Nate Silver versus elections. Rob Collie (00:01:04): I learned a lot of new vocabulary along the way, and I really must say that I felt pushed in this conversation in a good way. He was pressing us to be better, unintentionally, politely, but that's what happens when you have really intelligent, vibrant, capable people in your conversation. We all should be seeking out the Adams in our life because they pull us forward on that journey of improvement. And he's an entertaining fellow too, so let's get into it. Announcer (00:01:35): Ladies and gentlemen, may I have your attention please. Announcer (00:01:39): This is the Raw Data By P3 Adaptive podcast with your host, Rob Collie and your cohost Thomas Larock. Find out what the experts at P3 Adaptive can do for your business. Just go to p3adaptive.com, Raw Data By P3 Adaptive is data with the human element. Rob Collie (00:02:03): Welcome to the show. Adam Harstad, how are you, sir? Adam Harstad (00:02:08): I am super happy to be here today. Rob Collie (00:02:10): Well, I appreciate that. You're a really good sport. Talk about just blindly wandering into something like a volunteer. No one has had as little context for coming on this show, as we've given you, we've really given you nothing and you're still just all smiley, ready to go. Adam Harstad (00:02:27): Yeah. We got a family motto for our little kids, we wanted to create a sense of identity. This is who we are as a family and our family motto is Harstads try new things, so I'm pretty much up for whatever. Rob Collie (00:02:39): Damn, that's how this came about is I saw a tweet from you a while back that said, "Hey, if you're looking for a podcast guest..." You had this long list of things that you were down for. And I'm like, "I'm going to take him up on it. Let's see if he actually means it." And you totally did. Adam Harstad (00:02:54): Absolutely. Yeah. Rob Collie (00:02:55): I know you better than you know me because I've been following on Twitter and actually elsewhere for a very, very long time, not super closely because I'm not really that fanatical about fantasy football, at least compared to what I used to be, but you're a data guy. There's no two ways about it. And also, bring that humanity, there's so much creativity and that right brain stuff going on at the same time. And that's, to me, where those two worlds meet is just golden. I love that. Let me explain to the audience how I know of you. Rob Collie (00:03:28): And at some point it's only fair, I suppose we turn the tables and Tom and I explain to you who we are. There's random internet stalkers as far as you're concerned. Right? I first discovered you, I believe on the old Footballguys forums early 2000s and you were, I believe SSOG. Adam Harstad (00:03:36): Yeah. Rob Collie (00:03:36): So I was right. Adam Harstad (00:03:46): That was back when I was young and still somewhat of a tool, which thankfully I've outgrown. Tom Larock (00:03:51): What's SSOG? Adam Harstad (00:03:53): Yeah. Speaking of growing out of being a tool, I got on the internet when I was 16 years old and I needed an email address and Hotmail was the thing at the time. So I signed up for a Hotmail account and being 16 and extremely full of myself. My first email address was somesortofgenius@hotmail.com because that was the joke. I'd say something and people would say, "What do you think you are, some sort of genius?" Then I could respond back that, "Yes, I do think I am some sort of genius." Rob Collie (00:04:20): I do. Exactly. Adam Harstad (00:04:22): That lasted for a year or two and then I realized, this is a little bit much, even for me, I'm all about owning who I am, but this is a little bit much even for me. Then I got into the University of Florida, so I changed it to some Gator, which was relatively inoffensive. And then just over time, it had gone through multiple- Rob Collie (00:04:41): Relatively. Adam Harstad (00:04:42): Relatively, absolutely. Depending on where you're from, depending on who you're asking. Over time, it went through multiple iterations where the commonality was just the initials and so, it became a running joke. People ask what SSOG stands for, and at one point it stood for stuff and then after a while it just stood for whatever you wanted it to stand for. Rob Collie (00:05:00): Now you're just SSOG. Adam Harstad (00:05:02): It's a testament to the power of path dependency. And I think, if you ask other questions about other elements online profile, it's all the same thing. It's a reminder to myself that where we are is largely a function of where we started and the path we took to get here. Tom Larock (00:05:16): That's deep. Rob Collie (00:05:17): Yeah. It's a concise summary in a way of like everyone we've had on this show, we've yet to have a guest on this show whose career was a cold shot. We're still looking for that person who set out to be something in high school that they knew what they wanted to do with their lives and they went out and they did it and they love it and they're successful at it. We're still looking for that guest. Adam Harstad (00:05:38): You should meet my wife. That's her in a nutshell, which is- Rob Collie (00:05:41): Really? Adam Harstad (00:05:41): It's always amazing to me. I can't fathom that sense of who you are at that young of an age. I still have very little sense of who I am. I'm still working on it, but she's always known. Rob Collie (00:05:52): Wow. That's got to be really interesting, as a close up example. Adam Harstad (00:05:58): Yeah. Rob Collie (00:05:59): My initial reaction is, wow, that would be really invalidating for me, but then I recovered. I'd be okay. I'd be okay. I'd be okay with my zigzag path. Adam Harstad (00:06:06): I've managed to survive for 20 years now with it. Rob Collie (00:06:10): Footballguys forums were one of the handful of places on the internet back at that time, where if you were really crazy obsessed about fantasy football, which I was at the time, you could get exposure to some really smart people in the community and their opinions. Not everything posted on those forums was intelligent or worth following, but it was amazing how, after a while, you, as a silent lurker on those boards, I almost never said anything. Rob Collie (00:06:36): There's a handful of personalities like these seven or eight avatars personas that you start to associate with subconsciously like, these are competent people. SSOG absolutely was one of those. And so now all these years later, the mystery of SSOG, the origins of the acronym unraveled. This is the first time I'd heard that, but you also had the crying elephant is that Babar? Adam Harstad (00:07:00): I have no idea what it is, to be honest. Speaking of path dependency and everything about my online personas attribute to that path dependency. I was on a forum and I needed an avatar and Google image search was the brand new thing. And I'm like, "I'm going to play around with Google image search to see what it does." I went onto Google image search, and I typed in SSOG and I looked at the page of the results. But unbeknownst to me, I had accidentally... My left hand was shifted one key to the left. Adam Harstad (00:07:29): So I actually typed in AAOF and the crying elephant was one of the first results and I'm like, "This is really cool." And I love that story. That it's serendipity, that just my hand was shifted a centimeter to the left. And now I've attached it to everything to create this consistent online brand because this is back before people were using real names and having a persistent online persona. But I really like the avatar. I think it's a little cheeky. It's a little bit fun, a little whimsical. And it, I think is mysterious. People are always wondering who is the elephant? Why is he crying? Adam Harstad (00:08:04): But I also love it because it's a complete accident, and that is a good reminder to me sometimes when I need one. Rob Collie (00:08:11): You can definitely go check this out right now on Twitter, Adam Harstad. Is that right? Adam Harstad (00:08:16): Yeah. Rob Collie (00:08:17): You're not @SSOG, but you kept the SSOG icon, the original elephant icon, the AAOF icon as the case to be. Adam Harstad (00:08:26): Again, path dependency. I tried to register SSOG on Twitter first, there is an account on Twitter registered @SSOG that has one tweet. It says, "In a jerking mood." And that's the entire SSOG account. Rob Collie (00:08:41): Are you sure that wasn't you when you were- Adam Harstad (00:08:44): I’m positive. Rob Collie (00:08:45): ...half-awake one night? Tom Larock (00:08:47): [inaudible 00:08:47]. Adam Harstad (00:08:48): That's why I write under my own name as opposed to a pseudonym. At some point I probably made a transition. Tom Larock (00:08:54): That account is suspended. Adam Harstad (00:08:56): I think so. I think so. Just because it's registered in 2009, because I was on relatively early, not super-duper early, but I've been on for over 10 years now and that one beat me on and tweeted, "In a jerking mood." And then just never logged in again. Rob Collie (00:09:10): But that describes Twitter so well, don't you think? Adam Harstad (00:09:14): Absolutely. Absolutely. And they did the alternating caps and no caps. I love it. And that's why I write under my real name. Rob Collie (00:09:20): By keeping the elephant, I was able to connect you with who you had been on the forums. Are you still on the Footballguys forums at all? Adam Harstad (00:09:28): Not so much anymore. People think there's this grand conspiracy that once you start writing for Footballguys, that the bosses don't want you on the forums as much, which couldn't be further from the truth. It's more just, people have kids, people grow older, people have less disposable time. Rob Collie (00:09:41): Let's just go back for a moment. Florida Gators, I was raised with the Gators as a religion. I betrayed everybody and went to Vanderbilt. I was the first person in my family to break ranks, but then you ended up a Denver specialist, right? Adam Harstad (00:09:54): Yeah. I grew up in Colorado. Till I was 15, I lived in Colorado and then we moved out to Florida. I wasn't a diehard Gator or anything. It's just, I lived in Florida. It was in state tuition. It was the best public school in the state, so that's where I wound up. Rob Collie (00:10:09): Also, coincidentally, because I was raised so aggressively on the Gators. At an age when you're just defenseless, you're just going to take an imprint and it's going to stick. My favorite NFL team for a very long time was the Broncos because of the same color scheme. Adam Harstad (00:10:24): Orange and blue. Rob Collie (00:10:25): That's right. I suffered through all of those terrible blowout losses. I used Apple II Print Shop to make a banner for the Broncos and hung it on the front of my house in middle school just to have them get destroyed. Adam Harstad (00:10:41): At least they were making it though. Rob Collie (00:10:42): That's right. Adam Harstad (00:10:43): It's hard to complain too much as a Bronco fan. There was a three decade span where we had more Super Bowl appearances than losing seasons. Rob Collie (00:10:50): Yeah. When you think about being a fan for a sports team, it's a losing proposition, the probability. There's only one team that's going to end on a high note, that's it. There's one team. Take the number of teams in the league, divide one by that that's your percentage chance of ending the season happy. I just think the expected value of being a fan of a particular team is actually pretty low. Even the good years where you win the championship, I just don't know that that really makes up for the down. Rob Collie (00:11:19): I've become more of a sports fan for that reason. A little bit more mercenary. Here we go. Tom Larock (00:11:25): This isn't true. That's really extreme. First of all, you could say that about any sport, but what you really mean to say is only one team and so seasoned on the wind, but there's no way you're going to tell me that a Cleveland Browns fan didn't end last season happy to beat Pittsburgh in the playoff game after how many shitty seasons. They ended happy. They didn't win it all, but they're certainly happy. Rob Collie (00:11:50): Think about though, how the ground needed to be prepped for them to reach that level of happiness about mediocrity. Tom Larock (00:11:58): That's fair. Rob Collie (00:11:58): They had to have decades of misery. I just- Tom Larock (00:12:01): You’re talking to a Patriots fan; I know decades of misery. Rob Collie (00:12:05): I know. I just don't think that being a sports fan for a particular team was a buy and hold strategy. You inherently need to time that market if you want to win. Tom Larock (00:12:13): Buy and hold is interesting because as a child, my friends and I, we would pick one of the worst teams to start rooting for because we knew in 20 years, they'd be good and then we could say, we were a fan way back. I'll go get my Vikings jersey right now just to show you. Rob Collie (00:12:32): Any year now. There's this, and it's a well-known human tendency that we judge experiences predominantly by the very, very end. They've done studies where they'll play people, this beautiful symphony and then right at the very end, they end on a sour note and people will say it ruined the entire symphony. There's 30 minutes of music and it ends on one note. And I just think that's the wrong way of looking at it. Rob Collie (00:13:00): You had 29 minutes and 50 seconds of just the sublime transcended experience and 10 seconds of something bad and you end your season on a loss, but the season, isn't just the last game of the season. It's all the anticipation leading up. It's everything you do during the course of the year. A lot of troubles in life are just a matter of perspective and a matter of attitude. Tom Larock (00:13:20): I want to hear one of these now. Is it just a sad trombone at the end? Rob Collie (00:13:26): [inaudible 00:13:26]. Tom Larock (00:13:36): Because I'd just be sitting there laughing. This is the greatest thing ever. Rob Collie (00:13:40): The meta of it would immediately just like crash over me. Like, are you kidding, you designed an experiment like this? Adam Harstad (00:13:48): I think it doesn't resolve, you have the progression and it's supposed to resolve on a certain note and they're probably a half step flat. And I don't know if you know that much about music, but if something doesn't resolve, it creates this very real physical tension because you're anticipating it and you're anticipating and you're waiting for it. And it's really a cruel thing to do to someone, but at the end of the day, it doesn't change the first 29 minutes of the symphony. Tom Larock (00:14:12): Tomorrow night I'm going to Tanglewood to see John Williams in the movie night. Now he's 89. I'm pretty sure he'll still be alive for tomorrow, so that's good. But now I'm just thinking he's going to play the Star Wars theme and at the end of it, it's just a sad trombone. I'm just going to be sitting there laughing at the end of all these songs. People are going to be like, "What is wrong with that person?" Rob Collie (00:14:36): All right. I don't know anything about music theory. I freaking love music. I'm always, always, always deep, deep into music and stuff that I appreciate is usually like on the more complex end and I don't know anything. Let's talk about this for a moment. I heard the Imperial March one time in the major key instead of the minor key. The minor key Imperial March is very foreboding. The major key version sounds like this upbeat, let's go have fun. It's like a clown parade. Explain it to me like I'm five minor key versus major key. Rob Collie (00:15:13): Have we come to the point where I'm going to understand this in my life? Adam Harstad (00:15:16): The simple version is minor key sounds sad, major key sounds happy, but that's too simplistic. There are happy songs written in minor keys. There are sad songs written in major keys. It's been a lot of years since I have last taken any music theory courses and that's unfortunately some of the knowledge I've lost during my life. Rob Collie (00:15:32): Well, that's okay. Path dependency again. Right? Adam Harstad (00:15:35): Right. Rob Collie (00:15:35): I've also heard though that it's cultural, certain cultures will actually find the major key to be the sad one and vice versa. Adam Harstad (00:15:43): Yeah. Most of our experience of music is just based on historical context and our exposure to music. What sounds normal. A lot of our enjoyment of music is actually anticipation. There's pattern recognition and there are certain patterns that are common through music and the brain really loves anticipating those patterns and then when the patterns resolve in the way that we expect them to, the brain rewards us with a jolt of pleasure like, "I knew that was going to happen. That was very enjoyable and pleasant." Adam Harstad (00:16:10): Sometimes if the patterns resolve in ways, we were not expecting brain results us in a jolt of pleasure because this was fun and new and exciting. I'm eager to see where it goes next. And so, our experience in music is heavily shaped by the patterns and our experience to music prior to that. We can listen to a Western pop song and we're very familiar with the genre, we're familiar with the tropes. We're familiar with, we're going to go first course first. Maybe sometime we'll have a coda. Adam Harstad (00:16:40): There's the common building blocks of music and artists will change the order of them, they'll change the way they're put together. They'll change the makeup of those blocks, but they're building with the same blocks, whereas we can listen to Eastern music and it just sounds very foreign to us because we don't have that language. We don't have that fluency in it yet. Tom Larock (00:16:58): It's such an interesting topic, almost like this anticipation. I totally see that. Rob Collie (00:17:02): And there are some quirky misdirections in music that you wouldn't necessarily expect, but that are incredibly fun when they spring them on you. It's not just personality that I've been drawn to that it kept me reading your stuff over the years. And in fact that wasn't at the beginning at all right. It was the quality of the insight and most of it is data-driven. Most of it is analytical. It's not just about the evidence that you collect. It's also about the way that you process it that I have found valuable over the years in trying to defeat my friends and colleagues in a silly game. Rob Collie (00:17:35): How did you discover your passion for data? Adam Harstad (00:17:38): It's funny. I came at it obliquely. I've got a very strong left brain, classically left brain skills, math and seeing patterns and interpolation, things like that. But I think my love of football and my love of fantasy football has actually always been more narratively driven. And I like to say first and foremost, I'm a storyteller. And it's funny you say I'm a data guy. I know data guys who think I'm one of those narrative guys, narrative guys think I'm a data guy, I'm in that liminal space between. Adam Harstad (00:18:08): For me, it's all about, I have these pressing questions and I think the questions are the interesting part. And I'm looking for any insight, knowledge, any edge I can get to answer those questions. And getting back to path dependency, when I was getting into fantasy football in early, early 2000s, that was really the unexplored space. There's a website called Football Outsiders that was just launching and they were actually looking at data and they're saying, let's compare every play in this situation to every other play in the situation. Adam Harstad (00:18:40): If you ran for four yards in first and 10, was that a good play? Was that a bad play? Nobody had ever really dug into it. And so, that was the edge. That was where the edge is. And over time as analytics has gotten to be a bigger and bigger part of the sport of football and the hobby of fantasy football, I find often the edge is in this, not really contrarian to analytics, because I think analytics is good and useful and right. Analytics is a way of approaching problems by looking at the data and that's a very, very useful tool. Adam Harstad (00:19:13): But I think that there's some conventional wisdom that should apply to analytics that has gotten lost along the way in the rush to mine more data. So for me, it's never been specifically about the data. It's been about the questions and the answers. And for most of my career, the data has been the best way to reach those answers, but it's not the only way. It's more about finding the appropriate tool for the appropriate situation. Rob Collie (00:19:37): I think I'm starting to understand why I felt this remote kinship with you asymmetric, because that's how the internet works with lurkers and publishers. Our team is growing really rapidly at our company and we're a remote team. We're all over the country, even though we're full-time employees, it's a good problem to have, but it's become difficult to keep track of who everybody is. Name, face, okay. But what about personality? It's so hard. Yesterday we started collecting information to make flashcards for everybody. Rob Collie (00:20:11): Everyone had just to pick a "superpower" or something like that, but not something super professional, but like storyteller. I picked storyteller for me, folksy storytelling would be one of my, "Look out, he's going to do it." And then the football, I'm a data professional. There's no two ways about it. I worked on data tools at Microsoft, I worked on Excel, I worked on power BI, I'm CEO of a data analytics, business intelligence consulting firm. Yes, data professional. But that all started for me in the late '90s with the same sorts of things as what you're talking about. Rob Collie (00:20:45): Like how do I actually approach this game, fantasy football? How do I approach it more effectively? I stumbled upon the original value-based drafting article years ago, which put the different positions into perspective in terms of relative value. And I was just like, "Ugh" It's like the holy grail. It was this magical unlocking moment and what followed was just reams of spreadsheets and coding and all kinds of things. It was the only times, even though I was a software professional already, I was already working at Microsoft. Rob Collie (00:21:15): I was never really that into it. It was one of the few times that I was actually into it was when I started doing these sorts of things. Another thing that you wouldn't have the context for is that on this podcast over and over and over again, we talk about how we observe even, the value is in the hybrids of different centers of mass. In the business world of data, it's long been like these completely separate universes where IT was doing some data stuff and business was doing some data stuff and they almost never collaborated. Rob Collie (00:21:51): And really the software was built to create this divide in the old days, but now there's this rise of these IT/business hybrids. And again, it comes back to all these sorts of things that you were saying conventional wisdom is a shortcut for saying all kinds of things, all kinds of developed expertise and wisdom that you can't just set aside when you're using data. You need to integrate the two. IT can come into a business situation and bring all kinds of technical skill, but they've got a complete blank slate when it comes to the business knowledge of what's actually going on. Right? Adam Harstad (00:22:28): Right. Rob Collie (00:22:28): It turns out that you can't have one or the other, you have to have both. And that just keeps coming up on this show for good reason so I think there's a fundamental truth that we're experiencing there. Adam Harstad (00:22:39): Yeah, for sure. I think where fantasy football at is gotten a little bit too disdainful of other forms of received wisdom. There's this idea that what the data says is true, which is usually the case, unless you're making mistakes, analyzing the data, which is very, very easy to do, but there's also the sense that what isn't coming from the data is not necessarily false, but suspect. And as an example, there was a recent debate about whether throwing two running backs is a positive or negative for NFL offenses. Adam Harstad (00:23:11): And the data guys looked at it and they said, if you look at the expected value of every play, passes to running backs are less valuable than passes to wide receivers, which is true, I'm sure that's exactly what the data says. They're smart guys. They know how to analyze it. I'm sure that they're controlled for the appropriate confounders, various other pitfalls, but at the same time, I'm looking at it like, that may be so, but if you look at the best offenses of the last 15 years, disproportionately, they've thrown to their running backs a lot. Adam Harstad (00:23:40): You look at Sean Payton's New Orleans Saints throw to their running backs more than any team in the of football, and it's hard to find any offense in history that has had as much sustained success as the 2006 to 2020 New Orleans Saints. Bill Belichick, New England Patriots, you have James White, Shane Vereen. They historically have had a back whose only job is catching the football. Danny Woodhead was there, and it's hard to look at these and say, throwing to running backs is a bad play and yet a lot of the best offenses in history do it a lot. Adam Harstad (00:24:13): A lot of the worst offenses in history do it a lot too. I don't think it's a silver bullet, but it's hard for me to reconcile the idea that this is just outright bad with this observation that it's so often successful. And so, to me, the received wisdom is maybe we have reasons to be suspect of passes to the running back, but I can't just outright declare it bad, but that's not coming from data. That's coming from anecdote and observation, which is a powerful tool for receiving insight that I don't think is necessarily reckoned with enough. Tom Larock (00:24:45): On that, what I would say, and I got a whole lot I want to talk about when we talk about bias and things like that, and I just followed you on Twitter so I could retweet your thing about selection bias. I started fantasy football in the mid '90s, roughly. Well, I think we all did when it was really up and coming and no offense, but I'd never heard the Footballguys until today. I knew the Football Outsiders and I knew a handful of websites and this was before even ESPN or CBS Sports was offering this data, so I've been there. Tom Larock (00:25:16): What you're talking about just now with the throwing to the running backs, what hit me was all the data that you're examining is historical and you have no idea of what's going to happen next. You can say, everything we do in fantasy football, all we're doing is managing risk. I have a risk of scoring zero points, how do I make sure that I'm at least scoring something and that's all we're doing. We're just looking for a way to make sure we're producing some value on the field, so to speak. Tom Larock (00:25:48): Anyway, let's look at, if you could, as an example, before the 2007 season, when the Patriots had Moss and Wes Welker, nobody had any idea of just how many points that team was going to put on the board that year, because historically it just didn't happen. Maybe there was an offense from the 40s that ran something similar, but probably didn't have the same numbers generated. But when you talk about Belichick and a lot of coaches in the league, everything old is new again. Tom Larock (00:26:20): He just says, "All right, what talent do I have? Where would they excel?" This type of a formation. It's not really hard. They just look and say, "It's fairly basic. You need to do these things to be successful. Wes, go into the slot, run that way. Brady might hit you because Moss is just going to go de..." My point is simply, you can do all this analysis, at the end of the day you still don't know what's going to happen if somebody gets hurt. And to me, like I said, it's just managing the risk of scoring zero points. That's how I look at it each week. Tom Larock (00:26:52): I have a risk of scoring nothing, I can actually in this league be negative. How do I avoid that? Adam Harstad (00:26:57): One of my favorite articles I wrote was actually, you guys are probably familiar with the phrase, "The map is not the territory."? Rob Collie (00:27:02): No, I've never heard it. Adam Harstad (00:27:03): It's this idea that a map obviously is not the territory that it represents. It's a map. If you draw a mountain on the map, it's not like a mountain is going to magically spring from the ground and it's a representation. And I do a lot of historical modeling and that's a lot of my process is I will look at historically comparable players. Ja'Marr Chase comes in, what's the history of highly drafted rookie wide receivers? And from that, I'll create my baseline expectations. You need your prior and you can adjust your prior from there, but it's good to have that strong prior to start. Adam Harstad (00:27:38): But the problem is, let's say that one of the historically comparable players for Ja'Marr Chase is Larry Fitzgerald, that's going into the mix. If Larry Fitzgerald had torn his Achilles tendon as a rookie and never played another snap, the comparable players for Ja'Marr Chase would be worse, but his prospects wouldn't be any different. Realistically, it's not like Larry Fitzgerald tearing his Achilles would make Ja'Marr Chase more likely to tear his Achilles. Adam Harstad (00:28:05): Similarly, if someone like Charles Rogers who was a colossal bust had gone on to become a Hall of Famer, the comparable players to Ja'Marr Chase would look better, but Ja'Marr Chase's prospects wouldn't change at all. And it's something that I think you really need to reckon with when you're doing this kind of historical modeling. You need to remember that the map is not the territory. Every player in this situation has done poorly before, it doesn't mean that this player is going to do poorly now. Adam Harstad (00:28:28): But the full quote is, “The map is not the territory, but it has similar features accounting for its usefulness." If it's drawn correctly, it will resemble the territory enough that you can use it to navigate and it'll get you where you need to go if it's a good map. But the famous René Magritte has the treachery of images. It's the picture of the pipe and underneath he writes, "This is not a pipe." And everybody gets really mad because they're like, "Of course it's a pipe. Of course it's a pipe." Adam Harstad (00:28:54): And he said, "Okay, well, can you stuff it? Can you light it? Can you smoke it? No, it's not a pipe. It's a picture. It's not a pipe." The map is not the territory, but it shares similar features and that accounts for its usefulness. Rob Collie (00:29:06): We still need maps. Just be careful how you use them. Adam Harstad (00:29:09): Lewis Carroll wrote once about a country that created a map with a scale of one mile equals one mile, but then they didn't find it that useful, so now they use the territory as its own map and it serves almost as well. Rob Collie (00:29:20): Yeah, it's perfect actually. Tom Larock (00:29:24): That sounds more like a Steven Wright joke. Rob Collie (00:29:26): Yeah, it does. Sports, data, sports analytics. I think it draws a lot of the same personalities to it that business data draws, but the similarities between the two as domains, as professional activities, or even as hobbies, they don't actually share that much. There's some really, really, really important differences between the two. One of them is that sports is adversarial. It's a hundred percent adversarial. It's one team versus one team or one player versus one player. And the business world isn't like that. The business world inherently, it's competitive, but it's much more collaborative. Rob Collie (00:30:09): People you're interacting with are your customers, not your direct competitors. And so, that changes everything. There's just so many things that happen. Like when we were talking earlier about throwing to the running back, for instance, one hypothesis we can form is that if you're effective at throwing to the running back, you maintain an information advantage over your opponent. You keep them guessing, you're unpredictable in a way that works to your advantage. Being unpredictable doesn't work to your advantage in business. Rob Collie (00:30:40): There's nothing to be gained by, "You didn't expect that." It's just not like that. Tom Larock (00:30:46): You mean every time Google kills a product, that's just not helping them? Rob Collie (00:30:50): No one expected Microsoft to buy ProClarity and basically put an end to it back in the day and that was just good fun. Kept things interesting. Adam Harstad (00:30:58): Kept everybody on their toes. Rob Collie (00:30:59): Yeah, that's right. You never know. We might buy the number one front end for our server and basically retire it. Anyway. They don't do that anymore. That's the old Microsoft that did that kind of thing. Adam Harstad (00:31:13): Monopoly rules. Rob Collie (00:31:14): Yeah, that's right. And also, software engineers running the show. Anyway, we're not going to talk about Microsoft too much today. The other thing about sports is that almost everything that you would do analytically with it is predictive in nature. Whereas in business, not everything, there's still absolutely predictive stuff that needs to be done in business and we do some of that for our clients. But like, let's say if I was speaking to the other audience, the sports analytics audience, who's never worked a day of business analytics. Rob Collie (00:31:45): The analogy I would give you is, what if you only found out the score of the game, that's all you learned was the output like, "Shit, we lost again. Bad this time. 31 to 10." But you didn't get to watch the game. You don't get any box score and we're the ones that use all of this incredibly fine grain data that's happening in the business to eventually explain why things are happening the way that they are. Even just knowing where you are, knowing what's actually happening. Rob Collie (00:32:12): There's a joke in our industry, business intelligence by bank account, which is you get to the end of the year and you're like, "Hey look, how much money is in our account. Way to go team." Or, “Damn. Terrible year. Everybody just grit your teeth and try harder next year." That's reality. The lights are out by default and it's a tremendous oversimplification, but at the same time, there's a lot of truth to it, which is, we go around turning the lights on, even just explaining what has happened. Rob Collie (00:32:41): Typically, we don't want to be like on a one-year delay to understand what's happening, we want to know early in the month, what you're trending towards and why, so that you have an opportunity to make changes. It's all about forward changes. So many differences about sports analytics versus business analytics. And yet we're all moths drawn to similar flames. I watched the skills that you've developed over the years as I've been watching you, Adam, are very different than the skills that I require in the business that I'm in on a daily basis. Rob Collie (00:33:13): There are outlier applications for a lot of the things that you do in business. Absolutely, but it's like the average task, the average analytical question in sports versus business is really completely different. Adam Harstad (00:33:27): Well, and I think a lot of it too, is that in sports, the whole possibility space is so constrained. If you have an offense at the one-yard line, there's basically a hundred possibilities for that play. They can gain zero yards, one yard, two yards, three yards, so on and so forth. Whereas, if you have a startup at the business equivalent of the one-yard line, the possibility space is just so much larger. They can succeed, they can fail, they can succeed and fail in old ways. They can succeed and fail in new ways. Adam Harstad (00:33:56): Really, everything's on the table, but sports because it happens in a clearly defined arena of competition, under rules that are laid out to all the participants in advance and everybody has an opportunity to study, maybe some people are more familiar with the rules than others, but they're all constrained by the same rules. The world works in a very predictable and ordered way in a way that the real world outside of sports just doesn't operate. Adam Harstad (00:34:21): And so, yeah. It doesn't surprise me that there's not that much carry over because it's such an arbitrary space to analyze. People talk about sports as a metaphor for life, and maybe it works as a metaphor, but it's not really a great analog Rob Collie (00:34:34): At the heart of it, let's get metaphysical nerdy. Adam Harstad (00:34:36): Totally. Rob Collie (00:34:38): In the history of warfare, which has actually a lot in common with sports, whether we like it or not, that adversarial nature, the only constraints are the physical world. At any point in time, there's never been impervious armor. The armor of the day has always been vulnerable to the weapons of the day. There's never something that, you could just build, it was invincible. Of course, armor of today probably could stop everything from 2000 years ago, but at that particular point in time, and why is that? The same materials, right? Rob Collie (00:35:09): Same materials, same manufacturing, same technology is available to both weapon makers and armor makers. It's just that the weapon, only the weapon "knows" where it's going to hit. There's an information advantage that the weapon has. The armor has to be prepared for everything, but the weapon gets to pick whatever spot it happens to hit. There's an information advantage for the weapon and like in football or really in sports, going back to the thing we were talking about, the being unpredictable, but still effective seems to be like a crucial element of a well-designed offense. Rob Collie (00:35:42): And guess what? There've been very, very, very few defenses in the history of the NFL that are capable of shutting out their opponents, very similar to the armor and weapon thing. Information conveys a very, very specific advantage in adversarial situations. Of course, information also conveys an advantage in business. I can't put this topic down. You seem smarter than I am, so let's see what you think of this. Adam Harstad (00:36:05): I seem that way, for sure. SSOG, man. Projection. It's a careful projection of certain appearances. Yeah, I'm thinking about it. I think a lot of it's probably an incentives issue too. I don't know that there's the incentive to build impenetrable armor, wholly impenetrable armor like there is to build unstoppable weapons. If you look at history and you're mentioning that war is very zero sum, the societies that survived and the cultures that survived are the ones that aggressively expand it. Adam Harstad (00:36:38): If you had a hypothetical city state that just wanted to just sit in their city and never expand territory, eventually they're going to be overrun. And so it's almost like the rules of the conquest favor the weapon making over the armor making. You need armor that's just good enough to allow you to use the weapons that you've created. I completely agree that information asymmetry, especially in zero sum competitions is very important. One game theory concept I go to a lot is called a blotto game. Adam Harstad (00:37:11): And a blotto game is a game between two people where you each have a certain number of resources. Say you're a general and I'm a general and we're contesting three battlefields. And we each have five units that we can send to those battlefields. And whoever sends more units to a specific battlefield is going to win that battlefield. And whoever wins more battlefields is going to win the war. You can allocate your units however you want. Adam Harstad (00:37:33): You could send all five units to one battlefield, which will guarantee you that you conquer or at least hire that battlefield, but then you're leaving the other two battlefields undefended and the other general will almost certainly win the war. That's definitely a losing strategy you could send and there's also, you could do three, one and one, you could do two, two and one and you can evaluate all the possibilities and some of them will be Pareto optimal, some of them will be completely dominated by other strategies. Adam Harstad (00:37:58): There's a strategy that will always lose to another strategy or at worst tie it. And you can study these blotto games and there's variants on them. Say one side has more resources than the other. Say that some battlefields are worth double points. It's different if you're just going to play at once or if you're going to iterate it over and over again. But the best advantage you can have in a blotto game is having more resources than the other guy. If you have 10 units and I have two units, it's going to be trivially easy for you to win. Adam Harstad (00:38:26): And then the second best advantage you can have in a blotto game is knowledge of what the other guy is doing. Because if I know your strategy, it becomes very easy for me to just pick the strategy that defeats it. And in zero sum endeavors, fantasy football, among other things. I find a lot of instances that if you look at it, it really is just a more expensive version of the blotto game. Even football itself, I would call something of a blotto game. An offense has five receivers. These are its units. And it's trying to capture territory. Adam Harstad (00:38:58): It's trying to get a receiver into an unoccupied space by the defense so that it can complete a pass and progress the football, and then the defense has its seven defenders and it's trying to defend this territory. And really there are usually on most plays seven guys in coverage versus five guys in pass patterns and yet the offense wins more than the defense. And the big reason why is because the offense knows what it's going to do, it knows where it's going. Adam Harstad (00:39:23): It has that informational advantage and especially quarterbacks, Peyton Manning famously could read the defense before the snap and know what it was going to do. And so, it was much easier for Peyton Manning to win that particular blotto game than an average quarterback, because he had this fore knowledge of how the defense was going to be deploying its troops, what battlefields it would be contesting and which battlefields it would be leaving relatively open. Rob Collie (00:39:43): Have you been exposed to the concept of reactive armor? Adam Harstad (00:39:47): No, I haven't. Rob Collie (00:39:48): To me it's genius and it's only something that the Russians would have come up with and they had it back in the Cold War. If we look at the information advantage that the giant metal dart fired from NATO tanks has, it only has to worry about the place where it hits. All of its work is concentrated on this little square centimeter or whatever of armor that it impacts. And all that other armor is useless. All that weight that the tank's been carrying around, except for this one little column of metal that's in front of the dart, all the other stuff doesn't participate in his interaction at all. Rob Collie (00:40:27): And the Russians said, that's the problem, so when we get hit with a dart, we're going to have an explosive go off within our armor and slide the armor sideways in that section so that it's constantly feeding fresh metal into the interaction. It's crazy. And when the iron curtain came down, when the Berlin wall fell and NATO took some modern Soviet tanks out and tested them to know if the darts would go through, this armor stopped everything. And what was the predictable NATO response given the way that we think? Rob Collie (00:41:04): Just make the darts longer. But we see this, the defenses that are able to react like in football, more nimbly and quickly to what they see in front of them are the ones that are... There's no overpowering the offense when the offense has the information advantage, even though you have more players. Adam Harstad (00:41:25): There's one aspect of the defense, the pass rush actually is the whole situations are reversed when it comes to defending the quarterback. It's the offense that is on defense so to speak, that's reacting. And it's the defense that gets to choose, we're going to pass, rush these players. They don't know who's going to be rushing. We're going to rush from these angles, we'll be doing stunts to try and create confusion. So, for the most part, on a grand level, of course, it's the offense that's reacting. Adam Harstad (00:41:48): But it's interesting that there's also that sub competition at the same time where the defense has the first mover advantage. And if you look at player salaries, they largely reflect that. The players who get paid the highest are the wide receivers who have the informational advantage against defenses and then they're the defensive ends who have the informational advantage against offensive lines. Rob Collie (00:42:11): Interesting. Tom Larock (00:42:14): Every sport is action reaction. Adam Harstad (00:42:17): Yeah. Tom Larock (00:42:18): All right. Every sport is action reaction. What I find interesting about football is, and it's probably because it's easier to see a mistake in football, I think than in other sports. And to me, football is always, maybe it's because I hang around Patriots more often, and this is why I watch it's about just limiting mistakes. You already know, Peyton Manning, he's already figured it out and he's already got a play. He already knows what to do to get a few yards. Your job is to make sure he only gets those few yards. Tom Larock (00:42:51): So you limit the mistakes that you're about to make, and you keep the play in front of you. And I think when it comes to other sports, I think baseball, the obvious mistakes, somebody falls down, but there's a lot of base running mistakes. Like I didn't take the extra base like I could have. Things like that, that you don't really see unless you're really in tune with the game. Basketball is the same where mistakes are made. The obvious one, I dribbled off my leg, but the not so obvious one is you didn't set a good enough screen. Rob Collie (00:43:18): Well, the consequences. The consequences are so high in football. A mistake in football can have ungodly implications. Whereas the worst thing that can happen in a basketball play is the other team scores, three points, four points. They score something like 4% of their total for the game and then the slate is clean again. Adam Harstad (00:43:34): Well, by that standard it would be soccer. That's the worst because the consequences of mistakes in soccer are catastrophic. Rob Collie (00:43:39): Yes. I concur. For example, headbutting the opposing player in the final minutes of the World Cup final, because he insulted your sister or something. To me that is the most leap off the couch moment in all my sports watching history- Tom Larock (00:43:57): Really? Rob Collie (00:43:57): ...was when Zidane headbutted that guy and got ejected. Tom Larock (00:44:00): Not when Tyson chewed Evander's ear off? Rob Collie (00:44:05): No, because you know what? Two whole nations hung in the balance at that moment. Who cares about two guys punching each other? This is national pride. Tom Larock (00:44:15): He wasn’t punching him, Rob, he chewed his ear off in the rain. Rob Collie (00:44:19): That's fine. Zidane's headbutt still takes it for me. Tom Larock (00:44:22): No, that's fair. Yeah, that was a remarkable moment. Adam Harstad (00:44:26): I Agree. The consequences of a mistake. What's the max mistake? Tom Larock (00:44:30): In business it's the same thing because you want to mitigate risk. There's a lot of parallels where you're going to do these things and you know that say, Excel, isn't going to ship in time. What's the risk that we're after falling behind other competing products? We can do three of the four things in order to ship on time. The one thing that we're not going to do is going to be the least costly for whatever, but that's how the world works. It's not always about being perfect, it's about making sure that your shortcomings, your mistakes are limited. Rob Collie (00:45:04): Adam, you said that at the beginning that you have a saying at your house, Harstads try new things. I don't think we've ever really codified it, but we like to say Collies play offense. In business, it makes sense to play some defense for sure. Defense is also like the luxury of businesses that are already successful. On the way up, you can't defense your way to the top. You've got to offense your way to position of value with your customers. Rob Collie (00:45:34): We don't have to talk about it as an adversarial conquest thing. You have to create something in order to stand out, you have to create something in order to change the world. I think business is inherently again, until you reach the top and become the monopoly and then suddenly you're like, 'Yeah, build walls, build walls." Except for those rare cases, it's always about create. It's always about leaning forward and being on the creation side, the offensive side of the game. And even as a life philosophy, members of my family, my extended family, they play the game of life. Rob Collie (00:46:06): You could capture their strategy as, "Avoid disappointment. Don't do that. Don't live like that." In football if you wanted a quarterback that never throws any interceptions, I'm your guy. You can hire me. I'll go out there and spike it into the ground every single play and collect my money. You want a zero interception quarterback. Tom Larock (00:46:27): I'm the guy. Rob Collie (00:46:27): I am the optimal quarterback for you. I'm pretty sure that even at 47 years old, I can get the ball into the ground before the defense gets to me. Tom Larock (00:46:35): Just three QB sneaks. Rob Collie (00:46:37): No, no, no sneaks. That's dangerous. No, no. I'm not going to be... Mm-mm (Negative). I'm going to be running away from everybody and throwing it into the ground. Adam Harstad (00:46:44): It's really interesting you bring that up because there's a lot of research now that everybody used to think of sacks as a defensive line offensive line thing, but there's a huge body of research at this point that sacks really are a risk tolerance thing for a quarterback. Some quarterbacks will accept more pressure trying to get a playoff and others are more risk adverse and they'll get the ball away quicker. And so, the famous example, I use at this is, there are two quarterbacks in history who have had two seasons in the top 10 all time in soccer percentage. Adam Harstad (00:47:13): One of them is Dan Marino, very famously able to get the ball out quickly and avoid sacks. And the other one is Joey Harrington, who is very famously one of the biggest quarterback busts of all time. Basically nobody has avoided sacks as well as Joey Harrington avoided sacks. And that wasn't a good play. Tom Larock (00:47:30): Because he's not playing? Adam Harstad (00:47:31): Because he's not playing. He's unwilling to accept any risk at all. As soon as the pressure got anywhere near him, the ball is in the stance and his offense was horrible because they couldn't gain any positive gains because he was unwilling to accept that risk. And he wrote a really interesting piece after he retired about how, "My career was not a failure." Adam Harstad (00:47:50): And he talked about, "When I was in Detroit, I was asking my head coach for permission to take risks. And I just never felt like he was behind me and he was willing to let me fail. And so I was always too scared of something bad happening to really try to make something good happen." But yeah, if you look at the quarterback performances that correlate to quarterback quality, things like interception percentage are basically meaningless. Adam Harstad (00:48:11): At the very extremes, good quarterbacks tend to throw slightly fewer interceptions than bad quarterbacks, but it's not a very pronounced difference. The big difference is that good quarterbacks make a lot more positive plays. They get a lot more yards per attempt, they throw a lot more touchdowns and sometimes they'll have some negative plays mixed in there as the risk that they're taking. It's that level of risk that they're willing to tolerate to get that offense. Rob Collie (00:48:35): Let's go back to Joey Harrington and Detroit for a moment because when we were earlier talking about let's model the history of highly drafted rookie wide receivers, and I was going to make the joke, but it's not just a joke, there was some truth to it. Well, part of the decision tree needs to be well, are they being drafted in Detroit? And they did. They burned a lot of draft capital on highly touted players. And only one of them, as far as I know, really even came close. Megatron. He was the real deal. Rob Collie (00:49:05): But all those other people, how many people did they draft in the first? It was at least two or three, right? And none of them went anywhere. Adam Harstad (00:49:11): I think they took four receivers in the top 10 in five years. I think that Charles Rogers, who was one of the biggest busts of all time. Roy Williams, the wide receiver, who was okay and they traded him to Dallas and they recouped a lot of his costs. Mike Williams, who was another big bust and Calvin Johnson. Four in five years, and it became a running joke. But the thing is the fourth one was Calvin Johnson and he was probably the best pick of that era. Adam Harstad (00:49:38): There's a lot of sunk cost fallacy that we've tried this three times, we've tried drafting wide receivers and it hasn't worked out. I think a lot of people would have cut their losses and said, "I won't deal with that ridicule of picking another one." But to their credit, they stuck to their board and they said, "This guy is the best prospect on the board." And a lot of drafts success and failure, I think far more than people would believe is just randomness. Adam Harstad (00:50:02): Any team that had been on the clock at the number two pick when Charles Rogers was there would have taken Charles Rogers because he was the best wide receiver prospect, maybe one of the best ones of all time. Possibly the best- Rob Collie (00:50:13): Maybe. Adam Harstad (00:50:13): ...wide receiver prospect of all time. And everybody thought that. It wasn't a contrarian pick. That was the chalk pick and it's just Detroit happened to be the team that had that number two pick. Rob Collie (00:50:23): Yeah, but I wanted to explore the possibility that he might've succeeded elsewhere. We were just talking about Joey Harrington not having the support of his coaches. There's clearly been something culturally rotten with the Detroit franchise for a while, or at least that's an easy narrative to paint and we don't actually know, we don't get to run the experiment. The Charles Rogers experiment. We only got to run it once with one set of variables and that's in Detroit in that era. I agree with you though, on the sunk cost thing. Rob Collie (00:50:53): Being able to distinguish the difference between a sunk cost situation and a pot committed situation is one of the hardest and most valuable things to do in life. Sometimes you can cut your losses and that is the right move. It's silly to think of it that way when you're making picks, it would be really silly to think that wide receivers are terrible, we should never do that again. That would be a bad strategy. Let's drill in for a moment. Rob Collie (00:51:18): In poker, there are decision points in a game where you only have a 20% chance of winning the pot, but it still makes sense for you to continue and invest more money because the value of what's in the pot is so great, the expected value is positive. The one time you win out of five or whatever, will pay off so heavily that it doesn't matter. It pays for all the losses. Sometimes you have sunk costs. Rob Collie (00:51:42): Sometimes it's like, no, no, just because I threw $80 in there doesn't mean I should throw another 20 and distinguishing between those two circumstances, I think there's a lot of that in business, a lot of that in life is like, which situation am I in? Am I on the road to potential success and I need to put that next piece of incremental investment or effort in? Versus, no, let's not keep chasing a bad bet. There's no formula for this. There's no formula in life for being able to distinguish between the two, but the better you get at it, the more successful you're going to be. Adam Harstad (00:52:15): And that's another situation where poker, it's much easier because the possibility space is so constrained. In poker, you can just do the math and you can say, I'm pot committed or that's all sunk costs. There's a right answer in poker. And that right is knowable. The range of possible outcomes in investing a stock is it's worth nothing or it's the next Facebook. The range is so huge. There's no way to know in advance. That's another one where I like these contrived arbitrary games with their clearly defined rules, because I think they set themselves up to analysis so much more nicely. Rob Collie (00:52:49): Well, and they also set themselves up to be great metaphors for the more complicated playground of real life. I think they very purely, sometimes anyway, very purely participate in the forms that we see with infinite noise in the real world. It helps me as a learning tool, but yeah, in order for it to even exist, you have to constrain it like you do in sport. We can't let this go without mentioning the haiku. My personal favorite interaction with Adam on Twitter was when we were, I think you started it, Adam. It wasn't my idea. I participated. Rob Collie (00:53:21): I made my one contribution, but there was a whole thread years ago of football haiku, which is exactly what you expect. An analytically minded football Twitter account, we're going to start writing haikus. That's exactly what everybody would expect. Adam Harstad (00:53:38): That's why I say, the narrative guys think I'm a data guy and the data guys think I'm a narrative guy. And I always say, "I'm just a storyteller." Rob Collie (00:53:45): You're doing it right. If either extreme is painting you into the other corner, I think that's a sign that you're over the target. You're taking flack from both sides. Well, that's where you need to be. Adam Harstad (00:53:56): In general, I tend to think the things I believe are correct, because if I didn't think that I wouldn't believe them. Rob Collie (00:53:59): That's really funny. Isn't it? And that's strange. Adam Harstad (00:54:03): Yeah. It's weird how that works out. Rob Collie (00:54:05): You could almost extrapolate that to everyone. Everyone works that way- Adam Harstad (00:54:09): It's true. Rob Collie (00:54:10): ...which is really frightening, and validating at the same time, I suppose. It's all about uncertainty, navigating uncertainty. I think that's the thing that we were talking about with the NFL draft and things like that. No one really knows anything. No one knows what's about to happen. I love that at P3, now we're up to a 14-person fantasy football league and advertised it to the team as Basket of Virtual Volatile Assets, BVBA, that's what we're really calling it. That's the game we're playing. it happens to use football, but it's not really fantasy football. It's BVBA. Rob Collie (00:54:47): And most people who are playing with us have played before, but there's a few people who haven't and so, I'm putting together a primer for them. My PowerPoint deck is in progress with the animations and everything. And the very first slide in this is going to be a collection of predictions for the year, rankings, projected rankings of players and which teams are going to be good and all that. And then there's the giant word, ha-ha, over the top. It's all wrong. It's not even going to survive a week. And every year crazy things happen. Rob Collie (00:55:20): A player that we're not even talking about right now is going to be crucial to fantasy football success this year. A team, it isn't even a trendy team to be good out of the 32 teams. No one's picking to certainly improve is going to improve massively. It is such uncertainty and when I finally arrived at, in terms of my relatively successful, I did not want to play fantasy football in the league with Adam. Rob Collie (00:55:46): I'm not going to be that guy, and home leagues level of success, home and work league type of success is that it's just going in and just saying, "Look, my entire strategy is going to be based around the idea, the admission that none of us know anything." And you actually behave differently when you accept that. You make different decisions. It's not just about expectations. Rob Collie (00:56:12): You decide differently, and one of the things that I've seen that I've been doing for a long time and it's been good to me when I start listening now, like this year, I've really been listening to a lot of podcasts about fantasy football, it's the first year I've ever done this. And it's so amazing to see how widespread this notion is, that we really primarily only care about the best possible outcomes. When you're playing a zero sum game where you're one out of 12 or one out of 14, in order to win, you need exceptional things to happen in your favor. Rob Collie (00:56:42): And you need to maximize your chances of getting those exceptional things. To use a baseball metaphor, singles and doubles aren't going to win you the title. You need to be hitting triples and home runs and the nature of football, plus, I find this emotionally satisfying, it's like prospecting for gold. I'm not going to be drafting players or picking players who I know to be predictably mediocre. I'm going to be drafting and selecting for best possible range of outcomes and if they turn out to not be exceptional then I'm going to get rid of them. Rob Collie (00:57:12): This isn't genius, this isn't some new strategy. I hope that people at our company who are going to be competing in fantasy football against me aren't listening. but some of them probably are. I used to think that I might actually be able to pick players. You can't predict what's going to happen in an NFL season. It's crazy. And I love that. Adam Harstad (00:57:29): Real quick, I want to push back. You were saying you put up the list of rankings and then you were saying, "Ha-ha, they won't even last a week." Actually, empirically I found pretty consistently looking at it over the last 10 years that preseason ADP average draft position, the order that players were drafted in before the season better predicts rest of year performance than current year to date performance does until about week four. And that's when they retract parody. Adam Harstad (00:57:56): They will last a week. They'll last approximately four weeks, is one way to look at it. Or I guess another way to look at it is if you just drafted off of last year's order of finish minus guys who retired or who are clearly injured, things like that, that will be more predictive of year to date production up until about week three. My stance is everything we do in the off season, all of the analysis, all of the... Everything buys us one more week before we're wrong. If we did none of that, we could make it till week three before your due date was more powerful. Adam Harstad (00:58:28): But by doing all of that, it buys us one extra week before we're wrong. Rob Collie (00:58:33): That's amazing. Adam Harstad (00:58:34): In that sense, it does get you one extra week. Rob Collie (00:58:36): I just got snared on my own flare for hyperbole. Adam Harstad (00:58:42): I just figured you would like to put some numbers to it. Rob Collie (00:58:43): And I wouldn’t have been able to do that. Adam Harstad (00:58:45): And now you can. Rob Collie (00:58:46): Now we absolutely can. I loved that your arc so far in life goes from some sort of genius to the niche where I know nothing. There's something truly delicious about that and evolutionary. Adam Harstad (00:59:02): The fool thinks himself wise and the wise man knows he's a pool. Rob Collie (00:59:04): Exactly. And you can never pronounce yourself cured of this, of this disease because the moment you do now, you're back to the trap. You can never pronounce yourself healthy in this regard. It requires vigilance, constant vigilance and constant growth. But I like to think that I've had a similar arc, I've been on a similar arc from the know it all to know nothing and becoming comfortable with that. Did you watch Ted Lasso? Adam Harstad (00:59:31): I've seen the first season, yeah. Rob Collie (00:59:32): The speech he gives about be curious in the bar right before he kicks the guy's ass in darts? Adam Harstad (00:59:38): Yeah. Rob Collie (00:59:38): I found that to be one of my top 10 favorite moments of anything I've seen on any screen, was that sequence. Be curious. Adam Harstad (00:59:46): The cool thing about being wrong, it sucks being wrong, there's that whole ego threat where we identify ourselves with our positions and it's hard to get past that, even somebody like me, who's had so much experience being wrong. There's that ego threat that I don't want to be wrong, but the cool thing about being wrong, it's the first step to stopping, being wrong. You can't stop being wrong until you realize you're wrong in the first place. Rob Collie (01:00:09): It is a very difficult thing to come to terms with. I'm much better at it than I used to be. That's the only thing I'm comfortable saying. Again, I got to be careful. You can't be the one that says, "Yeah, I used to be like that, but I'm not like that anymore." "Yes you are." Adam Harstad (01:00:26): It's like modesty. It's impossible to declare yourself modest. That's right. I am probably the most modest person you've had on this show. If I'm being honest here. Overwhelming modesty. Rob Collie (01:00:38): If I say modesty is not my strong suit, am I being modest? Adam Harstad (01:00:42): I can self-deprecate better than anyone. I'm the most self-deprecating person you'll ever meet. Rob Collie (01:00:50): You know this from an emotional standpoint even from past years. After one week of play, you still would want to draft very differently based on what you saw. Adam Harstad (01:01:00): I think that's a trap though. Rob Collie (01:01:01): I agree. Adam Harstad (01:01:01): Yes. Emotionally people way overweight and their priors aren't weighted enough. They're reacting too much to new evidence and that's why I write that piece every year around week four and I say, "It might surprise you to know that pre-season ADP at this point still has every bit as much predictive power." Now, obviously some blend between the two, a mix of pre-season ADP plus year to date results will outperform either alone, but it's really a lesson about not discarding what we thought we knew just because somebody makes a huge, impressive catch on Monday night football. Rob Collie (01:01:32): Well, let me see if I can make a subtle, not a point, but a question. There are situations where we can talk about this is our best possible tool. It's 40% better than chance at doing something. You'd still be better off betting, if you could bet the field, if you could bet against the predictions, you'd still do better. There's a weird, subtle distinction there between best possible predictive power, which is factual, empirical and at the same time betting that it was wrong. The ha-ha still has some less than truth to it. Adam Harstad (01:02:08): Yeah. I'd say my entire niche in the industry and it's a weird niche to occupy, but I think my entire niche is just I'm the guy who knows nothing. I just aggressively embrace that uncertainty and rather than trying to know more things, I feel like I try to expound the limits of the things that I don't know. It really does change the way that you play and you manage your teams. And one example, one of my favorite things that I've done is about six years back, I looked at how players age and the popular conception of how players age is something called an age curve. Adam Harstad (01:02:44): You enter, you improve for three or four years, you hit your peak, you maintain your peak for however many years, depends on your position. And then you decline and then you're out of the league. And if you plot all those points together, it looks like a curve. And there's a lot of ways to drive this curve. If you average performance at every player age, it'll make this beautiful curve. And the age curve is an example of thinking that you know something. If you look at a player, the age curve will say, this player is going to do this this year, this next year, this next year, this the year after that. Adam Harstad (01:03:12): He's going to be out of the league at this age. It represents certainty and I never liked age curves because if you actually look at NFL careers, very, very rarely do they take on a curve shape. There's a lot of fluctuating, a lot of up and down. There's very few players who I would look at their career and say, a curve really describes what's happening here. And so, I gathered 30 years worth of data about year to year fantasy football value. And I said, "Let's try to make a mortality table out of this instead." Adam Harstad (01:03:42): If you're familiar with life insurance, in life insurance, they're basically making a bet for a life insurer that you will survive to this age. You'll survive to that age, but it's not a curve. It's not like you're 80% alive at age 64. You're still either a hundred percent alive or you're a hundred percent dead. It's a very binary state. They do tables that look at it probabilistically. And I looked at NFL player careers. And so, one thing is if careers are curve shaped, you would expect a player's last relevant season to be worse than his second to last relevant season, on average. Adam Harstad (01:04:13): But I looked at the top hundred wide receivers and running backs of the last 30 years and it was about 50, 50. 50% of them did better in their last season, 50% of them did worse than their last season before they just fell out of relevance completely. And so, I did, I modeled and I found that I think the mortality table is actually a pretty good fit for what's happening is players are basically reaching their true level of play. They ascend, they learn the position and then they reach their true level of play. Adam Harstad (01:04:39): And once they reach that level of play, they pretty much maintain that level of play. They'll have up years, they'll have down years, but they're fluctuating around that true level of play until at some point and usually without any warning whatsoever, they just fall off completely. They're great. They're great. They're great. They're done. And when you start thinking about aging like that, this wide receiver is 31, so I expect him to be a little bit worse than he was last year, and it's more, this wide receiver is 31, so I expect him probably to be about as good as last year or also maybe he will be completely useless. Rob Collie (01:05:11): Love it. Adam Harstad (01:05:12): And the way you manage your risks is very different. When an age curve would tell you that a 28-year-old wide receiver is a very safe bet. He's still on the upslope of his curve. He's going to get better or he's going to maintain, you have years before you have to worry about him disappearing. And you look at Dez Bryant who is an All-Pro wide receiver, never did anything after age 28. He was done and an age curve would never predict that. Whereas a mortality table would say, it's unlikely, but this is well within the range of possibilities. Adam Harstad (01:05:40): I played dynasty leagues where you keep your players from year to year, basically indefinitely, and it really makes you hyper aware of the risks involved. You never enter a season thinking that you're set at any position because you look and you're like, "Well, there's a 20% chance that this guy's just done. There's a 10% chance that this guy is just done." You add up all of those chances across everyone. And a lot of guys on my roster are never going to have another good season again. Adam Harstad (01:06:06): And on the flip side, sometimes you're more likely to take a chance on old players because well, everybody thinks he's in the decline. Julio Jones would be a great example this year. Everybody thinks he's going to be worse this year, but maybe he's not. Maybe he's just exactly as good as he has been for another five years. That's in the range of possibilities. It's unlikely, but it's not fundamentally unforeseeable outcome. Rob Collie (01:06:31): This tickles two ideas from daily life, whatever. One is, I have a joke, which is the average has a population of zero. When you were describing the curves, this is the aggregate average of everything. All the players that have ever lived. That curve never describes any one player. Adam Harstad (01:06:52): It's called the ecological fallacy, if you want a name for it. Rob Collie (01:06:58): Ecological fallacy. I think you'll appreciate this. From the business side for a very, very, very long time, honestly, even to this day, really, this still persists a lot of companies, they have their top number. They have their top level number for whatever metric it is, profit margin or customer retention or whatever. They know what their all up number for the entire company is, but they lack the ability to decompose it into the various segments of their business. If they've got 50 salespeople out there, what's the average win rate for a deal? Rob Collie (01:07:28): Now, let's say it's 25%. But if you go look at the 50 salespeople, none of them are at 25%. You've got some that are down at 3%, and then you've got a handful of outliers at 45% and you're not learning anything. There's no way to improve. One of our core philosophies is you need to know your top level number. You absolutely do. You need to know what your final score is, but the game is improvement. It's not knowing. Rob Collie (01:07:54): Knowing is a necessary step for improvement and you cannot improve unless you're able to effortlessly subsegment and see what's really going on to focus your attention in the right places. There's different prescriptions for different places. And so, when you're talking about this curve versus mortality risk, I was lighting up like, "Yeah." And the other thing that I've been wrestling with and really just torturing some of my friends with for a long time is this notion of single trial experiments and probabilistic models applied to single trials. Rob Collie (01:08:27): Julio Jones isn't going to have an 80% season probably. He's not 80% relevant or 80% still in the league. He's either going to have a good season probably or vanish from relevance. Probabilistic models of who wins the election. I know that's our best tool for predicting who's going to win an election, but there's something fundamentally silly about saying that someone has an 85% chance to win the election. They're not going to 85% win ever. They're going to win or they're going to lose. Rob Collie (01:08:56): And so, we need to come to terms with that, so when it's wrong, when something like that is wrong, it's more wrong than 15% wrong, in my opinion. Anyway, we don't need to pick that fight. I've done that a million times, a million different people. Adam Harstad (01:09:11): Nassim Taleb versus Nate Silver debate. Rob Collie (01:09:13): Yeah. I Think I'm on the Nassim Taleb side of that debate. Adam Harstad (01:09:16): I tend to be more with Nate Silver, where on any given prediction, you can't know if it was a good or bad prediction, but over a large enough sample, you can test your calibration. And if the things that you say will happen, 15% of the time are happening 15% of the time, I don't think it says anything about the predictions themselves, but I think it says something about you as a predictor being fairly well calibrated. Rob Collie (01:09:38): Sure, but of course, all that matters to me is its accuracy and of course that includes both the predictor and the methodology. Adam Harstad (01:09:45): Well, here's an interesting thought experiment. Let's say we have two different models. We'll go back to Ja'Marr Chase, rookie wide receiver for the Bengals, and we have two different models and these models are both extremely well calibrated. And you feed them different inputs and they will tell you what percent chance a player has of becoming a Hall of Famer. And we've tested these models, and we know, let's just say by faith that these models we're going to stipulate that they're extraordinarily well calibrated. Adam Harstad (01:10:11): And one model says that Ja'Marr Chase has a 15% chance of becoming a Hall of Fame wide receiver, and another model that he has a 40% chance of becoming a Hall of Fame wide receiver, because it's looking at different inputs and those inputs predict different chances. You know what are Ja'Marr Chase's true chances of becoming a Hall of Fame wide receiver? Is it 15%? Is it 40%? Is it somewhere between the two? If he does become, how right or wrong was each prediction? If he doesn't become, how right or wrong was each prediction? Adam Harstad (01:10:40): Two predictions, those can both be right. It seems fundamentally in conflict that one person says he has a 15% chance, one person says he has a 40% chance and both can be right at the same time, and yet it is. We're stipulating upfront that both are right. And I think when you grapple with data and especially predictive data, you need to develop a certain comfort level with that ambiguity, with we're just never going to know. We never will know. And there's no way we possibly could know, because as you say, it's a single run trial. We can't rerun it. Adam Harstad (01:11:12): But if we're aggressively testing our calibration and we're working hard and making sure that the things that we're saying are correlating with reality to some extent or another, it's almost like you're not really having faith in the predictor so much as you're having faith in the process to produce results that will have meaning and add value. Rob Collie (01:11:32): The most sophisticated version of my objection to all of this is actually, and not to what you were saying. I completely agree with the things you were saying. The most refined version of my objection to Nate Silver is that I think the domain he's applying these methods to is exactly the wrong one to give these methods a positive reputation, because here's the thing, on the eve of the election, the universe, this is can be a little weird. We're going to have to get a little weird to explain it, but the universe actually knows who's going to win. Rob Collie (01:12:03): There's no chance that it's going to swing 85, 15% tomorrow. That's not going to happen. We already know. The universe knows. Whereas in the case of Ja'Marr Chase, I can make this fuzzy argument that even the universe doesn't know if Ja'Marr Chase is going to be successful. Adam Harstad (01:12:18): I don't know if I took that argument. I have a thread on this on Twitter. That's saying basically what you’re saying. If I flip a coin and I say, "What are the odds that this coin comes up heads?" First, you want to know if the coin is biased, assume it's a fair coin. It comes up heads 50% of the time. I flip a coin. I say, "What are the odds this comes up heads?" You're going to tell me 50-50. Adam Harstad (01:12:36): If I were flipping this coin in front of some alien superintelligence who was so innately good at physics that he could divine from the starting point and the forces involved and wind resistance, he could tell with absolute certainty, which side was going to come up. He'd either say a hundred percent heads or 0% tails. And again, both of you are right. You say 50-50, you're correct. The alien says a hundred percent or 0%. If he's well calibrated, he's also correct. Fundamentally odds are not measuring anything intrinsic to the event itself. Adam Harstad (01:13:06): Odds are a measure of our ignorance surrounding the event itself. And people will get different odds for different things because they have different levels of ignorance. If I hold up a card from a deck facing me and I say, "What are the chances this card is an ace?" You're going to give me one number. If I then show you 13 cards from the deck and say, "What are the chances these cards are an ace?" You're going to give me a different number. The card itself hasn't changed. Rob Collie (01:13:29): I know. Adam Harstad (01:13:30): The odds are really meaningless to the card itself. They're just a measure of your level of knowledge and what you know. The odds are saying more about you than they're saying about the card. Rob Collie (01:13:39): Well, let me push back a little bit. Let's take your alien intelligence coin flip example. And instead, have it be a billiard ball on a frictionless surface or damn near frictionless surface, so that we could make the billiard ball bounce off of the rails 40 or 50 times. Once you get that number of collisions, maybe 50 isn't sufficient, maybe it takes a hundred. Rob Collie (01:14:01): Once you get the number of collisions high enough, even the alien intelligence won't be able to predict where it ends up at the end of a hundred bounces because even the quantum uncertainty and the vibration of the ball magnifies to great enough difference that you don't end up in the same place. Adam Harstad (01:14:15): Well, but the quantum uncertainty, that's what I'm saying. Again, the odds are a measure of our own uncertainty. Rob Collie (01:14:20): Completely. Adam Harstad (01:14:20): So if you increase the alien's uncertainty, you're going to decrease the odds [inaudible 01:14:24]. Rob Collie (01:14:24): Okay. But I'm saying, the distinction between the two. No amount of tomorrow uncertainty is going to change the outcome of the election. All that uncertainty balances itself out, large numbers. Whoever wins the election was going to win the election in a million sub universes or a billion sub... Whatever. They're going to win on all of them. Whereas Ja'Marr Chase, I think even the universe with perfect knowledge, you don't actually know. There's too many branching points in the equation of whether he turns into... Rob Collie (01:14:52): And this is why I think intuitively the masses have a point when they make fun of Nate Silver getting it wrong. Whereas they would be more accepting of the Ja'Marr Chase Hall of Fame percentage. We understand it's uncertain, but they get that there was a more wrongness about the election prediction than there is about the Ja'Marr Chase prediction. Adam Harstad (01:15:15): Yeah. I think the famous quote is, "All models are wrong, some are useful." And that's the question, any model. What you're getting at, I think is, there's something called epistemic uncertainty and something called aleatory uncertainty. I know you like learning new- Rob Collie (01:15:31): [crosstalk 01:15:31]. Oh my God. Write these down, Luke, I need to... Adam Harstad (01:15:35): Epistemic uncertainty is things that are theoretically knowable, we just don't know them. And I think you're suggesting that an election outcome, the day before that's epistemic uncertainty. This is fundamentally knowable. We just don't know it. And the odds that we're assigning are representing that epistemic uncertainty and aleatory uncertainty is irreducible uncertainty. It's even if we knew absolutely everything, this is fundamentally unknowable, it's irreducible. Adam Harstad (01:16:03): And it seems to me that you're suggesting that probabilistic forecasts in the case of aleatory uncertainty are okay. Something like Ja'Marr Chase is going to make the Hall of Fame, he won't make the Hall of Fame. Here's this chance. Whereas in the realm of epistemic uncertainty, you're saying there's just something hinky about them. There's something fundamentally weird uncomfortable. Rob Collie (01:16:23): It's not that they're not okay. It's just that they're a little less okay. Adam Harstad (01:16:27): It's fundamentally different. Rob Collie (01:16:28): An 85% number about something that is only going to be zero or a hundred, and I believe we'll be the same zero or a hundred, no matter what happens in between now and then, there's something very funky about even bothering to put a number. It's like the illusion of confidence. It's the illusion of precision. 85.6%. Not 0.5%, 0.6%, Nate? It's like what you were saying earlier like, when you're deep into analytics, you also have to maintain that other wisdom. Rob Collie (01:16:59): I just think Nate Silver in particular, this forecasting, the result of a national election, it's just the worst possible poster child for the analytics industry. We're drawn to it. We look at it like crazy, but just the worst place to have our reputations as professionals displayed. Adam Harstad (01:17:16): Yeah. It's funny. Footballguys, we obviously the people who write for Footballguys who are big into fantasy football we're pretty fluent with data and probabilities because what you're saying about your BVBA, that's basically data and probabilities. That's what fantasy football is. And we had a really interesting discussion after the 2016 election where some people were saying Nate Silver was very, very wrong because he said there was only a 36% chance. And other people were saying everybody else had the chances at 1% to 10%. Adam Harstad (01:17:45): Nate Silver was the only one out there saying there's a very real chance. 36% might not sound that much, but it's about the odds of a kicker missing a 48-yard field goal. If your team is lining up to kick a go ahead field goal from 48 yards with time expiring, how comfortable are you? That's what 36% feels like. It happens a lot, 36% of the time in fact. It's interesting that even in a place that's so comfortable dealing with risk and probability, there's still that fundamental divide and I think it really boils down to the appropriateness of putting numbers on epistemic versus aleatory uncertainty. Rob Collie (01:18:20): And I was with you for a long time that there's this over precision, but Phil Tetlock, I don't know if you're familiar with him, works on a project called The Good Judgment Project. And he wrote a book called Super Forecasting. And basically the CIA had this forecasting competition. They have a hundred questions. Here is this Middle Eastern country. What are the chances that they're going to suffer a coup in the next seven years? And Tetlock basically his approach to it was this wisdom of the crowd approach. A modified wisdom of the crowd. Rob Collie (01:18:49): Is he had a bunch of people making predictions, and then over time he saw that some people were more consistently right and so he started weighting their predictions more and more. And eventually he came up with this group of people who were definitely out predicting everybody else, usually with no specific domain expertise. And he named them super forecasters and then he's done a lot of studies about what makes these people so good at predicting basically one off events. And he found that when you look at the precision, the precision is meaningful. Rob Collie (01:19:17): If a super forecaster says there's an 83% chance that there's going to be a coup in this country versus an 80% chance, instead of just rounding to the nearest 10, he finds that that 3% is meaningful. That ignoring that 3% makes the aggregation worse, makes the outcomes less predictive. And so, I would say that for the most part, I'm skeptical of that appearance of certainty and that appearance of precision, just because most people are not super forecasters, but I have become more open to the possibility that actually no, that 0.3% might be doing some real work there. Adam Harstad (01:19:50): Yeah. I think the distinction again, when I'm not being just bombastic about it, when I'm really clear about it. My only objection here really is the reputational risk inherent. Rob Collie (01:20:03): Which is fair. Adam Harstad (01:20:04): I do want, if we're going to be making predictions, of course I want the methodology to be rigorous, if you're not artificial about it, the methodology that produces more precise numbers probably has more internal checks in the first place. It probably is the process that you use to arrive at that number was probably a superior process to the one that arrives at the round number. I'm okay with the methodology part of it. It's just that when we put it out in the public, it's the old nerds versus jerks argument all over again. Adam Harstad (01:20:34): But now we're in our middle age and the whole notion of using data to do things gets cheapened every time we put an 85% number on an election, and then it's either a much closer than expected or it goes the other way. I just wish again, for reputational reasons only, it's the rear view mirror, not the forward only allowed three predictions for an election. This person wins, that person wins, too close to call. Tom Larock (01:21:00): I just want to say, Rob, I'm not going to disagree with anything you've said, but what you're missing is the fact that 86% of Americans don't understand what the P value really means. Your talk about, this is a bad thing to put in front of people, but the real thing is, is that Nate Silver goes out and he describes things as if people understand what these concepts are and they really don't. Rob Collie (01:21:29): I'm saying two things. One is that you go to war with the army you've got right. The population that you've got. And so, they don't understand P value is great. But the other thing I'm saying is that there's something also fundamentally meaningless about his P value. It's because of this different aleatory versus pistemological, was that it? Tom Larock (01:21:46): Episcopalian, I thought. Rob Collie (01:21:48): [crosstalk 01:21:48]. Tom Larock (01:21:48): Episcopalian Rob Collie (01:21:52): Epistemic. Adam Harstad (01:21:54): If I can have one rebuttal. Rob Collie (01:21:56): Of course, we'll give you the last word. Adam Harstad (01:21:58): I'm very, very respectful of arguments that it's not enough to be right, it's are you advancing the cause? I think trying to think tactically, that's a very important skill to have, but I would also say, I don't think that there's been anybody in America who has done more for the understanding and acceptance of analytics, than Nate Silver over the last decade. I think the public is much more open to conversant in the language of analytics and I think that stems primarily to Nate Silver in FiveThirtyEight in his election forecasting. Adam Harstad (01:22:32): Maybe the 2016 election was a setback, but I think on the whole, he's been a very positive, because I see it in fantasy football too. Just the acceptance of alternate data sources. 15 years ago, you mentioned Football Outsiders and people dismiss you out of hand. Nowadays you get random people off the street who are citing DVOA and EPA per play to you. And I think Nate Silver was a big part of that revolution. Rob Collie (01:22:56): I agree with you. I think that's true. Can I not disagree, but adjust it a little bit? Let's bring back the average has a population of zero. I think he's probably reeled in some percentage of the population to the idea of analytics while pushing another percentage further away. Adam Harstad (01:23:15): Maybe. Rob Collie (01:23:15): But then in terms of real numbers, we're seeing, I totally agree with you. The analytics has benefited as a field from the exposure he's brought to it, for sure. But even if that were, and again, we're just making up numbers at this point, even if that were 20% of the world that was now more receptive as a result, that would be massive. But I see a lot of ha-ha out in the world and I just don't want the methodology that he uses to be unfairly judged by the domain in which we find it most entertaining. Rob Collie (01:23:48): And I do think that the reason why he's so famous for the politics stuff is more for the entertainment reason, which is not the right reason. I don't have any objection to the guy or his methods or anything like that. It really is just like, "Ugh, I wish we could be fascinated with something else." Adam Harstad (01:24:06): Maybe. Rob Collie (01:24:06): Adam, man, a thank you is appropriate because I've enjoyed the hell out of this. Adam Harstad (01:24:10): I did too. It was a lot of fun. Announcer (01:24:11): Thanks for listening to the Raw Data By P3 Adaptive podcast. Let the experts at P3 Adaptive help your business. Just go to p3adaptive.com. Have a data day.

We didn't know what to expect when we sat down with Greg Beaumont, Senior Business Intelligence Specialist at Microsoft specializing in serving Microsoft's Healthcare space customers' technical Power BI issues.  What we got was an insightful, delightful, and impactful conversation with a really cool and smart human! References in this Episode: The Game Azure Health Bot The Future Will Be Decentralized-Charles Hoskinson Spider Goats Episode Timeline: 3:10 - The magic of discovery with the Power Platform, It's all about the customers(and Greg has a LOT of customers!), and Greg's Data Origin Story 21:10 - The IT/Business Gap, Getting good BI and keeping data security is a tricky thing, The COVID Challenge hits Healthcare 43:00 - Power BI-Not just a data visualization tool, a very cool discussion on Genomics and using data to save lives, the importance of Data Modelling 59:10 - The Bitcoin Analogy, The VertiPaq Engine and when is Direct Query the answer 1:08:30 - We get a little personal with Greg, Azure/Power BI integration and Machine Learning, Cognitive Services and Sentiment Analysis Episode Transcript: Rob Collie (00:00:00): Hello, friends. Today's guest is Greg Beaumont from Microsoft. Like one of our previous guests, hopefully, Greg has one of those interface jobs. The place where the broader Microsoft Corporation meets its customers at a very detailed and on the ground level. On one hand, it's one of those impossible jobs. More than 100 customers in the healthcare space look to Greg as their primary point of contact for all things technical, around Power BI. That's a tall order, folks. And at the same time, it's one of those awesome jobs. It's not that dissimilar, really, from our job here at P3. Rob Collie (00:00:45): In a role that, first of all, you get broad exposure to a tremendous number of organizations and their problems, you learn a lot super, super quickly. When you're doing it right, your work day is just nonstop magic. The power platform is magic and not really because of the technology, but instead because of its impact on the people who use it, who interact with it, who benefit from it, whose lives are changed by it. And again, I can't stress this enough, software usually doesn't do this. And as we talked with him, Krissy and I just couldn't stop nodding, because we could hear it, he lives it, just like we do. And I hope that just leaps out of the audio for you like it did for us. Rob Collie (00:01:32): No surprises here, Greg didn't start his life as a data professional. He's our second guest on this show, whose original training was in biology. And so, some familiar themes come back again, that good data professionals come from a wide variety of backgrounds, that the hybrid tweeners between IT and business are really where the value is at today. And I love this about Greg, that we made a point of talking about how much easier it is today to break into the data profession than it's ever been and what an amazing thing that is to celebrate. Rob Collie (00:02:06): We talked about COVID and specifically its impacts on the industry. How that has served as a catalyst for many organizations to rethink their analytic strategy, the implications of remote work, data privacy and security. And of course, it wouldn't be an episode of Raw Data, if we didn't nerd out about at least one thing. So, we get a little bit into genomics and the idea of DNA and RNA as forms of biological computer code. And as you'd expect, and want, Greg is far from a one dimensional data professional, just such an interesting person, authentically human, a real pleasure to speak with, so let's get into it. Announcer (00:02:47): Ladies and gentlemen, could I have your attention, please. Rob Collie (00:02:51): This is the Raw Data by P3 adaptive podcast with your host, Rob Collie. Find out what the experts at P3 Adaptive can do for your business. Just go to p3adaptive.com. Raw Data by P3 Adaptive is data with the human element. Rob Collie (00:03:13): Welcome to the show, Greg Beaumont. How are you? Greg Beaumont (00:03:17): I'm doing well. How are you all? Rob Collie (00:03:19): I think we're doing pretty well. Greg Beaumont (00:03:19): Awesome. Rob Collie (00:03:20): Business is booming. Data has turned out to be relatively hot field, but I think it's probably got some legs to it. And the Microsoft platform also, well, it's just kind of kicking ass, isn't it? So, business wise, we couldn't be better. I think personally, we're doing well, too. We won't go into all that. What are you up to these days? What's your job title and what's an average day look for you? Greg Beaumont (00:03:39): So, I'm working in Microsoft and my title is Technical Specialist. And I'm a Business Intelligence Technical Specialist, so I focus almost exclusively on Power BI and where it integrates with other products within the Microsoft stack. Now, I'm in the Microsoft field, which is different from a number of guests you've had, who work at corporate and we're working on the product groups, which is that I'm there to help the customers. Greg Beaumont (00:04:01): And you hear a lot of different acronyms with these titles. So, my role is often called the TS. In the past, it was called a TSP. It's just a change in the title. Sometimes you might hear the title, CSA, Cloud Solution Architect. It's very similar to what I do, but a little bit different. But effectively from an overarching standpoint, our goal in the field as Technical Specialists is to engage with customers, so that they understand how and where to use our products, and to ensure that they have a good experience when they succeed. Rob Collie (00:04:29): Your job is literally where the Microsoft organism meets the customers. Greg Beaumont (00:04:34): Yep. Rob Collie (00:04:35): That's not the role I had. I was definitely on the corporate side, back in my days at Microsoft. I think the interaction between the field and corporate has gotten a lot stronger over the years. I think it's a bit more organic, that interplay, that it used to feel like crossing a chasm sort of thing. And I don't think that's really true anymore. Greg Beaumont (00:04:54): At a green, I think that's by design, too. So, with the more frequent release schedules and also kind of how things have changed under Satya, customer feedback drives the roadmap. So when these monthly updates come out, a lot of it is based off of customer demand and what customers are encountering and what they need. So, we're able to pivot and meet the needs of those customers much more quickly. Rob Collie (00:05:15): Yeah, you mentioned the changing acronyms, right? I mean like yes. My gosh, a thousand times yes. It's almost like a deliberate obfuscation strategy. It's like who's what? Why did we need to take the P off of TSP? I mean, I'm sure it was really important in some meeting somewhere, but it's just like, "Oh, yeah, it's really hard to keep track of." It's just a perpetually moving target. But at the same time, so many fundamentals don't change, right? The things that customers need and the things that Microsoft needs to provide. The fundamentals, of course, evolving, but they don't move nearly as fast as the acronym game. Greg Beaumont (00:05:52): Right. I think that acronym game is part of what makes it difficult your first year here, because people have a conversation and you don't know what they're talking about. Right? Rob Collie (00:06:00): Yeah, yeah, yeah. Greg Beaumont (00:06:00): And if they just spelled it out, it would make a lot more sense. Rob Collie (00:06:03): Krissy was talking to me today about, "Am I understanding what Foo means?" There's an internal Microsoft dialect, right? Krissy was like, "Is Foo like X? Is it like a placeholder for variable?" I'm like, "Yes, yes." She's like, "Okay. That's what I thought, but I just want to make sure." Krissy Dyess (00:06:18): That's why there's context clues in grade school really come into play when you're working with Microsoft organization, because you really got to take in all the information and kind of decipher it a bit. And those context clues help out. Greg, how long have you been in that particular role? Has it been your whole time at Microsoft or are have you been in different roles? Greg Beaumont (00:06:36): So, I should add, too, that I'm specifically in the healthcare org, and even within healthcare, we've now subspecialized into sub-verticals within healthcare. So, I work exclusively with healthcare providers, so people who are providing care to patients in a patient care setting. I do help out on a few other accounts, too, but that's my primary area of responsibility. Greg Beaumont (00:06:55): So, I started with Microsoft in 2016. I was actually hired into a regional office as what's called the traditional TSP role and it was data platform TSP. So, it was what used to be the SQL Server TS role. A few months later, the annual realign happened, I got moved over to Modern Workplace because they wanted to have an increased focus on Power BI, and I had some experience in that area. Plus, I was the new guy, so they put me into the experimental role. A year later, that's when they added the industry verticals and that's when I moved into what is kind of the final iteration of my current role. And the titles have changed a few times, but I've effectively been in this role working with healthcare customers for over four years now. Rob Collie (00:07:35): And so, like a double vertical specialization? Greg Beaumont (00:07:37): Yeah. Rob Collie (00:07:37): Healthcare providers, where there's a hierarchy here? Greg Beaumont (00:07:40): Yeah, yeah. Rob Collie (00:07:41): Those are the jaw dropping things for me is sometimes people in roles like yours, even after all that specialization, you end up with a jillion customers that you're theoretically responsible for. Double digits, triple digits, single digits in terms of how many customers you have to cover? Greg Beaumont (00:07:58): I'm triple digits. And that is one of the key differences from that CSA role that you'll see on the Azure team is they tend to be more focused on just a couple of customers and they get more engaged in kind of projects. And I will do that with customers, but it's just, it's a lot more to manage. Rob Collie (00:08:14): Yeah. What a challenging job. If you think about it, the minimum triple digit number is 100, right? So, let's just say, it's 100 for a moment. Well, you've got 52 weeks a year plus PTO, right? So, you're just like, "Okay." It is very, very difficult to juggle. That's a professional skill that is uncommon. I would say that's probably harder than the acronym game. Greg Beaumont (00:08:37): Yeah, there's been times I was on a vacation day and I got a call. I didn't recognize the number. I'm like, "Okay, I'm going to have to route this to somebody because I'm off today." And they're like, "Well, I'm the VP of so and so and we need to do this." And I'm like, "Okay, I got to go back inside and work now, because this is an important call." So, you have to be flexible and you're correct, that it makes it a challenge to have that work-life balance also, but the work is very rewarding, so it's worth it. Rob Collie (00:09:01): Yeah. It's something that vaguely I have a sense of this. I mean, transitioning from corporate Microsoft to, I mean, you can think of my role now as field. I'm much, much closer to the customers than I ever was at corporate. And yes, Brian Jones and I talked about it a little bit. And this is a bit of an artifact of the old release model that it was like every few years, you'd release a product, which isn't the case anymore. But that satisfying feeling of helping people, like even if you build something amazing back at Microsoft in the days that I was there, you were never really around for that victory lap. You would never get that feedback. It even never make it to you. Rob Collie (00:09:37): It was years later muted whereas one of the beautiful things about working closely with customers and our clients with Power BI, and actually the Microsoft platform as a whole, is just how quickly you can deliver these amazingly transformational like light up moments that go beyond just the professional. You can get this emotional, really strong validating emotional feeling of having helped. And that is difficult to get, I think even today, probably, even with their monthly release cycles, et cetera. By definition, you're just further removed from the "Wow" that happens out where the people are. Greg Beaumont (00:10:15): Yep. And I'm sure you all see that, too, with your business is that a lot of work often goes into figuring out what needs to be in these solutions and reports, but when you actually put it in the hands of leaders, and they realize the power of what it can provide for their business, in my case for their patients, for their doctors, for their nurses, it becomes real. They see it's actually possible and it's not just a PowerPoint deck. Rob Collie (00:10:38): And that sense of possibility, that sense of almost child-like wonder that comes back at those moments, you just wouldn't expect from the outside. I had a family member one time say, "Oh, Rob, I could never do what you do." Basically, it was just saying "How boring it must be, right?" It's so boring working with software, working with..." I'm like, "Are you kidding me? This is one of the places in life where you get to create and just an amazingly magical." It's really the only word that comes close to capturing it. You just wouldn't expect that, right? Again, from the outside like, "Oh, you work in data all day. Boring." Greg Beaumont (00:11:17): I'd add to that, that I'd compare it to maybe the satisfaction people get out of when they beat a game or a video game. That when you figure out how to do a solution and it works and you put in that time and that effort and that thought, there's that emotional reward, you get that I built something that that actually did what they wanted it to do. Rob Collie (00:11:35): Yeah. And after you beat the video game, not only did that happen, but other people's lives get better as a result of you beating this game. It's just like it's got all those dynamics, and then some. All these follow on effects. Greg Beaumont (00:11:46): It's like being an athlete and enjoying the sport that you compete in. Rob Collie (00:11:50): Yeah. We're never going to retire. We're going to be the athletes that hang on way too long. Greg Beaumont (00:11:56): Yep. Rob Collie (00:11:58): So, unfortunately, I think our careers can go longer than a professional athletes, so there's that. I can't even really walk up and down stairs anymore without pain, so. So what about before Microsoft? What were you up to beforehand and how did you end up in this line of work in the first place? Greg Beaumont (00:12:15): Sure. And I think that's actually something where listeners can get some value, because the way I got into this line of work, I think today, there's much more opportunity for people all over the world from different socioeconomic backgrounds to be able to break into this field without having to kind of go through the rites of passage that people used to. So, I was actually a Biology major from a small school. Came from a military family. I didn't have corporate contacts or great guidance counseling or anything like that. My first job right out of school was I said, "Oh, I got a Biology major. I got a job at a research institution." They're like, "Okay, you're going to be cleaning out the mouse cages." And it was sort of $10.50 an hour. Greg Beaumont (00:12:53): So, at that point, I said, "Okay, I got to start thinking about a different line of work here." So, I kind of bounced around a little bit. I wanted to get into IT, but if you wanted to learn something like SQL Server, you couldn't do it unless you had a job in IT. As an average person, you couldn't just go buy a SQL Server and put it in your home unless you had the amount of money that you needed to do that. Side projects with Access and Excel. Small businesses did things probably making less than minimum wage and side gigs, in addition to what I was doing for full-time work to pay the bills. Eventually caught on with a hospital where I was doing some interesting projects with data using Access and Excel. They wouldn't even give me access to Crystal Reports when we wanted to do some reporting. That was really where I kind of said , "Data is where I want to focus." Greg Beaumont (00:13:41): We did some projects around things like Radon Awareness, so people who would build a new house now, they're like, "Oh, I have to pay $1500 for that Radon machine down in the basement." But when you talk to a thoracic surgeon and their nursing team and you hear stories about people who are nonsmokers, perfectly healthy, who come in with tumors all over their lungs, you realize the value there and by looking at the data of where there's pockets of radon in the country reaching out to those people has value, right? I think it's that human element where you're actually doing something that makes a difference. So, that kind of opened my eyes. Greg Beaumont (00:14:14): I then after that job, I got on with a small consulting company. I was a Project Manager. It was my first exposure to Microsoft BI. It was actually ProClarity over SQL Server 2005 and we were working with data around HEDIS and Joint Commission healthcare performance measures for one of the VA offices. So, I was the PM and the Data Architect was building the SSIS packages, built out kind of skeleton of an analysis services cube. He asked me to lean in on the dashboarding side, and that's also where I started learning MDX because we were writing some MDX expressions to start doing some calculations that we were then exposing in ProClarity. And at that point, it was like, "This is magic." Greg Beaumont (00:14:57): From a used case perspective, what they were doing traditionally doing was they'd send somebody in from some auditing agency, who would look at, I think it was 30 to 60 patient records, for each metric and then they take a look at where all of the criteria hit for that metric, yes or no. And it would be pass/fail, how good is this institution doing of meeting this particular expectation. So, it would be things like, "Does a patient receive aspirin within a certain amount of time that they've been admitted if they have heart problems?" Something like that. With looking at it from a data perspective, you can look at the whole patient population, and then you could start slicing and dicing it by department, by time of day that they were admitted, by all of these different things. Greg Beaumont (00:15:38): And that's when I kind of said, "This is really cool, really interesting. I think there's a big future here." And I kind of decided to take that route. And from there, I got on with a Microsoft partner, where I stayed for about six years. And that's kind of where I was exposed to a lot of very smart, very gifted people. And I was able to kind of learn from them and then that led to eventually getting a job at Microsoft. But to make a long story short, today, you could go online and get Power BI Desktop for free. There's training resources all over the place, and you could skill up and get started and get a great job. I'd like to tell people take the amount of time you spend every night playing video games and watching television, take half that time and devote it to learning Power BI and you'll be amazed at how far you get in six to 12 months. Rob Collie (00:16:24): That's such good advice. I'm not really allowed to play a lot of video games, so I might need more time than that. But I had my time to do that years ago, learning DAX and everything. A couple of things really jumped out at me there. First of all, you're right, it was almost like a priesthood before. It was so hard to get your foot in the door. Look, you had to climb incrementally, multiple steps in that story to just get to the point where you were sitting next to the thing that was SSIS and MDX which, again, neither of those things had a particularly humane learning curve. Even when you got there, which was a climb, you get to that point and then they're like, "And here's your cliff. Your smooth cliff that you have to scale. If you wanted a piece of this technology," right? Rob Collie (00:17:11): You wanted to learn MDX, you had to get your hands on an SSAS server. The license for it. And then you had to have a machine you could install it on that was beefy enough to handle it. It's just, there's so many barriers to entry. And the data gene, I like to talk about, it does. It cuts across every demographic, as far as I can tell, damn near equally everywhere. Let's call it one in 20. It's probably a little less frequent than that. Let's call it 5% of the population is carrying the data gene and you've got to get exposure. And that's a lot easier to get that exposure today than it was even 10 years ago. Greg Beaumont (00:17:50): I'd completely agree with that. The people in this field tend to be the type of people who likes solving puzzles, who like building things that are complex and have different pieces, but who also enjoy the reward of getting it to work at the end. You've had several guests that have come on the show that come from nontraditional backgrounds. But I'm convinced that 20 years ago, there were a lot of people who would have been great data people, who just never got the opportunity to make it happen. Greg Beaumont (00:18:14): Whereas today, the opportunity is there and I think Microsoft has done a great job with their strategy of letting you learn and try Power BI. You can go download the dashboard in a day content for free and the PDF is pretty self-explanatory and if you've used excel in the past, you can walk through it and teach yourself the tool. I think the power of that from both the perspective of giving people opportunity and also building up a workforce for this field of work is amazing. Rob Collie (00:18:42): Yeah. I mean, all those people that were sort of in a sense like kind of left behind, years ago, they weren't given an avenue. A large number of them did get soaked up by Excel. If they're professionally still active today, there's this tremendous population of Excel people if they were joining the story today, they might be jumping into Power BI almost from the beginning, potentially. And of course, if they were doing that, they'd still be doing Excel. But there's still this huge reservoir of people who are still tomorrow, think about the number of people tomorrow, just tomorrow. Today, they're good at Excel and tomorrow, they will sort of, they'll have their first discovery moment with Power BI. The first moment of DAX or M or whatever, that's a large number of people tomorrow who are about to experience. It's almost like did you see the movie The Game? Greg Beaumont (00:19:36): I have not. Rob Collie (00:19:37): There's this moment early in the movie where Michael Douglas has just found out that his brother or something has bought them a pass to the game. And no one will tell him what it is. He meets this guy at a bar who says, "Oh, I'm so envious that you get to play for the first time." Also, this is really silly, but it's also like the ACDC song For Those About To Rock, We Salute You. For those about to DAX, we salute you, because that's going to happen tomorrow, right? Such a population every day that's lighting up and what an exciting thing to think about. Do you ever get down for any reason, just stop and think, "Oh, what about the 5000 people today who are discovering this stuff for the first time." That is a happy thing. Greg Beaumont (00:20:16): Yeah, I actually had a customer where one of their analysts who turned out to be just a Power BI Rockstar, he said, "I'd been spending 20 years of my life writing V-lookups, and creating giant Excel files. And now, everything I was trying to do is at my fingertips," right? And then within a year, he went from being a lifelong Excel expert to creating these amazing reports that got visibility within the organization and provided a ton of value. Rob Collie (00:20:42): And that same person you're talking about is also incredibly steeped in business decision-making. They've been getting a business training their whole career at the same time. And it's like suddenly, you have this amazingly capable business tech hybrid, that literally, it just like moved mountains. It's crazy. We've talked about that a lot on the show, obviously, the hybrids, just amazing. And a lot of these people have come to work for us. Rob Collie (00:21:09): That's the most common origin story for our consultants. It's not the only one. I mean, we do have some people who came from more traditional IT backgrounds, but they're also hybrids. They understand business incredibly well. And so, they never really quite fit in on the pure IT side, either. It's really kind of interesting. Greg Beaumont (00:21:26): Yeah, I think there's still a gap there between IT and business, even in kind of the way solutions get architected in the field. It's understanding what the business really wants out of the tool is often very different from how IT understands to build it. And I think that's where people like that provide that bridge, to make things that actually work and then provide the value that's needed. Rob Collie (00:21:47): There's such valuable ambassadors. It's just so obvious when IT is going to interact with a business unit to help them achieve some goal. It's so obvious that, of course, who you need to engage with IT. IT thinks, "We need to engage with the leaders of this business unit." They've got the secret weapon, these hybrid people that came up through the ranks with Excel. The word shadow IT is perfect. These people within the business, like they've been Excel people for their entire careers, they have an IT style job. Rob Collie (00:22:22): Almost all the challenges that IT complains about with working with business, you take these Excel people and sort of put them in a room where they feel safe. They'll tell you the same things. They're like, "I had exactly the same problems with my 'users,' the people that I build things for." And yeah, there's such a good translator. And if the communication flows between IT and business sort of through that portal, things go so much better. That's a habit. We're still in the process of developing as a world. Greg Beaumont (00:22:51): Yeah. And in healthcare that actually also provides some unique challenges. With regulation and personal health information, these Excel files have sensitive data in them, and you have to make sure it's protected and that the right people can see it. And how do you give them the power to use their skills to improve your organization, while also making sure that you keep everything safe. So, I think that's a hot topic these days. Rob Collie (00:23:15): Yeah. I mean, it's one of those like a requirement, even of the Hello World equivalent of anything is that you right off the bat have to have things like row level security and object level security in place and sometimes obfuscation. What are some of the... we don't want to get to shop talky, but it is a really fascinating topic, what are the handful of go-to techniques for managing sensitive healthcare information? How do you get good BI, while at the same time protecting identity and sensitivity. So often, you still need to be able to uniquely identify patients to tie them across different systems, can identify them as people. It's really, really, really tricky stuff. Greg Beaumont (00:24:02): And I think just to kind of stress the importance of this, you can actually go search for look up HIPAA wall of shame or HIPAA violation list. When this information gets shared with the wrong people, there's consequences and can result in financial fees and fines. And in addition to that, you lose the trust of people whose personal information may have been violated. So, I think a combination of you said things row level security and object level security as a start, you can also do data masking, but then there's issues of people export to Excel. What do they do with that data afterwards? Greg Beaumont (00:24:37): And then there's going to be tools like Microsoft Information Protection, where when you export sensitive information to Excel, it attaches an encrypted component. I'm not an MIT expert. I know how it works. I don't know the actual technology behind it. But it attaches an encrypted component where only people who are allowed to see that information can then open that file. So, you're protecting the information at the source and in transit, but you're still giving people the flexibility to go build a report or to potentially use data from different sources, but then have it be protected every step of the way. Greg Beaumont (00:25:11): So like you said, without getting too techie, there's ways to do it, but it's not just out of the box easy. There's steps you have to go through, talk to experts, get advice. Whether it's workshops or proof of concepts, there's different ways that customers can figure that out. Rob Collie (00:25:28): Yeah. So because of that sort of mandatory minimum level of sensitivity handling and information security, I would expect, now that we're talking about it, that IT sort of has to be a lot more involved by default in the healthcare space with the solutions than IT would necessarily be in other industries. Another way to say it, it's harder for the business to be 100% in charge of data modeling in healthcare than it is in other industries. Greg Beaumont (00:26:02): Yep. But you can have a hybrid model, which is where the business provides data that's already been vetted and protected and there might be other data that doesn't have any sensitive data in it, where it's game on, supply chain or something like that. But having these layers in between, the old way of doing things was just nobody gets access to it. Then there was kind of canned reporting where everybody gets burst in the reports that contain what they're allowed to see. But now, you can do things in transit, so that the end users can still use filters and build a new report and maybe even share it with other people. And know that whoever they're sharing with will only be able to see what they're allowed to see. It gets pretty complex, but it's definitely doable and the customers that are doing it are finding a lot of value in those capabilities. Rob Collie (00:26:48): That's fundamentally one of the advantages of having a data model. I was listening to a podcast with Jeffrey Wang from Microsoft and he was talking about it. And I thought this was a really crisp and concise summary, which is that the Microsoft Stack Power BI has a model-centric approach to the world whereas basically, all the competitors are report centric. And what does that mean? Why does that even make a difference? Well, when you build a model, you've essentially built all the reports in a way. You've enabled all of the reports. You can build many, many, many, many, many like an infinite number of different reports based on emerging and evolving business needs without having to go back to square one. Rob Collie (00:27:28): In a report-centric model, which is basically what the industry has almost always had, almost everywhere, outside of a few notable examples, Power BI being one of them. When a report centric model, every single change, I remember there being a statistic that was just jaw dropping. I forget what the actual numbers were, but it was something like the average number of business days it took to add a single column to a single existing report. It was like nine business days, when it should just be a click. And that's the difference. And so, preserving that benefit of this model centric approach, while at the same time, still making sure that everyone's playing within the right sandbox that you can't jump the fence and end up with something that's inappropriate. Very challenging, but doable. Greg Beaumont (00:28:15): Yep. That reminded me of an old joke we used to tell in consulting and this was back in the SharePoint Performance Point with Analysis Services days is there be a budget for a project, there'd be change requests along the ways, they discover issues with the data. And at the very end of the project, they rushed the visualization to market. And they're like after six months, with 10 people dedicated on this project, "Here's your line chart." Rob Collie (00:28:39): Yeah. I had a director of IT at a large insurance company one time, looking me in the eye and just brutally confess. Yeah, my team, we spent three months to put a dot on a chart. And that's not what you want. Greg Beaumont (00:28:59): Right, right. Rob Collie (00:29:01): That was unspoken. This was bad. To the extent that you're able to tell, what are some of the interesting things that you've seen in the healthcare space with this platform recently? Anything that we can talk about? Greg Beaumont (00:29:15): Yeah, so I think I'd start with how everything changed with COVID. Just because I think people would be interested in that topic and kind of how it changed everything. I actually had a customer yesterday at a large provider who said, "COVID was the catalyst for us to reconsider our investment in analytics, and that it spurred interest from even an executive level to put more money into analytics because of the things that happened." So obviously, when it hit everybody was, "What in the world is going on here?" Right? "Are we even going to have jobs? Is the whole world going to collapse or is this just going to be kind of fake news that comes and goes?" Everybody was unsure what was going on. Greg Beaumont (00:29:50): At the same time, the healthcare providers, a lot of them were moving people to work from home and these were organizations where they had very strict working conditions because of these data privacy and data security considerations, and all of a sudden, you're in a rush to move people home. So, some of my counterparts who do teams, they have some just amazing stories. They were up all night helping people set up ways to securely get their employees to a work-from-home type experience, so that they only had essential workers interacting with the patients, but then the office workers were able to effectively conduct business from home. Greg Beaumont (00:30:25): Additionally, there were use cases that were amazing. So, Microsoft has now what's called the Cloud for Health where we're effectively taking our technology and trying to make it more targeted towards healthcare customers and their specific needs, because we see the same types of use cases repeat from customer to customer. One of those use cases that came out of COVID was called Virtual Visits. And I actually know the team that built that solution, but because of patients who were on COVID, they didn't know how contagious it was. Greg Beaumont (00:30:56): There were people being put on ventilators, who weren't allowed to see their families and they were setting up a team's application, where people were actually able to talk to their family and see their family before they went under, right? There were chaplains who were reading people their last rites using video conferencing, and things like that. So, it was pretty heavy stuff, but I think from a healthcare perspective, it showed the value technology can provide. Greg Beaumont (00:31:21): And from our perspective in the field, it's like we're not just out there talking about bits and bytes. It kind of hit home that there's real people who are impacted by what we're doing and it adds another kind of layer of gravity, I'd call it, taking what you do seriously, right? I had another customer, they were doing some mapping initiatives with some of the COVID data because they wanted to provide maps for their employees of where the hotspots were. Greg Beaumont (00:31:46): And we were up till I think 11:00 at night one night working through a proof of concept. And they said, "Yeah, what's next is we also want to start mapping areas of social unrest." I said, "Wow, social unrest. Why are you worried about that?" And they said, "Well, we expect because of this lockdown, that eventually there's going to be rioting and issues in all different parts of the world." And at that time, I just kind of didn't really think about that, but then a lot of those things did happen. It was kind of just interesting to be working at night and hearing those stories, and then seeing how everything kind of unfolded. Greg Beaumont (00:32:18): Another example, look it up, there's an Azure COVID Health Bot out there and then there's some information on that, where you can ask questions and walk through your symptoms, and it will kind of give you some instructions on what to do. Another one that is even popular now is looking at employees who are returning to work. So, when people return to work find out vaccination status, "Are you able to come back to work? Are you essential? Are you nonessential?" I don't think a lot of customers were prepared to run through that scenario when it hit. Greg Beaumont (00:32:48): So, having these agile tools where you can go get your list of not only employees, but maybe partners that refer people to your network, because you might not have all the referring doctors in your system. So with Power BI, you can go get extracts, tie it all together and then build out a solution that helps you get those things done. I'd say it was eye opening. I think for customers and also for myself and my peers, that we're not just selling widgets. We're selling things that make a difference and have that human perspective to it. Rob Collie (00:33:20): Yeah, that does bring it home, doesn't it? That statement from an organization that COVID was the catalyst, evaluating and investing in their analytic strategy? Greg Beaumont (00:33:29): Yep. Rob Collie (00:33:30): Being in BI, being an analytics is one of the best ways to future proof one's career because at baseline, it's a healthy industry, there's always value to be created. But then when things get bad, for some reason, whatever crisis hits, it's actually more necessary than ever because when you've been in a groove when a an industry or an organization has been in an operational groove for a long time, any number of years, eventually, you just sort of start to intuitively figure it out. There's a roadmap that emerges slowly over time. Now, even that roadmap probably isn't as good as you think it is. If you really tested your assumptions, you'd find that some of them were flawed and analytics could have helped you be a lot more efficient even then. Rob Collie (00:34:14): But regardless, the perception is that we've got a groove, right? And then when the world completely changes overnight, all of your roadmaps, your travel roadmaps, none of them are valid anymore. And now, you need a replacement and you need it fast. And so, what happens is, is that analytics spending, BI spending, whatever you want to call it, or activity, actually increases during times of crisis. So, you got a healthy baseline business. It's an industry that's not withering and dying in good times, but it actually it's like a hedge against bad times. Rob Collie (00:34:47): When I saw that research years and years ago, when I was working at Microsoft Corporate, we just come out of the dot-com crack up, we'd seen that BI spending it across the IT industry was the only sector that went up during that time where everything else was falling. It's like, "Oh, okay." So, not only do I enjoy this stuff, but I really should never get out of it. It's like one of the best future proofing career moves you can make is the work in this field. And so, I mean, we've seen it, right? The early days of the COVID crisis, you're right when no one knew the range of possible outcomes going forward was incredibly wide. The low end and the high end were exponentially different from one another. Rob Collie (00:35:29): And so, we experienced in our business, sort of a gap in spring and early summer last year. We weren't really seeing a whole lot of new clients, people who are willing to forge a brand new relationship. Again, what happens when a crisis hits? You slam on the brakes. No unnecessary spending first of all. Let's get all the spending under control, because we don't know as a company what's going to happen in the industry, right? You see a lot of vendor spending freezes and of course, to other companies, we're a vendor, right? So, our existing clients, though, doubled down on how much they used us and how much they needed us. Rob Collie (00:36:08): And then later in the year, the new client business returned, and we actually ended up, our business was up last year, despite that Q2 interruption and sort of making new friends. And this year, holy cow like whatever was bottled up last year is coming back big time. And so, yeah. You never really want to be the ghoul that sort of morbidly goes, "Oh, crisis." From a business perspective, yeah, anything that changes, anything that disrupts the status quo tends to lead to an increased focus on the things that we do. Greg Beaumont (00:36:43): Yeah, I think something you said there, too, was when you don't know what's going to happen was when the business intelligence spending increased. I mean, the intelligence and business intelligence, it's not just a slogan. The purpose of these tools is to find out the things you don't know. So when there's uncertainty, that's when BI can provide that catalyst to sort of add some clarity to what you're actually dealing with. Rob Collie (00:37:06): Yeah, I've been using, even though I'm not a pilot, I've never learned to fly a plane or anything. I've been using an aviation metaphor lately, which is windshield is nice and clear. You might not be looking at the instruments on your cockpit very much, right? You know there's not a mountain in front of you, you can see how far away the ground is. And you could sort of intuit your way along, right? But then suddenly, whoosh clouds. And oh, boy, now, you really need those instruments, right? You need the dashboards, you need the altimeter, you need the radar. You need all that stuff so much more. Rob Collie (00:37:37): And so, and our business has kind of always been this. The reason I've been using this metaphor is really for us, it's like given how fast we operate, and I think you can appreciate this having come from a Microsoft partner consulting firm before Microsoft years ago, our business model, we move so fast with projects. We're not on that old model with the original budget and the change orders and all of that. That was all dysfunctional. Rob Collie (00:38:01): It was necessary, because of the way software worked back then, but it was absolutely dysfunctional. It's not the way that you get customer satisfaction. So, we've committed to the high velocity model. But that means seeing the future of our business financially two months in the future is very difficult relative to the old sort of glacial pace, right? If there's a mountain there, we're going to have months to turn around it. Krissy Dyess (00:38:26): To add a bit to your analogy there, Rob. I am married to a pilot and I have gone up in the small tiny airplane. And before the gadgets, there's actually the map. The paper map, right? So, you had the paper map, which my husband now would hand to me. And he'd tell me, "Okay, let me know the elevations of different areas to make sure we're high enough, we're not going to crash into the mountains." Krissy Dyess (00:38:47): What's happened is people just they got used to different ways that they were doing things. They were forced into these more modern ways. And I think even now, this wave of seeing this catalyst we can change and how are other people changing is also driving the people to seek help from others in terms of getting guidance, right? Because even though you've had the change, it doesn't necessarily mean that the changes that you made were 100% the right way and you can learn so much from others in the community and the people that are willing to help. Krissy Dyess (00:39:24): And I think that's one of the things too, that our company provides as a partner, we're able to kind of go alongside. We've seen what's works, what doesn't work, what are some of those pitfalls? What are those mountains approaching? And we're really able to help guide others that want to learn and become better. Rob Collie (00:39:42): Yeah. I mean, this is us getting just a little bit commercial, but you can forgive us, right? That high velocity model also exposes us to a much larger denominator. We see a lot at this business that accumulates. The example I've given before is and this is just a really specific techy, so much of this is qualitative, but there's a quantitative. It's sort of like a hard example of like, "Oh, yeah, that's right. This pattern that we need here for this food spoilage inventory problem is exactly the same as this tax accounting problem we solved over there, right?" As soon as you realize that you don't need to do all the figuring out development work, you just skip to the end. Rob Collie (00:40:22): And really, most of the stuff that Krissy was talking about, I think, is actually it's more of the softer stuff. It's more of the soft wisdom that accumulates over the course of exposure to so many different industries and so many different projects. That's actually really one of the reasons why people come to work here is they want that enrichment. Greg Beaumont (00:40:38): Yeah, that makes sense. Because you see all these different industries and you actually get exposed to customers that are the best in the business for that type of, whether it be a solution or whether it be a product or whether it be like a framework for doing analytics or something like that. So, you get that exposure and you also get to contribute. Rob Collie (00:40:55): Even just speaking for myself, in the early days of this business, when it was really still just me, I got exposure to so many business leaders. Business and IT leaders that, especially given the profile of the people who would take the risk back in 2013, you had to be some kind of exceptional to be leaning into this technology with your own personal and professional reputation eight years ago, right? It was brand new. So, imagine the profile of the people I was getting exposed to, right? Wow, I learned so much from those people in terms of leadership, in terms of business. They were learning data stuff from me, but at the same time, I was taking notes. Greg Beaumont (00:41:33): Everybody was reading your blog, too. I can't count the number of times I included a reference to one of your articles to help answer some questions. And it was the first time I was introduced to the Switch True DAX statement. And then I'd print that. Rob Collie (00:41:47): Which- Greg Beaumont (00:41:48): Sent that link to many people. "Don't do if statements, do this. Just read this article." Rob Collie (00:41:53): And even that was something that I'd saw someone else doing. And I was like, "Oh, my God, what is that?" My head exploded like, "Oh." Yeah, those were interesting days. I think on the Chandu podcast, I talked about how I was writing about this stuff almost violently, couldn't help it. It was just like so fast. Two articles a week. I was doing two a week for years. There was so much to talk about, so many new discoveries. It was just kind of pouring out in a way. Krissy Dyess (00:42:24): Greg, you came in to the role around 2016. And to me 2017 was really that big year with the monthly releases where Power BI just became this phenomenon, right? It just kept getting better and better in terms of capabilities and even the last couple years, all the attention around security has been huge, especially with the health and life science space. And last year, with this catalyst to shift mindsets into other patterns, working patterns using technology, do you feel like you've seen any kind of significant shifts just compared to last year or this year? Greg Beaumont (00:43:05): Yeah. And so something that burns my ears every time I hear it is when people call Power BI a data visualization tool. It does that and it does a great job. Rob Collie (00:43:11): I hate that. Greg Beaumont (00:43:12): But it's become much more than that. When it launched, it was a data visualization tool. But if you think about it at that time, they said, "Well, business users can't understand complex data models, so you have to do that in analysis services." Then they kind of ingested analysis services into Power BI and made it more of a SaaS product where you can scale it. There's Dataflows, the ETL tool, which is within Power BI, which is an iteration of Power Query, which has been around since the Excel days. So, now you have ETL. You have effectively from the old SQL Server world, you have the SSIS layer, you have the SSAS layer. With paginated reports, you have the SSRS layer. And you have all these different layers of the solution now within an easy to use SaaS product. Greg Beaumont (00:43:55): So this evolution has been happening, where it's gobbling up these other products that used to be something that only central IT could do. And now, we're putting that power by making it easier to use in the hands of those analysts who really know what they want from the data. Because if you think about it, the old process was is you go and you give the IT team your requirements, and they interpret how to take what you want, and translate it into computer code. Greg Beaumont (00:44:21): But now, we're giving those analysts the ability to take their requirements and go do it themselves. And there's still a very valid place for central IT because there's so many other things they can do, but it frees up their time to work on higher valued projects and I see that continuing with Power BI, right? But like we're adding AI, ML capabilities and data volumes keep increasing then capabilities I think will continue to expand it. Rob Collie (00:44:46): Greg, I used to really caused a storm when I would go to a conference that was full of BI professionals. And I would say that something like, "What percentage of the time of BI project, traditional BI project was actually spent typing the right code?" The code that stuck, right? And I would make the claim that it was less than 1%. So, it's like less than 1% of the time of a project, right? And everyone would just get so upset at me, right? But I just didn't understand why it was controversial. Rob Collie (00:45:19): Like you describe like yeah, we have these long requirements meetings in the old model. Interminably long, exhausting, and we'd write everything down. We'd come up with this gigantic requirements document that was flawed from the get-go. It was just so painful. It's like the communication cost was everything and the iteration and discovery, there wasn't enough time for that. And when I say that the new way of building these projects is sometimes literally 100 times faster than the old way. Like it sounds like hyperbole. Greg Beaumont (00:45:53): It's not. Yeah. Rob Collie (00:45:54): It can be that fast, but you're better off telling people, it's twice as fast because they'll believe you. If you tell them the truth, they'd go, "Nah, you're a snake oil salesman. Get out of here." Greg Beaumont (00:46:07): Yeah. And I think the speed of being able to develop, too, it's going to basically allow these tools to be able to do things that people didn't even dream of in the past. It's not just going to be traditional business use cases. I know in healthcare, something that's a hot topic is genomics, right? Genomics is incredibly complex then you go beyond Power BI and into Azure at that point, too and Cloud compute and things like that. Greg Beaumont (00:46:31): So, with Genomics, you think about your DNA, right? Your DNA is basically a long strand of computer code. It is base pairs of nucleic acids, adenine, thymine, and guanine, cytosine that effectively form ones and zeros in a really long string. Rob Collie (00:46:46): Did you know it effortlessly he named those base pairs? There's that biology background peeking back out. Greg Beaumont (00:46:52): I did have to go look it up before the meeting. I said, "Just in case this comes up, I need to make sure I pronounce them right," so. Rob Collie (00:46:59): Well, for those of us who listen to podcasts at 1.5x speed, that is going to sound super impressive, that string there. Greg Beaumont (00:47:05): Yeah. I should call out, too, though that I'm not a genomics expert, so some of what I'm saying here, I'm paraphrasing and repeating from people I've talked to who are experts, including physicians and researchers. So, this long string of code, if you sequence your entire genome, the file is about 100 gigabytes for one person, okay? At 100 gigabytes, you can consume that, but if you want to start comparing hundreds of people and thousands of people in different patient cohorts, all of a sudden, it gets to be a lot of information and it gets very complex. Greg Beaumont (00:47:35): If you think of that strand of DNA as being like a book with just two letters that alternate, there's going to be paragraphs and chapters and things like that, which do different things. So, one of the physicians I spoke to worked with Children's Cancer. Here's kind of where the use case comes in. So, you take something breast cancer where there's BRCA1 or BRCA2, BRCA1, BRCA2 genes where if you have it, there's a measurable increased probability that you'll get that type of cancer within a certain age range. There's a lot of other diseases and cancers, where it might be 30 genes. And depending on different combinations of those genes, it changes the risk of getting that specific type of cancer. Greg Beaumont (00:48:17): But this physician told me that there are specific children's cancers, where they know that if they have certain combinations of genes, that they have a very high probability of getting this cancer. And when the child actually feel sick and goes to the doctor, it's already spread and it's too late. So, if you can do this sequencing, basically run it through machine learning algorithms, so it will determine the probability, you could effectively catch it at stage zero. Because these cancers, it's something that could be related to growth hormones and as you're growing up, and as you become an adult, you're then no longer at risk of getting that childhood cancer. So, if they could identify it early and treated at stage zero, instead of stage 4, it sounds sci-fi, but the tools are there to do it. Greg Beaumont (00:49:01): It just never ceases to amaze me that you watch the news and they talk about self-driving cars and identifying when a banana is ripe, and things like that. But it's like, you know what? These same tools could be out there changing people's lives and making a measurable difference in the world. I think just especially post COVID, I'll expect to see a lot more investment in these areas. And also, interest because I think that might be one of the positives that comes out of this whole experience. Rob Collie (00:49:27): I do think that the sort of the worlds of Medicine and Computer Science are on a merging course. Let's not call it collision course. That sounds more dramatic. There is a merging going on. You're right DNA is biologically encoded instructions by an RNA. The mRNA vaccine is essentially injecting the source code that your body then compiles into antibodies. It's crazy and it's new. There's no two ways about it. Rob Collie (00:49:56): mRNA therapies, in general, which of course they were working on originally as anticancer and sort of just like, "Oh, well, we could use it for this, too." And there's all kinds of other things too, right? Gosh, when you go one level up from DNA or some point of abstraction, you get into protein folding. And whoa, is that... Greg Beaumont (00:50:15): Crazy, yeah. Rob Collie (00:50:16): ... computationally. We're all just waiting for quantum computers, I think. Greg Beaumont (00:50:20): Now, I'll have to call out that I'm making a joke here, so people don't take me seriously. But if you think about it, the nucleus in each of your cells contains an important model of that DNA, right? There isn't just a central repository that everything communicates with. You have a cache of that DNA in every cell in your body, except red blood cells, which perform a specific task. There may be more of the power automated the human body. But cheap attempt at a joke there, so. Rob Collie (00:50:44): Well, I like it, I like it. Let's go in with both feet. I've also read that one of the reasons why it's difficult to clone adult animals is because you start off with your original DNA, but then you're actually making firmware updates to certain sections of the DNA throughout your life. And so, those edits that are being made all the time are inappropriate for an embryo. Greg Beaumont (00:51:09): Yep. Rob Collie (00:51:10): And so, if you clone, you create an embryo, right? And now, it's got these weird adult things going on in it. That's why things kind of tend to go sideways. It can all come back to this notion of biological code and it's fascinating. A little terrifying, too, when you start to think of it that way. I've listened to some very scary podcasts about the potential for do-it-yourself bioweapon development. There was this explosion back, in what, in the '90s when the virus and worm writers discovered VVA. Remember that? We call them the script kiddies that would author these viruses that would spread throughout the computer systems of the world. And a lot of them, the people writing these things were not very sophisticated. They weren't world renowned hackers. Greg Beaumont (00:51:53): For every instance where you can use this technology to cure cancer, you're right that there's also the possibility of the Island of Dr. Moreau, right? You go look up CRISPR Technology, C-R-I-S-P-R, where they can start splicing together things from different places and making it viable. And 10 years ago, they had sheep that were producing spider webs in their milk and it's just, there's crazy stuff out there if you kind of dive into the dark depths of Biology. Now that we went down the rabbit hole, how do we correct course, right? Rob Collie (00:52:23): Well, we did go down a rabbit hole, but who cares? That's what we do. Greg Beaumont (00:52:26): Even you kind of step it back up to just kind of easy use cases in healthcare, so one of the ones that we use as a demo a lot came from a customer, and this was pre-COVID. But something as simple as hand washing, you don't think about it much. But when you're in the hospital, how many of those people are washing their hands appropriately when they care for you. And there's some white papers out there, which are showing that basically, there are measurable amounts of infections that happen in hospitals due to people not washing their hands appropriately. So, a lot of healthcare organizations will anonymously kind of observe people periodically to see who's doing a good job of washing their hands. Rob Collie (00:53:04): I was going to ask, how is this data collected? Greg Beaumont (00:53:06): This customer actually had nurses who were using a clipboard and they would write down their notes, fax it somewhere, and then somebody would enter it into Excel. So, there was this long process. And with another TS, who covers teams, we basically put a PLC together in a couple days, where they enter the information into a power app within teams, so they made their observation, entered it in. It did a write back straight to an Azure SQL Database at that time. Now, they might use the data verse. And then from Azure SQL DB, you can immediately report on it and Power BI. It even set up alerts, so that if somebody wasn't doing a good job, you could kind of take care of the situation, rather than wait for two days for the Excel report to get emailed out, and maybe lower the infection rates in the hospital. Greg Beaumont (00:53:53): So, it saved time from the workers who are writing things down and faxing things just from a sheer productivity perspective. But it also hopefully, I don't know if it will be measurable or not, but you'd have some anticipated increase in quality, because you're able to address issues faster. And that's the simplest thing ever, right? You can spend a billion dollars to come up with a new drug or you can just make sure are people washing their hands. Rob Collie (00:54:17): Both data collection and enforcement, they happen to be probably the same thing. There's like, "Oh, I'm being watched." The anonymity is gone. That's a fascinating story. Okay. What kinds of solutions are you seeing these days? What's happening out in the world that you think is worth talking to the audience about? Greg Beaumont (00:54:38): We're seeing this ability to execute better where the tools are easier to use, you can do things faster, but there's still challenges that I see frequently out there. So, I know something that you all are experts in its data modeling and understanding how to take a business problem and translate it into something that's going to perform well. So, not only do you get the logic right, but when somebody pushes a button they don't have to go to lunch and come back, they get a result quickly. That's still a challenge. And it's a challenge, because it's not always easy, right? I mean, it's the reason cubes were created in the first place was because when you have complex logic and you're going against a relational database, the query has to happen somewhere, but like that logic. Greg Beaumont (00:55:19): So take for example, if somebody wants to look at year over year percent change for a metric and they want to be able to slice it by department, maybe by disease group, maybe by weekend versus weekday, and then they want to see that trend over time. If you translate that into a SQL query, it gets really gnarly really fast. And that problem is still real. One of the trends I'm seeing in the industry is there's a big push to do everything in DirectQuery mode, because then you can kind of manage access, manage security, do all of those necessary security things in one place and have it exist in one place. Greg Beaumont (00:56:00): But when you're sending giant gnarly SQL queries back to relational databases, even if they're PDWs with multiple nodes, it gets very expensive from a compute perspective, and kind of when you scale out to large number of users, concurrency is still an issue. So that's something where you look at recently what Power BI has come out with aggregations and composite models. That's some of the technology that I think can mitigate some of those problems. And even if we think about something like Azure synapse, right? You can have your dedicated SQL pools then you can have a materialized view. A materialized view is effectively a cache of data within synapse, but then you can also have your caches in Power BI, and kind of layer everything together in a way that's going to take that logic and distribute it. Greg Beaumont (00:56:46): Does that make sense? Rob Collie (00:56:47): It does. I think this is still a current joke. The majority of cases where we've encountered people who think they want or need DirectQuery, the majority of them are actually perfect poster children case studies for when you should use cash and import mode. Right? It turns out the perceived need for DirectQuery, there is a real percentage of problems out there for which DirectQuery is the appropriate solution and it is the best solution. But it's the number of times people use it is a multiple of that real ideal number. Rob Collie (00:57:17): I think part of it is just familiarity. Still, I've long talked about how we're still experiencing as an industry the hangover from most data professionals being storage professionals. Everyone needed a database, just to make the wheels go round. The first use of data isn't BI. The first use of data is line of business applications. Every line of business application needed a database, right? So, we have minted millions of database professionals. this is also why I think partly why Power BI gets sort of erroneously pigeonholed as a visualization tools, because people are used to that. They're used to, we have a storage layer and reports layer, that's it, right? Rob Collie (00:57:56): Reporting services was Microsoft's runaway successful product in this space. Paginated reports is still around for good reason. And I think that if you're a long-term professional in this space with a long history, even if you're relatively young in the industry, but you've been working with other platforms, this storage layer plus visuals layer is just burned in your brain. And this idea of this like, "Why do you need to import the data? Why do you need a schedule? Why do you need all this stuff?" It's like as soon as people hear that they can skip it, and go to DirectQuery, they just run to the comfort zone in a way, right? Greg Beaumont (00:58:32): Yeah. Rob Collie (00:58:32): I'm teaching DAX and data modeling to the Excel crowd. I have a real tortured relationship with the related function. Should I tell them about it in their first class? Because I know what's going to happen. They're going to use it and they're going to gravitate right back to that one giant Franken table model where they use the relationships and then use the related function to turn them all into one big wide table and miss the whole point. And so, it's like, "Do I even tell you about it?" It's like, "Do I even tell the IT director that DirectQuery is a thing?" Because, again, it has its purpose. I'm glad it's on the platform, but it's overused. Greg Beaumont (00:59:09): I think people confuse single source of truth with a single source of data. Rob Collie (00:59:13): Totally. I've heard people say, "How many copies of the data do I need in my organization?" Right? In a very folksy combative tone. Well, you like caches? What about caches? Are you okay with caches? Greg Beaumont (00:59:24): And this is another analogy I sometimes use and it's intended to be humorous and keep people's attention. I'm not trying to make a direct comparison here. I just want to call that out. But I call it the Bitcoin problem. So, with Bitcoin, it can handle I think it's 4.7 transactions per second. And people want to use it as a currency the way you use a credit card where Visa may be handled 1700 transactions per second. So there's a problem with going DirectQuery against Bitcoin and that it can't handle the concurrency and the scale. Greg Beaumont (00:59:55): And so, there's a lot of these crypto projects out there that are trying to create basically ways to kind of resolve all the transactions and then periodically true up with the source. And I'm not an expert in that area either. I just, I think it's fun to read about. During COVID, I watched some things on Bitcoin when I was stuck at home. I saw a presentation, if anybody gets a chance to check it out, called The Future Will Be Decentralized by Charles Hoskinson, who was the founder of Cardano. And that's when it kind of clicked that they're not just creating fund money, they're creating effectively a whole new economic system or they're trying to create a whole new economic system. And some of the technologies might actually someday replace the Cloud. It's really interesting stuff that they're doing. Greg Beaumont (01:00:36): But kind of circling back to where I started, it's kind of the same thing with the database. If you just try to run all the logic directly against the source, you're running massive amounts of logic for massive numbers of users in parallel. And caching reduces some of that pressure and it also allows people to have kind of specialized use cases where you're not doing 20 joins every time you select a filter. You do it once, and then you filter from those results. Rob Collie (01:01:03): The Vertapak Engine, the end memory column store, all my years at Microsoft, that was the only thing I was ever close to that felt like what you would expect from a software company in a movie. This was science fiction. This technology was developed. It was sci fi and it was real, and it's still sci-fi today. It's so amazing what it is capable of. It is mind blowing the performance aspects of what it can do, and how effortlessly it can perform them. Rob Collie (01:01:36): And to leave that out of your implementation like this magic piece of software, it's impossible what it does. It's still impossible. I still, I don't even remember how it works anymore. To leave that out, you're really leaving a lot on the table. And so, let's talk about what would be some cases where DirectQuery is the right answer? Greg Beaumont (01:01:54): Near real time, so when you need data quickly, and you need it to be in the hands of the users without anything in between DirectQuery is absolutely the best use case in that scenario. There's other solutions. I know you've dug deep with Denny Lee recently on Big Data, where when there's just massive amounts of information, you don't want to cache that information. The purpose of caching is not to go get everything. It's to reduce the complexity of the logic. So, if you have a gigantic database, and you need to go get details from it, absolutely DirectQuery is the best option. Greg Beaumont (01:02:27): And just when you kind of hit the technical limit of the caching within a tool like Power BI, you have to go to DirectQuery. I mean, there's just a certain point where you get up in the hundreds of gigabytes for a cube and it's going to perform better on DirectQuery mode. Just because the technology kind of hits that limit, where the benefits you get kind of max out and start to degrade. Rob Collie (01:02:49): I worked with Chris Finland, when he was in the field on a project where we ended up with, at that time, it was 2013 and maybe, it was 2014. Anyways, SSAS tabular and 3 billion records in the biggest fact table. And this thing was running on 32-gigabyte VM and it was all loaded in RAM. It was having no trouble at all. And this was despite it having an incredibly complicated fact record structure, such that every single fact record in the model, all 3 billion rows, every single one of them was an inception to date number. Not what happened to that month, or that day. As of that day, the number in that database was this was what has happened in this corner of our business rewinding 50 years. This is the grand total over time. Rob Collie (01:03:37): And so, even to get the current activity in a particular timeframe, it was a time intelligence measure. The most basic measure in the entire cube like, "What happened that month? How much revenue came in that month?" It was time intelligence, right? You had to take current number and subtract the yesterday number to know what it was. It was like the lights should be flickering every time someone touches this thing. It worked great. I was just, it was stunning. Greg Beaumont (01:04:05): So, the one thing I hear where people I work with are going to strongly disagree with me on this is a lot of people still think that caching and middle layers are a Band-Aid until the DirectQuery technology gets better. This is just my personal opinion based on what I've seen and what I've experienced. I see over time where I mean, just imagine this scenario, okay? So, you have a solution that requires row level security and you can have a little note of compute on your local computer that contains just the rows that you're allowed to see with kind of a distributed tabular model. Greg Beaumont (01:04:37): That doesn't exist today, but it could potentially in the future versus taking all that data and putting it in one place inside of a data center somewhere and having everybody communicate. To me, it just seems like it would be at least something to consider, right? I'm not an expert in the area, but I don't think that caching and distribution of kind of the logic is going to go anywhere soon. I think it's here to stay for some time. Rob Collie (01:05:00): And then you've got this technology whose primary purpose is storage and retrieval. And then you've got this technology that its primary purpose is analysis. And they're going to make different tradeoffs. They have to make different tradeoffs. In fact, one of the reasons why you consider not near real-time, right? Why is near real-time a good use of DirectQuery? Well, because you can't rebuild the Vertapak model multiple times a second. That's a tradeoff, right? It can't be updated at the individual record level like a regular database can be because it made tradeoffs. Rob Collie (01:05:33): I think you could almost mathematically prove that the Vertapak engine is close to theoretically optimal, in terms of how fast it is at what it does. You just can't sideline that thing. And it's not like the storage engines are ever, ever, ever, ever going to support a mode of DirectQuery that's going to be that fast. So, yeah, I think that's the way to look at it, right? Is it like you want to use the magic engine, sometimes you just can't. And you should be disappointed at those times. And then happy that DirectQuery is an option, but you should be disappointed that you weren't able to use the magic thing that's going to make everything better. Greg Beaumont (01:06:09): I'd add it's usually less expensive, too, but usually the cost of doing it that way is less expensive for the organization and the query performance is still usually better. Rob Collie (01:06:19): Yeah. It's a funny story that when I was working on that solution with Chris, back in the day. This was a Christ's reaction. I'm pretty sure that somewhere in the account team, there was a bit of like, "Oh, really?" When we found out that we were able to, because this is back in a very different licensing model. The world back then was very, very different in terms of how Microsoft licensed their software and it was per CPU per machine, whatever, right? And the fact that this gigantic model, with the entire financial history of this Fortune 500 firm, been around forever, was stored in this one model, and was able to be run on a single 32-gigabyte VM was a bit of a bummer to the people who are trying to sell software, right? Greg Beaumont (01:07:04): Yeah. Rob Collie (01:07:05): It's like this absolute apex predator of a project. We get one VM of additional footprint, are you kidding me? Greg Beaumont (01:07:14): Yep. Unreal. Rob Collie (01:07:16): Hey, Microsoft's loss is your gain, customer. Greg Beaumont (01:07:24): I do see still challenges with creating those cache layers. You look at a tool like Aggregations, where it's allowing you to have hidden summary tables sitting behind your fact tables or alongside your fact tables. It's, you really have to understand data modeling to set those up. And you have to understand how it works within the context of the tool and the context of what people are using, but if you go look at the roadmap, within Power BI, you'll see auto ags on the public roadmap, which is the Automatic Generation of Aggregations based upon query patterns. I'll be interested to see how that actually looks when it comes out. I don't have access to anything that's not publicly available. That's out there in the public. Greg Beaumont (01:08:03): And then on the synapse side, materialized views is kind of the same thing. And you'll also see a roadmap item for the query accelerator with synapse where it's going to look at the queries. It's getting from Power BI and then spin up materialized views, which for all practical purposes, as I mentioned before, another version of a cache, that will then kind of self-tune the model to get better over time. And hopefully alleviate some of the need for people to actually learn how to do it manually. Again, it's moving from PaaS to SaaS as the other components have. Rob Collie (01:08:34): And those sorts of improvements that are in the works. I mean, this is where some of your colleagues get the idea that we're just sort of sitting around waiting for the day of DirectQuery parity. There are developments being made to improve forever, right? We can always improve. We're going to get to Vertapak level. Greg Beaumont (01:08:48): Yeah. I see those demos where they say that NLP will replace the data analyst. When you're younger, it's like, "Oh, no, I'm going to be out of a job." Now, it's like, "No, that's I'll be long dead before that ever happens." Krissy Dyess (01:08:59): So, Greg, do you have any hobbies outside of work? Greg Beaumont (01:09:03): Yeah, we actually kind of live off the beaten path a little bit. It's effectively kind of a small, almost a hobby farm. It used to be a horse ranch, so I spend a lot of time doing stuff in the yard. And this last weekend, residing my garage. I spend a lot of time doing family stuff and things like that. When I'm not working, I don't to be sitting in a desk. It's like I want to go build something with my hands or I want to go somewhere and do something and travel that kind of thing. Krissy Dyess (01:09:32): Have you always been in Minnesota, too? Greg Beaumont (01:09:34): No. So, I actually came from a military family. I think we moved seven times when I was a kid. Lived all over the country, but we ended up kind of landing here. And my wife's family is from here, so I've got roots that aren't going to be severed. We could be here for some time. Yep. Krissy Dyess (01:09:50): Yeah. I kind of have the same thing here in Arizona. I mean, I was able to move around until again, I found my family here and it does make it hard to move when you had set your roots. Greg Beaumont (01:10:00): Yeah, with things like teams, everything's virtual now. Krissy Dyess (01:10:03): Yeah. It's not the same. Greg Beaumont (01:10:06): Yeah, yeah, yep. Rob Collie (01:10:07): What is it with Minnesota and BI, though? There's something to it, right? We have more consultants, full-time consultants working for us from the State of Minnesota than any other state. Greg Beaumont (01:10:17): Yep. I know a lot of your employees are from Minnesota, and also a lot of people I work within Microsoft are from Minnesota in the data world. I don't know the full answer to that question. I do know that we have a lot of industries here that are very data-centric, right? So, you have a lot of device companies. You have large, probably one of the largest insurance companies that is based out of Minnesota. Greg Beaumont (01:10:39): There's a lot of kind of medical innovation happening in Minnesota with the Mayo Clinic down in Rochester and University of Minnesota. And there's also a lot of schools that have very good math programs, and very good engineering programs, even all the way up to Fargo and North Dakota. They upgrade engineering schools up there. So I think there's just kind of a hub of education and technology and industry that kind of combines to kind of find those 5% that you talk about and give them that opportunity. Rob Collie (01:11:11): Yeah. On a per capita basis, Minnesota has got serious game in the data space. It's tempting to think of it as, "Ah, it's just small sample size." But I don't think so. I think there is something in the water or equivalent. Maybe, it's the absolutely brutal winters. You've got to find something to... it's like, "Where do you go for football players, we go to the places where it's warm all year. Florida, Texas, California. What do you go for data people? Well, you need to go someplace where if you step outside, four-month window, you just die." Krissy Dyess (01:11:41): It's longer than four months. Greg Beaumont (01:11:44): It is, yeah. It's probably a five-month winter. The ideal situation would be able to stay here until New Year's, and then probably come back in April, right? I have friends and family, they're going to hate me for saying that because they snowmobile and ice fish and my neighbors across the street will put up an ice house, out on the lake. And so, some people actually love- Rob Collie (01:12:06): They're building structures on a lake. Let that sink in. Greg Beaumont (01:12:12): Yep. I've heard of stories where up on Lake Mille Lacs, you'll drive a mile or two out on the lake to get to your ice house. And I've heard stories of people who aren't from here going out to the ice house and saying, "Where's the lake?" And they're like, "Oh, the shore is two miles that way." Rob Collie (01:12:26): Yeah. That dawning moment of, "Oh, my God. " Greg Beaumont (01:12:30): Yep, yep. Rob Collie (01:12:33): There were street signs. Greg Beaumont (01:12:35): Well, yeah. They actually do have street signs on the lake in the winter. Some of these were posted. Rob Collie (01:12:41): I grew up in Florida. And I thought when I went to school in Tennessee, I thought, "Oh, my God, is it cold here," right? And then eventually, I ended up living in Cleveland, I'm like, "Yeah, this is really cold." And then I took a couple of trips in the winter, in February, to Minneapolis. Like, "Oh, my God." I couldn't even keep the ice off of the highways underneath the overpasses. No amount of salt was going to do it. No, no, this is super frozen. Whatever that is, I don't know. Yeah. Greg Beaumont (01:13:10): I had a co-worker once, who was born and raised in India, had never left Southern India, and came up here on assignment without ever having seen snow. And it was below zero when they got off the plane. So, I mean, you can imagine the shock, because it is something you have to acclimate to. Rob Collie (01:13:29): I can't imagine. I would need, I would need [crosstalk 01:13:31]. And then I remember sitting there going, "Oh, that's right. There's an entire country north of here. What is wrong with those people?" Krissy Dyess (01:13:38): It's just- Rob Collie (01:13:41): It just seems like the absolute northern edge of human expansion, and then you realize, No, there's a whole industrialized nation up there. Greg Beaumont (01:13:49): Yeah. People think of this as being the arctic tundra, but all of Canada is basically north of us. Rob Collie (01:13:54): My friend, David Gaynor, who is going to be on an upcoming episode of the show, he grew up in Alberta, you know it's? Greg Beaumont (01:14:00): Yeah. Krissy Dyess (01:14:01): Every year we go up in Phoenix. We can go up north, just a couple hours in the Flagstaff in December, maybe January, get a little bit of snow. And the kids, they come up. Families, they bring their children to see snow for the first time and they all do it. They all stick their hands in it. And then that sensation, that burning sensation starts to kick in, and then two minutes later, they're all crying, going back into where they came from. And it's just like every year, you go up and you see these kids for the first time when they touch the snow, right? Like touch it and then immediately in tears. Rob Collie (01:14:35): Right. Yeah, it's cool. Hi, Greg, so here as we're sort of closing up, what are some of the things that you see coming, whether they're new technologies or adoption trends that you think are most significant or perhaps you also find particularly personally exciting? Greg Beaumont (01:14:51): Yeah, so if we look at some of the new capabilities we've seen in both Power BI and on the Azure side, there's a lot of focus on Machine Learning. AI And combining data from different places to get insights. Something that I think is kind of extremely valuable, but it's just not as prevalent in demos and presentations and things like that is the integration between something like Azure Machine Learning and Power BI, where it's still hard to create a good machine learning model. You probably want, especially in healthcare, you want real data scientists creating your machine learning models. But it used to be really hard to then put that into practice, right? You might have something that does a great job of predicting, but then how would an analyst use that data unless somebody else is just providing it to them. Greg Beaumont (01:15:38): Now, you can literally go into Power Query your data flows, and select a machine learning model that you have access to, and then take the corresponding columns of data and map them to the inputs of that machine learning model. Hit go, it will do all the work for you. You don't have to configure any APIs or write any code. And then you're getting access to that predictive technology at your fingertips. Greg Beaumont (01:16:02): There's also Auto ML if somebody wants to learn about machine learning, where you can start building simple machine learning models right in Power BI. What I found, though, is that by the time somebody really understands Auto ML, they're usually ready to graduate to the Azure side of the house. But I see that integration of, not only being able to get all of this data from all these different places and tie it together, but then be able to go beyond doing simple math and using machine learning algorithms is kind of the next big thing in both healthcare and beyond. Rob Collie (01:16:36): That's a fascinating topic. Long time ago, when I was first working with PowerPivot, I had some friends who had left Microsoft and gone off and formed a machine learning startup, and some of them are back at Microsoft now. Really, really, really smart people. And it was natural for me to try and to collaborate with them and vice versa at that time. None of the PowerPivot models that I was building, it turned out none of them had anything interesting to be found with machine learning. Rob Collie (01:17:04): And it was a hard lesson, which was by the time you're done aggregating, overwhelming majority of Power BI models and reports operating at an aggregate level, by the time you're done aggregating, you've kind of lost all of that grain level variation that is interesting to machine learning. So, I learned at that moment that a lot of these technologies are meant to operate at, you can think of it as being like operating at the fact record level, not on the aggregates. Rob Collie (01:17:33): And so, whenever I hear about machine learning and Power BI coming together, my brain immediately goes back to those old days of, "Oh, no, these two are incompatible." And that's my first instinctive response. I have to think about it a little bit longer before I go, "Okay, there's actually ways they can interplay." And I haven't tried this thing that you were talking about, but it sounds amazing. Rob Collie (01:17:54): At the Power Query level, you could be importing additional columns, you're mapping columns that I'm assuming that you would get back an additional column or multiple columns, with some sort of predictive score, right? Maybe like the percentage chance that this customer is going to be leaving. Attrition risk or whatever, or things of that nature. What's the easiest way to get started with that stuff? Greg Beaumont (01:18:18): I think I'd add two things there. So, the easiest way to get started is right in Power BI Desktop. If you open Power Query, it's in the ribbon on the far right hand side, you might have to enable it. But you could start with cognitive services. And you could just say, "For each row of data, for this column, tell me what language that comment was written in?" And you can count how many responses are in Spanish versus English versus Portuguese or whatever it may be. Greg Beaumont (01:18:41): Another example would be sentiment analysis, right? And this one is always funny in healthcare, because sentiment analysis is looking at words and then saying positive, neutral, negative from I think zero to one. But in healthcare, the words mean different things, so there was one that came out as being extremely positive. It was tenderness, right? Because in healthcare, it means you're sore and you hurt. But outside of healthcare, it's a positive emotional word, right? Yeah. Rob Collie (01:19:08): There's also doctor speak in general, which is like it requires a completely different sentiment filter. I had a salivary gland tumor removed recently, which I'm fine now. But if you read the pathology report on what was going on with me, right? As a human being, non-trained professional. Greg Beaumont (01:19:25): Scary, yep. Rob Collie (01:19:25): And you read that, you'd be like, "Oh, man, Rob, you're going to die." So, I don't... yeah. I wouldn't want to the sort of the vanilla sentiment analysis looking at that. Greg Beaumont (01:19:37): I wanted to add one more use case, too. So, you referred to doing predictions on aggregations. One use case where that might actually be applicable is let's just say somebody wants to do a simple forecast. Right? You can do this right in Auto ML. I'm actually working on a demo on for it right now. I don't have it ready to go where the analyst comes in and says, "I want to forecast at the level of the individual provider by day, by disease category, by department," something like that. And then you do the forecast and you find it's not very accurate. Greg Beaumont (01:20:09): Well, you can maybe make an aggregation where you roll up the forecast to the level of by physician by week, rather than day, so on and so forth. And change the level of granularity. Rerun the Auto ML test to see how accurate it is. And then you could go back to your data science team and say, "Maybe we want to do the predictions at this level of granularity, because that's the accuracy level that I'm looking for." I agree with you that 99 times out of 100, you want the most granular data for those types of efforts. There are those scenarios we're kind of... Rob Collie (01:20:41): Totally. Greg Beaumont (01:20:41): ... coming up with summary tables to do the predictions to have that be more agile. I think it's going to create a lot of value. Rob Collie (01:20:47): I'm mostly just reflecting my frustration from that era. We were failing to find anything useful. My friends at startup were telling me, "Yeah, Rob, you just don't understand how this stuff works yet or you can't aggregate like that." I was still very stubbornly insisting that "Okay, come on." There's still entities in the world, for example, like a store. Let's say you're a chain with 500 locations. You have all kinds of interesting attributes at each of those locations. "Is it a two-story store? Is it the deluxe store? It's blah, blah, blah, blah, blah. Does it have the pharmacy built in or not?" But all kinds of these aspects, right? Rob Collie (01:21:18): And you would never predict future transactions on a transaction level? Doesn't make any sense, right? What would you forecast this store's revenue to be next month, right? So, completely valid machine learning problem. And so, I'm glad we did circle back to this because I never had the right kind of data to drive that sort of analysis, that sort of machine learning analysis. It just didn't happen to exist in the models I was using at that time. I wouldn't want people to come away from this going, "Oh, no, you can never. Machine learning and aggregate level are incompatible." That would be the wrong conclusion. It was just harder to get to that point than I had expected. I sort of naively expected it to just like, "Okay, here we go jump off the page." And it didn't. Greg Beaumont (01:22:03): I agree with you. If they could ever find a way to combine multidimensional compute with predictive technologies that would be kind of the Holy Grail. Rob Collie (01:22:12): Greg, I can't thank you enough. You brought so many really interesting perspectives. I'm really grateful for the thoughtful approach that you've taken and I think people are really going to appreciate this episode. So, many, many, many thanks. Thanks for being here. Greg Beaumont (01:22:26): Yeah. And thanks for the opportunity. And I was listening to the show even before Krissy reached out. The service you're doing for the community here is absolutely fantastic. Thank you. Rob Collie (01:22:35): Thank you very much. That's really gratifying to hear. Announcer (01:22:37): Thanks for listening to the Raw Data by P3 Adaptive Podcast. Let the experts at P3 Adaptive help your business. Just go to p3adaptive.com. Have a data day!

Beyonce, Prince, Madonna...Like so many of these iconic one word name celebrities in the music world, Chandoo is as unique and talented as they come in the data world!  His story is quite inspiring, his heart and soul are warm, and his brain is brimming with great ideas! All Things Chandoo: Chandoo.Org Chandoo's Youtube Power BI Playdate Budget VS Actual Articles Some Creators and Channels that inspire Chandoo: ElectroBoom Weezy Waiter Hybrid Calisthenics Ali Abdaal  References in this episode: Mike Miskell Tribute To The Wolf Episode Timeline: 4:40 - Chandoo's introduction to Excel was born from necessity (like many of us!), The birth of Chandoo.org (often imitated, never duplicated), and the uniqueness of Chandoo that makes him a huge success 31:50 - Chandoo's Excel Dashboarding is exquisite, his transition to Power BI, and what really matters in one's career 54:10 - Chandoo the Excel celebrity and the Power BI celebrity, Lambda functions, and a curveball question for Chandoo about working for Microsoft 1:06:25 - Chandoo and Rob cross paths, Chandoo's iconic hair, the Game-Changing features of Power BI, and some Power BI hacks 1:33:45 - What's next for Chandoo? Episode Transcript: Rob Collie (00:00:00): Hello friends. Think for a moment about the people that you're aware of, who only go by a single word name. They're usually musicians, Prince, Madonna, Cher, Beyonce. There are a couple of non-musician examples that come to mind like Oprah, for instance. These tend to be celebrities on the world stage. Well, today's guest is the rare exception that pulls that off within the Excel, Power BI, and data community. And I'm talking, of course, about Chandoo. Chandoo is one of the completely original early stage MVP-type celebrities within our community. He blazed a path that now hundreds, if not thousands of people have followed. And sometimes with things like this, it's really that first-mover advantage that really sets someone apart and he did, in fact, have that kind of first-mover advantage. But he is still, to this day, so incredibly unique that I challenge anyone to actually truly duplicate him. Rob Collie (00:01:06): He is legitimately one of a kind. And for me, he's been there literally since the beginning, even physically, since the beginning. He and his family came to live near us in the United States for a summer. That first summer after which I had formed P3 as a company. With someone as gifted as Chandoo, it's always easy and tempting to sort of assume that they've always been doing what they're doing. And he is very gifted, but it's not like those gifts, where always from the beginning, oriented towards something like Excel. Just like many of us, he had to have his collides with moment, the moment where you bounce off of Excel or you stick to it and obviously, he's stuck. So, of course, we go back to and explore that origin story. And also, like many professionals in this space, Chandoo has, over the years, branched out from Excel into Power BI, creating such wonderful offerings like the Power BI Play Date, which we talk about a little bit. Rob Collie (00:02:07): So, we talk about that, what it's like coming from the Excel background and digging into Power BI. He had some unexpected observations there that once I heard them, I was just nodding. "Yep. Yep. That's right." And that conversation also then led to a familiar conclusion that again, I wouldn't have expected from Chandoo, but of course, I should have. And another part of the conversation, we also talked about where he looked for inspiration, where he looked for stimulation and new ideas. It was great to catch up with an old friend, who was also just a wise and dynamic soul. So, without further preamble, let's get into it. Announcer (00:02:48): This is the Raw Data by P3 Adaptive podcast, with your host, Rob Collie and your cohost, Thomas LaRock. Find out what the experts at P3 Adaptive can do for your business. Just go to p3adaptive.com. Raw Data by P3 Adaptive is data with the human element. Rob Collie (00:03:12): Welcome to the show, the one and only Chandoo, how are you? Chandoo (00:03:17): I am doing good, Rob. How are you? Rob Collie (00:03:19): Fantastic. Been looking forward to this for a while. We've been trying to schedule this for probably three or four months now. And here we are like a power reserve. We saved a Chandoo interview very carefully for that six months over the podcast. Actually, how many months are we in now, Luke? Is this our 10th month? Luke (00:03:37): Start on October, early October. Rob Collie (00:03:40): We're potentially in our 10th month. That's what we do. We lose track of time. You're one of the sort of original internet celebrity instructors, often imitated. There's a lot of people who I've seen, sort of explicitly trying to follow in your footsteps and to varying degrees of success. You're not a formula that really others can follow because there really is, and this is awesome to say this. There really is only one you. I've learned that when we actually met. I didn't know that over the internet. How'd you get started on Excel? That was the beginning, right? Chandoo (00:04:22): It's a long story, but that's what we're here for, anyway. Rob Collie (00:04:25): That's right. Chandoo (00:04:26): So, I first remember using Excel all the way back in 2000. There were times before that I used it, but 2003 is the first real moment in my life when I actually used Excel for something. And this is not even to do anything with what I'm doing nowadays with Excel or Power BI. So, the reason why I use it at that point in life is I was preparing for some computer exams. So, I just finished my graduate studies in computer science, and I started working, but simultaneously, I was preparing for some MBA exams. And in India, there is a lot of competition when it comes to getting into a good college for doing your masters. So, they have all these highly competitive exams where sometimes, upwards of 200,000 people will take the exam and just about 500, 600 people will actually be admitted into the college. Rob Collie (00:05:20): Wow. As like a 0.1% acceptance rate. Chandoo (00:05:25): Yeah. You look at the Ivy League and other top university acceptance rates and then, take it to India. Then, it is nowhere near, like you'd be amazed at the craziness that goes on with some of these places. There are a couple of reasons like India has billion people, right? Obviously, there's lots of competition. On top, there were fewer universities at that point of time. The government has added many more now, but still, with our number of people, it is very less compared. So, there is all these factors for that reason. The competition is very high. As part of preparation strategy, everybody would go and take a lot of extra lessons outside just to learn how to prepare for the exam. And then, they'll take these mock examinations sometimes upwards of 25 or 50 in a year just to prepare for the real thing. And there's only one real thing that's a physical thing at that time. Chandoo (00:06:19): So, you can't really make mistakes when the real exam happens, but you have all the luxury of making mistakes in this mock-up stage or that you can learn. And because there is a lot of data coming in from all these exams, right? When I take an exam, there's like 200 questions or 150 questions and I would attempt some. I'll get some right, some wrong. I could use Excel to just keep track of what I'm doing in these exams, what mistakes I'm making, and if I spot a pattern like this automatic question, I'm making the same mistake again and again, then I will change my study of course to plan and address that particular gap or try to change my strategy, so that I won't attempt that area of questions and instead, focus my time on other things. Chandoo (00:07:01): So, that's really when I used Excel and I made this massive spreadsheet just to keep track of what I was doing in those exams. And it kind of really helped me finally get a good grade in that and get into college for my masters. But obviously, you can say Excel is built for anything and everything. So, that was one of the use cases, but I was not really using any of the formulas or none of the power of Excel. And I didn't even know what it is capable of, but that was the one vivid memory of Excel early on. Rob Collie (00:07:35): Do you still have a copy of that spreadsheet somewhere? Chandoo (00:07:38): Many people ask me this. This is simply because back in 2003, 2004, internet is still kind of very nascent in India. It started off as a Yahoo Group. I don't know if you remember, like Yahoo Groups. It's like a collaboration. Rob Collie (00:07:52): I do. Chandoo (00:07:53): But then later on, the forums were a big thing. So, 2003 was the time when in India, we have these preparatory forums where many of us who are all over the country would log in there once in a while, share our stories of how we are preparing, what we are doing, what is going on right, what is going on wrong. So, we could all learn from each other and collaborate, and win this exam. So, I posted a story of how I prepared when I finished the exam and the spreadsheet was part of that story. And then, many people asked us, "Can we get a copy of this?" But in those days, I didn't even have internet at my home. I would go to my workplace to submit something to this forum. So, the spreadsheet was in my home computer and I think I lost it. I don't think I have it anywhere, or it's probably still in my Yahoo Mail. The password of which I no longer remember, or even use. It's gone. Rob Collie (00:08:46): So much of things like that from that era, for me, even though I had great internet at the time, so many of those things are lost because we didn't really have the cloud file storage yet. Today, anything that I ever think is even remotely, possibly valuable, immediately gets saved to Dropbox. I've got terabytes of Dropbox space that I'm never going to ever use in my life. So, everything is saved past a certain point. But before that, it's kind of almost like in geology, it's below this certain rock layer where the earth just kind of ground, everything's gone. So, it makes sense that it's gone. Do you remember how many columns were in that spreadsheet? Roughly, was it question number and right or wrong answer, that kind of thing? Was that what it was? Chandoo (00:09:33): It's not exactly like that. It was not even structured that way because I didn't even know how to use Excel at that point. I think I started off putting stuff in a notepad file or something. And then, I thought, "Man, this sucks because there is no way to visually see or identify things here." So just, I opened an Excel spreadsheet and started putting it there. This is not a podcast on that exam, but that exam used to have like four or five different sections. It is all quite random. You wouldn't believe, there is no set pattern or anything. The number of questions, number of sections, everything could change at any point. Chandoo (00:10:07): There is no official director that these are the things that you would be tested, but the general outline is you would have questions on English, you'd have questions on mathematics. And then, the mathematics itself is split into couple of areas. So, one is arithmetic and then the other is it's called logical reasoning. And then, sometimes, they would further split that into understanding data and graphs and making business decisions from it. So, three or four sections, essentially. So, there's, I think, four big columns. Some of them had further split into multiple columns based on what the heck I was doing. If I think, "Oh, maybe I should keep track of this." Then, I would just put something there and fill some color in there just to remind me what it is. Rob Collie (00:10:51): My daughter is, right now, in the middle of taking the college entrance exams, SAT and ACT here in the United States, and it would never occur to me to spreadsheet. And she's trying to get her scores to a particular level to get to a particular college, right? It takes some effort. It would still never have occurred to me. And now, I'm wondering if it should have. Never have occurred to me to make a spreadsheet, where she's performing well and where some opportunity to raise score. Chandoo (00:11:18): They probably have access to better tools and apps and stuff like that these days. But yeah, a spreadsheet is the original app, I think. Rob Collie (00:11:27): Yeah, it is. It is. I think that necessity is so often the spark. The Olympics just wrapped up. You watch these events where everyone looks like they're doing exactly the same thing. They're using exactly the same form. And then, it's like a couple of millimeters or something that separates the gold medalist from the fifth place. The expert watching says, "Oh, see right here where this person's little toe kind of flaps the wrong way. That was a big mistake." That's what costs them. And it kind of seems like that when there's 200,000 people competing for a few hundred spots. It's like that, right? Like one question is going to drop your rank by potentially thousands of people. Chandoo (00:12:12): Yeah, totally right. Rob Collie (00:12:13): The pressure. Chandoo (00:12:14): There is a lot of pressure and I think, it is probably one of those formative things in my life, too, that having been through that journey. So the exam, I took it during my final year of college because I thought I know why go and work for some time. I might just finish my graduation and then, just go for post-grad. But I didn't get anywhere near the required cutoff to actually go in and make it for the colleges. So, and I felt really bad because I thought all this was like something that I would easily get. Chandoo (00:12:44): I used to have this self-perception that, "Yes, I'm awesome." In college, you are in a bubble, right? You're not really aware of this wider world out there where there's another 195,000 people who are also writing this. So, that was the wakening call for me. And then I thought, "Oh man, I need to actually sit and strategize this and prepare for it." Like I'm attacking this rather than just wake up and go and right. So, that's preparation became a real thing and I prioritize that, set aside time for it every day. And then, we'll track the shit out of it every day, really. Rob Collie (00:13:20): Yeah. Like I've told this story on this podcast before, but it's metaphorical. I go out to a field day, almost like miniature Olympics for a middle school. I was probably like in eighth grade, and I was going to run this race. It's one lap around the track, which to me seems like a distance race. Your kids can be a fast jog and that starting gun went off and I come out in the fast jog and the other guys are all sprinting from the very beginning. And there's this moment of realization like, "Oh, it's going to be like that." Next thing you know, I'm sprinting. I think I've experienced multiple junctures in my life that are like this. You think you're just going to go and do your thing and just be yourself and be excellent and just be your own self-image that you've very carefully curated for yourself without realizing it. And then, the real world goes, "Oh no, uh-uh (negative). That's not going to cut it." It's a real shock, isn't it? Chandoo (00:14:22): Yeah. Rob Collie (00:14:22): I've had many of those. Chandoo (00:14:24): And I think, that is necessary, especially, probably if you get that kind of a shock too late in your life, you might be too set in your ways to change anything. But when you are becoming an adult, when you are still forming your opinions and ideas about the world, having as much of these experiences as needed is very much necessary, I feel. I mean, even today, I would welcome that kind of things. But growing up, I look back and I think, "Ah, man, that was really what made me who I am today." Rob Collie (00:14:54): Microsoft was a big moment like that for me. That was a moment that lasted years. That was a bad one. I still have all kinds of relatively civil disagreements with my ex-wife about raising our teens. And I'm always of the opinion that like, "Oh no, no, no. The earlier they can experience failure, the better because the consequences are lower. The amortized benefit over time is greater." She's of the opposite. She's there to catch them and prevent any sort of failure, very proactively avoiding failure for them. And I'm like, "Oh no, no, no, no, let them fall. It's it's good for them." Chandoo (00:15:38): I feel like maybe, I have lucked out. I mean, obviously, every parent is so protective of their child, but early on, I think when I was in fifth class, which is like year five in school, I was sent to a boarding school and I never really went back home. I just bumped it from one boarding school to boarding college, to uni, which is also not my place. So, I was never really around my parents for them to kind of catch me if I make stupid choices. It was all like, "You figure it out." And this is all in late '90s, early 2000s when there is no internet, no mobile phone. I still remember, if I ran out of the money, I would have to write a letter, post it, and this would take minimum of three days unless I do some sort of an express mailing, which obviously costs more. Chandoo (00:16:27): So, I'd go for the cheapest thing, postcard. And then, I'll go to my home three days later and they would have to money order the money. There's no bank account concept also. So, they'll have to send it through a postal money order. So, there's actually a lag of like seven or sometimes upwards of 10 days time. And sometimes, they may not even have the money. They might say, Oh wait, we'll send it to you after the first week of the month or whatever." It's all like, yeah. You figure it out, really. Rob Collie (00:16:56): Yeah. There's a week of maybe not eating. Chandoo (00:17:01): You'll have to figure it out. That's pretty much it. Rob Collie (00:17:04): That's it, yeah. All right. So, that was your first brush with it, like for real. But then, obviously, later, your Twitter handle, is it still r1c1? Chandoo (00:17:15): Yes, it is. I wouldn't let go of that. Rob Collie (00:17:18): No, that is an awesome one. I mean, even people who use Excel a lot don't always know about R1C1 notation. So, you end up in a very different strata of Excel skill. At some point later, you ended up in a number of other countries at one point, right? Like you were moving around the world, working for, was a consulting firm. Chandoo (00:17:40): Yeah. I think the real shift to Excel began a little later, especially after I finished my post-graduation. I started working as a consultant with one of the biggest technology companies in India and they basically go around the world, help other companies do their IT better. And it's a very large company. And I was working within the finance and insurance vertical of that company. Obviously, I am not really there to develop software because my role there is to understand what the clients want, translate that into technical terms, so that the software developers, designers, and testers can do their job. So, essentially, I'm a business analyst and it's a fancy word of saying that you would be using PowerPoint and Excel every day. That's pretty much what I was doing. I was building a lot of models, making presentations, taking complex concepts, and simplifying them into Word or Excel, so developers can take that and do their job better. Chandoo (00:18:39): So, early on, I realized, "Man, if I don't know Excel, I'm going to just stay behind in this job." And that's not something that they teach in college. The college is all about, how do you prepare marketing strategy for the fortune, 500 company? And here I am, just sitting in the cubicle, figuring out, "Oh, how do I analyze this? And how do I figure out what's going on with these bunch of projects so that we could improve something?" So, Excel became the real world application that I would use six to eight hours every day. And there were all these colleagues right next to me who do all these amazing models in Excel to figure out the costing for a project or all sorts of things. And I would know nothing about that and I felt really bad. Chandoo (00:19:21): But early on in that job, I was not really doing anything worthwhile. I was just kind of like an apprentice. So, I would only do odd jobs. So, I had a lot of, you could say free time, but I would think that as learning time. So, all I would do is I'd open up Excel. I'd click on random buttons to see, "Oh, what this does. Oh, indirect function, what this would do." So, that got me really curious and I started building some silly things for my personal life, like I'll bill a budget in Excel just to understand how things work, how to make it better. And at one point, I thought, I steadily bumped into something that looks so interesting. And I thought nobody in the world would know about this. I felt like, and I discovered something and they already had my chandoo.org website by then because I am always fascinated by tech. Chandoo (00:20:08): So, I had website created couple of years before, really just as a personal project and I put all my personal life stories there. So, I thought, "Oh, maybe I should just put it on my blog and talk about this new thing that I discovered in Excel." And I put it there. Obviously, it's not a discovery. It's something that people have been doing for ages. It's just that in my own silo, I thought this was new. But when I put there, I got a random comment from somebody in a different part of the world. And that was a weird experience because up until that time, the only people who read the blog are my friends or people who I personally know. I'll tell them, "Hey, I have this blog," and they'll go and read it and they'll comment. But then, I got this comment from a strange dude all over in a different part of the world saying, "You know what, you could also do this to improve the chart." Chandoo (00:20:57): And that kind of blew me like, "Oh, there is actually a community of Excel users who are collaborating and sharing information." And I started slowly doing that over time. And one thing led to another and it kind of blew really out of proportion that at some point, I was actually doing two jobs, right? This consulting job, as well as maintaining the blog in the weekends and nights, just keeping up with the traffic, as well as sharing information, collaborating with people in the comments and email. It became too much. But I also thought, maybe I could go and launch a product here to see if this could become a business. And again, none of this was intentional. It was simply, I would write an article and people will say, "Hey, if you put a template around this, we would buy it." Chandoo (00:21:43): And then, I thought, "Oh, really? You'd pay for this? Okay. Let's just see this." So, that's how things really happen. So, this all began in 2006, but around 2009, after three years of doing that, I left my job so that I could just do this full time. And by then, I had a bunch of not really products. I had two products, main products. So, one is an online Excel class, and the other is a set of project management templates built in Excel. And that's pretty much where it kind of really went from a blog website to a business and a life thing for me. Rob Collie (00:22:20): There are some echoes of some other people's stories in that. There's a little bit of parallel for me. I started my blog after you started yours. I started mine in 2009, long before I really knew what sort of business opportunities would come out of it. I kind of knew that there was a consulting company to be created around this new stuff, but the world wasn't ready for that. I wasn't ready for that. So, the blog existed for a long time before we became a company. It sounds a little bit like Bill Jelen story. It sounds a little bit like Adam Saxton, Guy in a Cube, right? Like it's almost always this side thing. That's just like a passion thing that eventually morphs into something more. Chandoo (00:23:08): You could kind of say that the formula, but again, there are many people who might either give up halfway through the journey simply because life got in the way, or they'd never really got to a point where it could become a self-sustaining thing. And also, some other people might be so lucky as today. From day one, they vision it as a business. But for many of us in this particular group, I think it all happened almost like a series of accidents really, rather than... Looking back, you might think, "Oh, that was a genius strategy to have a blog and this and that." And there's nothing really deliberate there. Rob Collie (00:23:48): Oh, I completely agree. It's like the same thing people tell me about the books that I wrote. "Oh, it's such genius that you wrote it in that informal non-tech book tone, Rob." And I go, "Well, it turned out though," but at the time, it was just a survival strategy. I couldn't get through writing that thing in the other voice. Chandoo (00:24:09): Yeah, I wouldn't have imagine. I think that's the thing, right? It is always good to look back and try to figure out or maybe there's a picture that we draw with all these random dots on the paper. There were other dots... Rob Collie (00:24:24): Or just let other people draw it for you. It's usually more flattering, than what you would draw for yourself, looking back. One of the things that we do on this show is we compliment our guests. We almost like attempt to make you uncomfortable with praise, but it's authentic, right? We don't go out of our way to manufacture things. So, again, I've seen multiple people, almost like explicitly try to copy the Chandoo formula. They've looked in from the outside and gone, "Wow, look at that," right? And go and try to copy it. And it's easier said than done because it turns out that the person behind the Chandoo formula is a little bit unique, like your personality and creativity and humanity. Rob Collie (00:25:14): You integrate that into this technical stuff in a way that you either have that or you don't. You can coach it up in yourself to a certain extent, but to go with all the hard work, there are some innate characteristics that we all look into them or don't look into them and that creativity and that sense of fun and whimsy, it's easy to tell when someone's forcing it. If people have very, very, very good radar for that, you're just so dang quirky in a such a good way. I mean that completely, as a compliment, I call some of my best friends freak shows. It's so cool and to have gotten to know you personally, we haven't necessarily kept in the closest touch, but we definitely got to know each other personally back in the day, and that was awesome. Chandoo (00:26:13): It is awesome. Talking about that formula, you could say it's a formula, but I would say it's one of the proven ways of growing your online brand and making it into a sustainable business. And it's nothing new that I invented. I think you could say, maybe I had lucked out by starting early because around 2003, 2004, that's pretty much when the ecosystem of these blogs and in personal branding was kind of like picking up in a more rapid fashion, just because there's more people with internet, there is more... For example, back in '90s, if you have to create a website, you wouldn't really know where to begin. But 2000s was slightly different because there's software like WordPress or BlogSpot and other stuff, which makes it easy for anybody to get them and then, put their... Chandoo (00:27:03): Which makes it easy for anybody to get on and then put their story out in the front of millions of people. Of course, people may or may not read it, but it was easy for me to put it out. And I think what I did early on is I would read a lot of blogs about growing an online business and an online brand. And this was also not deliberate, it so happened that those were the guys who were loudest in the blogosphere. So if for every 10 articles that are out there, five and six of them would be about the small business or teaching stuff or selling stuff. There's a lot of that, and I would read that and I would think, "Oh, this is a good idea, maybe I should include it in what I'm doing. And this is a good idea, maybe I should do it." But there is also some things that you are gifted with, not really gifted, but those are the things that were a part of your personality even before you jumped into this business world. Chandoo (00:27:54): You either grew up as an introvert or an extrovert, you either have flair for technology or you don't, and you either have good understanding of the language or you don't, and all of those things. So that's really our personality mix. So there is a strange combination of all of these weird things that really helped me reach the audience and say things. And also, keep it fun. I look back and I think, "Oh man, I put a joke in here without even trying." I think that's because I really enjoy... That's the way I liked to say things. My kids are now quite old and they're at a point where they're getting annoyed with all the jokes that I put, but they also appreciate that Dad probably is not going to ever be serious about... I mean, I am serious, I think about everything, but it's just that he's not going to be a strict dad, he's going to be a fun dad. That's really the kind of thing that they say. Chandoo (00:28:53): So that's really me. And I think that was part of the thing. But people can go and take the formula, which is really what I did. When I launched my first online course, I had no clue what to do. So I read this article, they were already doing some online courses in a different field, and one of the suggestions they gave is, you don't have to record the whole thing to sell it. Up until that point, I was thinking I had to create this 20 hour course before I could actually go and sell it. But they said, "Maybe make one or two modules first and then go and start marketing, go and start selling, because there may not be a market for what you're offering, so go and do it." Chandoo (00:29:33): So that's really what I did. I was working in Sweden at that time, and Nishant and Nakshatra were just born, and Jo was with them in India. Because of my consulting job, I'd go to all these places. So I was in Denmark and Sweden that time. And I launched this course, I said that, "Hey, there is Excel School now, please go and sign up." and I created only one module, one or two modules. Then I sold it, and I thought maybe five or 10 will buy it, it's about 60 to $100, the course. In my mind, that was a lot of money. Even today, it is a lot of money, but I felt like at that point, that is big bucks. And I think around 100 people bought it. And that really scared the shit out of me because when you take 100 times 100, that's almost $10,000 really. Chandoo (00:30:22): And $10,000 was sitting in my PayPal account, close to that. And $10,000 is close to my salary if I'm working in India, that's my annual salary at that point in life. But because I was working in Sweden, I would get overseas payments, so it was almost $50,000, that's how much I was making at that time. But I was thinking in my Indian mindset, "I'm making all my annual salary by selling this one course, which is not even ready." So it scared me. And I thought, "Man, I need to do it right by these people. They paid for it, they bought it, I need to deliver it to them, I only made two modules." So I left my job, went back to India, finished recording the rest of it and launched the course. So that's how I learned, and that's the formula that I show in my blog and sharing my personal stories, because I want others to take these ideas as well. But I think the key thing people might miss out is putting their personality into it. Chandoo (00:31:19): If you just want to fake it all the way, then it might be hard, but if you bring yourself in your perspective and your life and your values into it, that will make it your own, and you're no longer cloning anything you're taking the best of what is working for others and mashing it up. Rob Collie (00:31:35): Now, at some point on this journey, not to narrow you in too much, you were running Excel School, it's general purpose. One of the things I think you became known for as an outlier, even within that space was the dashboarding that you would do in Excel. Now that's where we saw the Mozart in Chandoo. I mean, holy cow, people would look at the stuff that you would build in Excel, and it's gorgeous, it's just so beautiful. And everyone... Not everyone, but a lot of people that I knew, very wise, people knew that the quality of their work was going to be judged by the visual impact that their spreadsheets would have. And people would go to your site, and again, they would go to your site for many reasons, but the one that I disproportionately encountered was people saying, "Yeah, we go get the slick Excel visuals from Chandoo." And this is particularly relevant as the world is experiencing the onset of Power BI. And I know you've diversified, you're not just the Excel guy anymore. I mean, heck you did a Power Pivot class for that, in what, 2012, 2013. Rob Collie (00:33:07): I honestly haven't kept close tabs on what you've been doing with Power BI. And that is a real shame because if, and again, I haven't looked, maybe I haven't looked because I don't want to feel inadequate, but as rich of a canvas as Excel is for dashboard creation, oh my gosh, Power BI has really hit critical mass on the things you can do in their report canvas. I feel like now I need to have a Christmas morning moment where I go open up a bunch of Chandoo-approved Power BI reports and go, "Oh my God." Does it speak to you? How's that transition been? Chandoo (00:33:51): Yeah, it's been very good, but also there were a couple of things that stopped me from really going full on when the Power BI way was going up. The number one thing is, between 2015 and 2016, that's when Power BI was gaining that initial momentum, I have been blogging and talking about Power BI as well, but we also chose to move from India to New Zealand. So that was a big move, you are taking all your life that you have been rooting in one country and then now suddenly you uplift and you go to a different part of the world. It is both physically and emotionally very hard experience to go set yourself up in a different place, make new friends and start your life all over again. And also around 2015, you could say, I reached a point where, and I'm not trying to brag or anything, it's just the fact of the matter is, I reached a point where there is no financial incentive that would motivate me to do things. Chandoo (00:34:54): I am very happy with what I have in my life. I have a very good family, enough money to sustain whatever I want to do for the rest of my life, and everything was there. So there is really no carrot in front of me that will chase me to go and get it. I mean, I would only have to do it if I am enjoying this. So for me, the enjoyment started shifting slowly from running a website to other things, like maybe becoming a better cyclist or being around the kids with their life or playing with Lego or doing video games or doing other fun craft things. Because one of the challenges of being creative in any field, I guess, you can't be creative all the while if you're just doing it not for fun reasons, but for something else. I thought, "Maybe I had my day, I'm enjoying things. I don't need to push myself harder." So that's when I turned a blind eye to Power BI, not just to Power, to Excel also. And I would only blog once or twice a month, and that's pretty much it. Chandoo (00:35:57): I would still produce good quality content that I'm enjoying, but I got myself into a place where there are so many other things and balls juggling in the air that I thought, "Okay, this is enough." But after settling down in New Zealand and after things calmed down a bit, that's when I started thinking, "Okay, I need to figure out what I'm doing with my time. You're not really doing it for money or anything, but there is also, you have time." I try to rekindle that passion for data and for helping people become good in their lives. So naturally I reassessed like, "Okay, what are the things that we have available today? So there's Excel, there is obviously Power BI and then there is other tools coming in." Simultaneously, I would do some consulting work for the local government here in New Zealand. So I'd get into situations where the data or the challenges were different than the ones that I have experienced previously. So I'm learning a lot, and I thought, "Okay." Again, my go-to point when I learn something new is, put it out on the blog so other people can also learn. Chandoo (00:37:00): So I created a course on Power BI, it's called Power BI Play Date. I teach dashboards and stuff like that in there. I tried to replicate some of that Xcel crafting and that sort of dashboard mindset, which tries to tell a compelling story and provide a good narrative to the end user rather than just use things for the heck of using it within Power BI. Now, Power BI is a different platform altogether. So it has its own rules and it has its own canvas and things like that, where there are set limitations imposed by the nature of things. Like in Excel, you may have to explain 10 things, but within Power BI, because of the interactions, you don't have to explain 10 things, you have to let your audience know that there are 10 things there, but only bring the important bits out and let them figure out the rest. Chandoo (00:37:50): So I do this and I enjoy it. I run the course and I do more around Power BI these days than I do on Excel. I run corporate trainings and stuff like that as well. It is a different platform and I enjoy building stuff on Power BI. What I do find a little bit lacking though, and I think it's just still evolving, it's too early for us to go and put judgment on Power BI on this space, which is the visuals, sometimes they are not up to the mark and not everything that you want to achieve to get the correct and accurate representation of the information, are straightforward within Power BI. There's probably custom visuals AND heavy customization you could do, but one of my core principles when I build anything with any software is, that we humans should be lazy. But if I am ending up clicking 300 times to format a bar chart, then I'm like, "What the heck? This should be simple." Rob Collie (00:38:46): Yeah. It is very clicky with the formatting. Chandoo (00:38:53): Yeah. I mean, there is Format Painter, but I feel like even after all the formatting, it will not get you nowhere near as good as a visual that you could produce in R or Excel, or any other tool for that matter. This is simply because I think they went in a different direction, maybe deliberately to enable that sort of interact to things. So everything needs to interact, or hence not everything that you could do in other tools is possible. But it's a visual software, the whole output of whatever you create in Power BI. You might build an amazing model and beautiful measures, but nothing is visible until you put a visual there. So the visuals need to be the hero of that platform, but I feel like the focus has been heavily on the data and modeling side of things. You need those, I guess, but now that they're stable, I wish Microsoft would put in more effort into the visual space and try to make them right and make them easy for the audience to build and work on them. Rob Collie (00:39:53): If you're interested in providing feedback, I can certainly connect you with the people that would like to hear it. Chandoo (00:40:00): I think. Rob Collie (00:40:02): It is very difficult. So, it's funny, the job that you worked at the consulting firm, you're the business analyst, that's exactly the job I had at Microsoft, which is trying to absorb what the customers need. And what they want and what they need aren't necessarily the same thing. Try to absorb all of that and then translate it to the tech crew to implement, while at the same time trying to simplify everything. That's exactly...So you were doing that for custom line of business software projects, probably, for the consulting firm, and I was doing it for things like Excel, but it's the same job. Chandoo (00:40:34): Yeah. Rob Collie (00:40:34): And for the people at Microsoft who have this job, doing that for Power BI, it's actually really hard sometimes to see the forest for the trees. You're so down in the details, it is a gift for someone in that role to be given any sort of thoughtful, structured feedback, or thoughtful, structured advice. Like on the visual layer, I would not be one that you would want to take that kind of structured advice, it's not really my forte, different beast, the Chandoo. Rob Collie (00:41:10): Okay. I was going to make this joke, which is that you're doing it wrong. If you have that kind of perspective where you reached the tipping point where the financial incentive isn't the primary driver, in my experience, from watching a bunch of Microsoft executives anyway, that's when you need to tell yourself that it isn't enough. And you need to just pick a taller hill and go climb that, and never be complete, never be fulfilled. And there are so many people like that. I haven't reached that point in my life that you're describing. That's something I strive for. I think that I'll be more like you and less like some of the people that I saw at Microsoft, who had everything, and still wrecked themselves after having everything. And it was really sad to watch it. I think a lot of celebrities in business are driven by this perpetual insecurity, that you fortunately, you're not driven by that. Chandoo (00:42:07): Yeah. I think, again, it's not portraying myself as I have no insecurities or I don't feel inadequate in any which way, it's just that at least I am aware from time to time, and I take a point... Like if I feel anxious for some reason and feel myself like I'm running towards this or that from time to time, I try to at least pull myself back and take a stop and at least try to admire what is already there, what is available and what we have achieved. And that lets me calm down a bit. Obviously there is no value in running for itself, but you don't want to be standing still and just admire the beauty. Also, there is some amount of effort you need to put in because that will make you feel fulfilled, having some fulfillment in your day, but it need not be just the amount of money that you are generating on an ongoing basis alone. Chandoo (00:43:00): At least that's my value. They might derive satisfaction just by running and chasing more money, and that's what makes them happy, they can do it. So you remember the time when you were not there or you were there, but we all went to Chicago from Cleveland when I was in US? And Jocelyn and I, we were driving in one car. So we rented this car, and I think you were driving in another car or something. And we went to, was it Jocelyn's sister or was it- Rob Collie (00:43:26): Yeah. Chandoo (00:43:26): ... your sister? Okay. Yeah. So we were driving in the car and Jocelyn was telling me all about her life story and how she met you and all of that, how both of you met each other while working at Microsoft and some of the hard times that she had and all of that, it was a very deep talk because Chicago is not nearby. So it was like a good four or five hours drive if I remember correctly. The topic turned into money topic as well. And Jocelyn was saying about few different things and this and that. And the topic turned on me, and I remember canvasing to her that I find it really hard to spend money because I grew up in a very poor family. I mean, it's not probably the poorest family by Indian standards, but it is still poor family. And there were times when I was growing up, when we would not know exactly where our next meal would come or how we are going to pay for school fees. Chandoo (00:44:17): And there were points of time where I had to pull out of school because we couldn't afford school fees and all sorts of that. There was a lot of hardship. As a kid I never really thought of that as hardship, it was just the experience. So you're growing up, but there was a lot of uncertainty, and that makes you who you are. As I grew up and as I started making money, that insecurity that if I don't have money, then I will struggle. Not only me, but whoever is dependent on me will also struggle. So that made me an obsessive saver where I will try to save everything for tomorrow rather than be in the moment and enjoy what I have today. And even when I have big money and I have lots more to spend, I would be always like, "I don't need anything. I'm happy with what I have. I'll just put it off for tomorrow." Chandoo (00:45:07): So I was telling Jocelyn that I find it really hard to spend money with the amount of money that I make. I still try to just spend maybe 10 or 20% of what I earn and everything else is going towards the saving or investment or whatever. So you could say maybe I'm chasing that instead of chasing money, I'm trying to chase for some better tomorrow. I mean, I do realize that there is no better tomorrow, today's as good as it gets. So you need to take a moment, chill out and enjoy. But I think having that awareness is more important than just chasing. If you know why you are chasing something, then you will enjoy it. Rob Collie (00:45:42): Agreed. The other part of that story also resonated with me, which is you had a little time to recharge your batteries, pursue some other things. And then you come back around and you say, "Hey, this Power BI thing, that is a worthy thing to explore, that is a worthy development path for myself." It's almost like the opportunity to, like your favorite movie, you would love to be able to watch it again for the first time, experience it a new. Now, Power BI isn't like Excel, it's not the same thing, it's similar in some ways, but it's the closest you're ever going to get to being able to climb the Excel hill again, is to climb the Power BI hill. And in the end, you end up with this same sort of polished, interactive output, a symphony being played over some data. And for whatever reason, sickos like me and you, that speaks to us. Chandoo (00:46:43): Yeah, we enjoy it. And it is a very good challenging environment for you to learn and master and talk about it. It's a different experience altogether to do things in Power BI, because despite all it's visual, that's what the software is for. Unlike Excel, there is no area where you're building the calculations, everything is in this black box. Well, technically not a black box, you can still see the measures and all that, but a lot goes behind scenes than what is out there. So explaining that, and because I try to view everything from the explanation I write, because my job, I feel like is to do something and then also explain it. So every time I build something, I'm like, "Okay, how am I going to explain this?" Because I don't want people to be like, "Ta- da, this is showing up now." So it needs to be having that steps as well. So I try to think in that direction, and that is an interesting challenge in itself to take something like that and make it more reachable to the audience, I guess. Rob Collie (00:47:44): Just thinking about that, I think about you're going through that and doing that, you're creating videos, right? Chandoo (00:47:49): Yeah. Rob Collie (00:47:50): So I've got to thank you, you taught me Camtasia. Chandoo (00:47:55): Oh, well. Rob Collie (00:47:55): Yeah. And not just like, "Oh, here are where the buttons are," you taught a bit of the art of it. Chandoo (00:48:02): Oh, well, I really appreciate it. And I think, I feel like I have learned more Camtasia in the last year than all of my life together. This might surprise you lik, "What the heck are you talking? You are using Camtasia all the way back in 2013 as well." This is because about a year and half ago, I decided to switch from blog first to YouTube first. So now all my content is primarily produced for YouTube. And if needed, I will put a blog article, but sometimes I'll just link to an older article because there is a lot of content already. And I feel like there is no extra value in writing another article just for the sake of maintaining a YouTube video. So primarily all the content that I'm creating is for YouTube. And the YouTube presents a different challenge. If I'm creating a course, people are hooked on it, they paid for it, they logged in, they're setting time to learn, so they will watch me go through all the steps for 15 minutes to understand. Chandoo (00:48:57): But on YouTube, it's a different game altogether. The audience have many other distractions. There is also the aspect of how much time they can set aside in their day. Many times people are not really deliberately sitting down, "Okay, I'm going to have a YouTube sesh now." Instead they're doing something, and then suddenly they'll go onto YouTube to see quickly how to do certain things, or maybe they're having their tea break or lunch and they just want to watch a video. So that time span is very limited, and we want to address something valuable, provide good content and share something fun with them. So the videos need to be shorter, but they still need to be just as useful, fun and engaging. So I'm learning more on Camtasia in the last one year, like how do you combine various things, how do you add more effects, how do you present your story, how do you view this together. But yeah, it's good. Rob Collie (00:49:52): Tom's not here today, but one of his pet peeves is the cliche you hear over and over again, "There's more data created in the last year than in the entire human history before that." Well, here's another example of that, "Chandoo has learned more about Camtasia in the last year than he has in all of human history before." And when you said that you've learned more in the last year about Camtasia, my jaw did in fact drop. I'm like, "Oh my God, I need to come see this." Basically, everything I know about video editing in Camtasia, I learned from you, and in a very short period of time, so I need another bootcamp. Chandoo (00:50:29): You might have taken those and you might have gone really well past that point. Obviously that's really what happens with technology tools, the software evolves, we use it day in, day out. Then we realize, "Oh, we could do this. We could do that". Yeah, maybe watch some of my YouTube videos and let me know how that is, if you enjoy not just the video, but also the editing. Rob Collie (00:50:51): When you're watching something that's well done, you don't really notice. Chandoo (00:50:55): Yeah, obviously that's the whole point, right? Rob Collie (00:50:58): Right, the techniques. But then it was different essentially sitting at the editing console with you and you going, "Okay, so here I would probably do something like this." And then I'm like, "Oh, I would have never thought to do that. That's that's awesome." Certain pieces of software, certain tool sets are ones that I tend to evolve my skills over time on my own. I'm not really making videos these days. Maybe I'll be evolving otherwise. I would say that my Camtasia skills are basically frozen in 2013 where you taught me. Chandoo (00:51:30): Well, that's a nice compliment. And yeah, I think if you're not making videos, there's almost no value in learning the skills, because it just keeps changing and they have newer version now coming up every year. So sometimes you learn something, and the next year, boom, there's another way of doing it. And then we're like, "Why did they even bother learning this in the first place?" Rob Collie (00:51:53): The people at our company that play in our fantasy football league, and who've been subjected to my fantasy football gloating videos, they owe the production quality of those to you. I can't credit you for the singing quality, the vocals in those videos are terrible. And there's nothing you could do, even Chandoo couldn't correct my singing. And no, those videos are not available for public consumption. We are not going to- Chandoo (00:52:19): Maybe you should probably- Rob Collie (00:52:19): ... unlisted for a reason Chandoo (00:52:20): ... do that as the next episode of Raw Data, we're all singing. Rob Collie (00:52:25): On the previous episode, we talked about rewriting an AC/DC song, Dirty Reads Done Dirt Cheap. AC/DC really lends itself to alternate vocals. It wouldn't be the first time I've rewritten an AC/DC song, but then someone's got to get on the mic, things get ugly. Well, I'm one of those artists, when I write the alternate lyrics, I can't let someone else sing it for me. I've got to go do it myself, and again, it's sad. It's kind of neat. I mean, on one hand you could say that you were early to the internet. I'm going to use the word celebrity because I don't think really, any other word is better, and celebrity is not the perfect word, but one of the early adopters, one of the first movers in that space. Of course at the same time, that's years later than Bill Jelen. Chandoo (00:53:13): Yeah. Rob Collie (00:53:14): Which is crazy, right? I mean, it's like... Chandoo (00:53:17): I mean, imagine how much vision or... I don't want to say random and [inaudible 00:53:23] all his effort. It's completely his vision to have that started and even have a publishing company and all of that empire built. Rob Collie (00:53:32): Amazing, yeah. And as you say, he's been on the show and he has, absolutely it was not deliberate, it was still not a called shot. Chandoo (00:53:42): Yeah, but even if it's not deliberate, I think the biggest quality with some of these people like Bill, they have is, they listen, they see what's happening, they get the feedback, they tap into their emotions, they take a deliberate action from time to time. He could have started MrExcel forum and left it there, but he realized, "Okay, people are getting help from this. I need to... Chandoo (00:54:03): And left it there. But he realized, okay, people are getting help from this, I need to work on this, improve it better for them and people are buying these over priced Excel books that are sometimes way too detailed or way too complicated. I need to change the market. So, those are deliberate actions. You couldn't say one day he woke up and suddenly found a printing press in his house or anything. Rob Collie (00:54:21): Yeah. Agreed. So, what has it been like, having been early to the Excel internet celebrity phenomenon, but then joining the Power BI game... Not late, but very much in progress. Just like me, when I was first blogging about Power Pivot, I basically didn't have competition. I was the only weirdo obsessed with this stuff and writing about it like violently almost. I couldn't help myself. Whereas if I started that today, I would be joining a field that is very crowded by comparison. How has that been different? And I know that it's a different point in your life. So of course, it's going to be different anyway, but what have you noticed that's different about those two different journeys? Chandoo (00:55:10): I didn't really notice any difference, this is because the audience that I have been cultivating over time, they have also gone to a point in life where they are naturally migrating to Power BI and they already trust me, they know me, they have joined the courses or they have learned from me previously. So for them, it's easy to relate to the content that I produce because, it's like same teacher is teaching you 101 and then 102 class kind of thing. So, it's easy for them to relate. So, I had the ready audience either by luck or by that... Rob Collie (00:55:47): Cultivation. Chandoo (00:55:48): Yeah. So, it wasn't really like a fresh start. Like I would go and put, learnpowerbi.com as a website and put there. I'm already putting it on my website, so it's easy for people to connect the dots. But what I did notice is that audience, especially because Power BI is like an evolving platform and people have been using it way before even I started writing or we making videos about it, some of the people have already shifted away to those channels or those platforms to learn more. So, they are kind of tuning me out for Power BI because they're thinking Chandoo will teach us Excel, these other people will teach me about Power BI. So, the engagement or the feedback that I would get on Power BI related stuff is significantly lower than the Excel stuff that I would produce. So, I could clearly see that happening both on the YouTube channel as well as on my website. This is the reason why I got into self-doubt at some point thinking, should I even bother making a course about it, because it's a big investment of time on my side. Chandoo (00:56:55): And if I'm not benefiting a lot of people, then it would be just a futile exercise of me recording videos, producing everything, marketing it, and just simply annoying people if they're not ready to buy or whatever. But then when I launched the course, to my surprise, people were willing to pay and join. And that was the good, positive feedback for me. So, I went and I did that a few more times. So, it is good experience for me. All in all, I'd say it's a very positive experience. Last month on my YouTube channel, what I've been doing is, last Friday of every month, I do a live stream. So, Power BI is one of the most requested topics for live stream and the live stream that I did on Power BI, which was in June, was a massive success. Like we had quite a few people show up and go through the thing. And even on replay... This is a live stream, right? We are talking. There is lots of valuable content, but there is also a lot of content. I'm not going to call it. Rob Collie (00:57:52): There's valuable content and then there's content. Chandoo (00:57:55): So, there's a lot of stuff where I would just randomly read comments and flash them on the screen to say what people are asking or muse about things and all of that. And even on replays, people are watching all of that. So, this is good indicator that now there is more. And every time I ask a question on my community like, "What do you want to see next?" Power BI was the highest asked item. So, there's more people asking for that and I believe this is simply because people explain, they like my style, define me to be their teacher. So, they want me to teach it. And I think that is a good indicator for me. I will be creating more Power BI focused videos in the rest of this year and get more into Power BI. Not to say I'll ditch Excel. I'll keep using Excel because, Excel has continued to be the big platform that is used by millions of people all over the world. And I would love to be of help to them. Rob Collie (00:58:49): I think Excel is also experiencing a sort of Renaissance. Chandoo (00:58:53): Yeah. Rob Collie (00:58:54): The re-imagining what all it is that can happen in Excel. Some of the fundamentals of Excel are not being changed. They're being expanded in ways that we really haven't seen, maybe ever. There's a lot of fresh opportunity, a lot of fresh topics to talk about in Excel. A lot of things to dive into. Chandoo (00:59:14): Exactly. Especially the way they are expanding the formal language into more dynamic world and probably the terrible name, but the Lambda functions and all of that. Rob Collie (00:59:27): On the podcast with Brian Jones of Excel, I told him multiple times, "You're going to rename this at some point. You're going to rename it." Chandoo (00:59:38): The moment you see Lambda, you'll be like, "This is like another bot text." Nobody's going to even type that into Excel. Like, "What is Lambda?" Rob Collie (00:59:49): Yeah. I told him my favorite thing about the Lambda functions is that you hear the name and you immediately know what they do. Chandoo (00:59:56): Yeah. Rob Collie (00:59:58): So, are you getting into Lambda functions? Chandoo (01:00:00): I don't want to use the beta version. This is just by choice. I don't have access to Lambda function yet. I'm itching to play. I could just enable it with a click. I know that, but I don't want to make them. Simply because I don't want to ruin my Excel by changing the user experience from time to time. And I don't have to compete with them. I couldn't be really bothered to do that. But I know what they're capable of. I watch other people do it on YouTube and I did help play with them on my personal laptop the other day. It is a very good addition. I feel like this is not to again, go and say negative things about the amazing work this Excel team is doing. There is a lot of energy put into the more abstract way of doing things. I would say Lambda and Map and Reduce are at a very high level. Chandoo (01:00:46): And even I have done a lot of programming and I believe you may have already done some programming too. Even for us, it would be a hard concept to understand such a very generic version of things. And then actually capitalize on that raw power that you are getting now. But what would really help end users is, at least the way I hear when I talk to people or trying them is, some of the more things that should be done readily. Just to give one simple example, the other day I was training some people in Australia and they were asking, "How do I remove the spaces within the text?" So you have two words, but there's some extra spaces in the middle. And then I said, "Oh, you could use trim." And then they're like, "Trim? What is that?" Because when you hear the word trim, unless you have a very good background in the language or the history of computers, you wouldn't really guess that- Rob Collie (01:01:39): Right. Chandoo (01:01:39): ... this is the one that removes spaces. And then she immediately said, "Why doesn't it say remove spaces?" Rob Collie (01:01:46): Yeah. Chandoo (01:01:46): This is the usability that I'm talking about. We could add more synonym functions or if you go on internet and search, one of the common things that people ask with VLOOKUP is, "How do I VLOOKUP the second value or how do I get to everything with VLOOKUP?" And Excel still doesn't have a function. And they say, "Oh, you can use filter", or you can use this or that, but why not take the VLOOKUP and make, when now there is XLOOKUP also, but they had the opportunity to take the XLOOKUP and also make it more like XLOOKUP filter. So, I feel like some of that energy also needs to go into these mainstream things. Might sound like ranting here. But... Rob Collie (01:02:26): No. This is important. I share these beliefs. I think you're a bit more sophisticated in your beliefs all along these lines, where I'm a bit more intuitive, emotional about them. You can refine them to very specific points very quickly and effortlessly. I'm going to ask you a wild question out of the blue. If Microsoft came to you one day and offered you a job, let's ignore the money for a moment. How much they were paying you, whatever and you didn't have to move. Would you accept job on the Microsoft product teams? Chandoo (01:02:58): I might accept. In fact, this is not something that I told many people, but a while ago I did actually put my hand up for a job, because I saw one in the MVP group, we get some emails from product managers. The email content was, they're looking for a person who is at the intersection of Excel, Power BI and the data visualization. I said, "Yeah. I'm not really looking for a job or anything. I don't really have the energy to do a full-time job. But if you are happy to take somebody remote and if you're willing to take someone part-time for a couple of days a week, I might be willing to do this, because I believe I can contribute in this space." But I think they were actually looking for a specific role within a specific city in US. So, it didn't happen. Chandoo (01:03:45): I also questioned like, it's easy for an outsider to make noise and complain and bitch about things. But when you are there, you will then suddenly come across these 75 constraints on every little thing that I want to do and there's a lot of internal drama and politics and whatnot goes on in these organizations, right? So, there might be genuinely people trying hard, but get just pushed aside, because there're other priorities or paying customers are asking you to do this or that. So, I wouldn't really know for sure. Rob Collie (01:04:16): Well, I do. I've had that job and you are correct that very often, some of the things that seem very frustrating on the outside. Why the hell? But on the inside, there's a really good reason. Chandoo (01:04:31): Yeah. Rob Collie (01:04:31): It wouldn't even help the world to hear it really. It's too mundane, it's really boring. So, you're never going to hear that reason on the outside. But the thing is, it's also that clarity is very hard to come by. When you're in that job, almost by definition, this isn't always true. They've been hiring people over time that came from the user ranks, the customer, that our number of former intense customers, who do now work for them and the clarity that someone like you would bring, would be absolutely worth it in a big way and incredibly valuable. Even with your very mature caveat, that some things are a lot harder than they look. I agree. There's always particular logistics about every particular position or whatever. Just take this at face value, you would definitely be an asset to either of those teams. Not that you need it. I'm not saying, "Oh, Chandoo, you're looking for you're... You're wandering, you're lost. Let me help you find yourself." No. Not at all. You might decide after a taste of that. You'd be like, "Nah." Chandoo (01:05:43): If I didn't know any better, I would say this podcast is like an interview, like a software... Rob Collie (01:05:48): Yeah. I'm going to open the door behind me here. And they're going to see the whole team is there. Chandoo (01:05:53): Welcome to Microsoft. Rob Collie (01:05:55): They don't pay me anymore, Chandoo. Why would I do their work? Chandoo (01:06:00): Oh wow. Rob Collie (01:06:01): So, that's cool that you're reasonably significant subset of your Excel audience, has joined you and it's called Power BI play date? Chandoo (01:06:13): Yep. Rob Collie (01:06:13): That is such a cool name. Someone could try to now imitate that, they come up with a class and call it Power BI playroom. And it just wouldn't be the same. You mentioned this earlier, but the fateful summer of 2013, you and your entire family, all four of you came over from India and moved in, in Cleveland. I know a couple of miles from where we lived. We were in Cleveland until 2015. So, that was when we were there and we got to hang out for... Not an entire summer. Chandoo (01:06:48): Pretty much everyday. Rob Collie (01:06:48): Pretty much every day, up until the point where I destroyed my knee. Chandoo (01:06:52): Yeah. Rob Collie (01:06:55): We had just taught a class together. You and I in Columbus, Ohio. I think I was teaching on my birthday and teaching on Jocelyn's birthday, June 22nd. And she wanted to go to the trampoline park in Columbus, after we were done teaching. I also met one of the students at that class, that joint class that we did together, was Mike Miskol of command and his right-hand man at the time, Donovan. Command ended up being one of the ground floor clients that launched our company. We ended up doing a lot of consulting work for them over the years. And, we've talked about Mike on the blog multiple times. Sadly, he was the wolf. He passed away a few years ago, but that was a heck of a summer. A lot of things happened that summer. We didn't make it back from that class in Columbus in one piece. Rob Collie (01:07:50): I remember we all went to the drive-in movies outside of Cleveland one time, after I'd hurt myself. Chandoo (01:07:57): Yeah. Rob Collie (01:07:57): And I was sitting in the back seat of the Jeep with my leg out, across the bench. And I'm watching this drive-in movie through the crack between the headrest and the window. It was a Despicable Me two or something like that we watched. Chandoo (01:08:14): Yeah. Rob Collie (01:08:15): Your kids were little. Chandoo (01:08:16): Yeah. Rob Collie (01:08:17): I guess mine were too. Your children, I remember describing them as luminous. They just glowed. It was like a light meter up to them and like, "Oh! They're emitting light." Such cool young people. We missed y'all when you left. Chandoo (01:08:38): Yeah. If I look back and think about some of the best experiences in my life, that will be definitely one of them, because there was so much fun that we had and it was, because I'm not really working and Joey's also not working. We are working in our business, but we are not really physically going somewhere. So, we had all the freedom and the weather was really good. And, you guys were just around the corner. So, if we need anything or we felt lost, we could always call you or just walk to your house and you'd be there to help us. And it was such a really fun time. And when you're having things like that, we were enjoying, I always thought, we would repeat that a couple of years down the line. We were talking about these kinds of things. Chandoo (01:09:23): I remember leaving a small box of some utensils and stuff like that in your basement when we left, because we were thinking we'll come back another time in a couple of years and we will repeat it. Rob Collie (01:09:33): Yeah. Chandoo (01:09:33): But, life has different plans every time. We went back from that and things kept moving in different directions. You started your business and I had this vision that we should probably move and live in other country. So, we started looking for that and we moved to New Zealand. And even when we were in New Zealand, we would have some video calls where you'd say, "We will come and visit you for a holiday or something." And look now where we are. We can't even leave our countries if we want to. Rob Collie (01:10:01): I know. They'd let me leave, but your country wouldn't let me in. Chandoo (01:10:05): Yeah, exactly. Rob Collie (01:10:07): I think you chose well. See, there you go. That foresight again. You're like, "Okay. There's going to be a pandemic in a few years. And New Zealand really seems to have their act together. And the US looks... I don't think so. You didn't see that coming, but good choice. Obviously, I was a little bummed when you didn't move to the US but now, I can't help but look at it and go, "Oh, good move." Chandoo (01:10:38): Yeah. Again, nothing was planned. We just were thinking we would go to either Australia or New Zealand and we just kind of flipped a coin and then it was New Zealand. So, that's where we applied and we got in and we're happy we are here. But again, it was not deliberate at all. Rob Collie (01:10:55): The future holds many possibilities. We need to come visit. You live in one of the coolest places. They chose to film Lord of the Rings there. That's how cool. Chandoo (01:11:07): It's an amazing place and there is so much natural beauty and people are just nice. Rob Collie (01:11:12): Do you mind telling the story of the now absolutely iconic chandoo.org avatar? Can you tell me how that came to pass? Because when you got off the airplane in Cleveland to come visit. Chandoo (01:11:30): It's the other way. Rob Collie (01:11:30): Really funny. You didn't look like that. Chandoo (01:11:34): This is like... Probably there were many crazy accidents, all this journey, but this has got to be one of the craziest accidents because, I had my website chandoo.org way before I got into Excel, the original website is called... Not that it matters anymore, but it's called Modus Indoramus and it's nothing to do with anything that I'm doing nowadays. But, I thought in those days, that's a good name. So, I went with that and it used to be hosted on Blog spot. And later on, when I finished my management degree and I started working, I needed to rename that to reflect my new stage of life. So, I went with the name, Pointy Head Dilbert. This is simply because, I like Dilbert cartoon and I find that Dilbert is this technical guy. And the point, he had boss is this, or lack of better word, a dumb ass who is like a manager. Chandoo (01:12:26): And I find myself in the junction of these two. I got my technical degree and now I'm a management degree holder. So I thought, "Oh. Let's fuse these two things together." And I come up with this brilliant name... Well, it's not brilliant, but I thought it is brilliant. Pointy Head Dilbert. And there was a point where the logo of the blog was actually Dilbert with the point he had boss's ears on his head. Rob Collie (01:12:48): Okay. Chandoo (01:12:48): So, I kind of photoshopped it. That was the logo. Fast forward to 2008 and we moved to US for work Joey and I and around that time, I was fascinated with MacBooks and all. I got my first computer, which is a MacBook and the MacBook got delivered home and it had this photo booth app, using which you could kind of take a selfie, but it was cute, your facial features. So, one day morning I got up and then I was doing my usual routine of checking mail and stuff like that and I opened the photo booth and I took some selfies and one of that was this iconic hair picture. So, it's really just my selfie in the Mac. And then later on during the day or the next day, I thought, "You know what? I could use this because it has pointy hair", because the hair is kind of really stretched out. And I opened Photoshop and I cropped that image and polished it a bit, a change of the saturation and whatnot, and replaced that on my blog. Chandoo (01:13:49): And people were like, "Oh wow. This looks amazing. We love it." Yeah. Later on, I renamed the whole thing as chandu.org instead of Pointy Head Dilbert. But, that is how it came up. The sad thing is, I don't have the original picture anymore. The MacBook died. So, I have no access to the original picture, not even higher version of it. There is nothing there. The only thing that is on my website is the only one that I have. And I thought at some point maybe I could get someone to do a vector drawing of this, but I never got into any sort of merchandising or anything like that. So, there was never really a need for a higher resolution version of this. I'm not printing coffee mugs or anything. But, I am really glad that I had a picture like that and I could use it. I am also glad that I still have some hair on my head after all these years. Rob Collie (01:14:39): Your hair was much shorter, that summer we spent together. I can see now with your current hair, how that picture might've happened. Did you also distort the hair a little bit? Chandoo (01:14:50): No. It's the photo booth app. Rob Collie (01:14:52): Your hair was actually that tall, that morning? Chandoo (01:14:54): It was tall, but what the photo booth does is I think if you have an iPad or iPhone, you can actually test it. It will basically take a central point. I think the point was somewhere here on my head and it'll stretch every pixel out. Rob Collie (01:15:07): I see. Chandoo (01:15:08): So, that's how the hair kind of became too wild. But I did have fairly long hair at that point and I think because I just got up, it was all over the place. Rob Collie (01:15:17): Well, what a happy accident. Because again, so iconic. I didn't know you as Chandoo, for a long time. All I knew was there was this guy on Twitter, R1C1, who simultaneously had this really quirky sense of humor and at the same time, like this mastery feel. And it was very much enhanced by that icon, by that logo. I would see that icon and I go, this is someone who lives at the top of some pyramid. You need to climb in order to have an audience with him. It's amazing how much power that had, when I didn't even know that you were Chandoo. You were just R1C1, this bad-ass, who was also really funny. It was only eventually, one day I clicked through him like, "Oh." Yeah, I expected your website to be like pyramid living bad-ass dot com. Chandoo.org was a much friendlier name. Again, you were very friendly on Twitter. It's just that I expected some sort of Kung Fu master. Chandoo (01:16:29): That was also not intentional. I think I'm just being... Rob Collie (01:16:32): You're just being funny. Right? Chandoo (01:16:33): Yeah. Exactly. Rob Collie (01:16:34): I think it's great. So, going back to Power BI for a moment, what are some of the things about power BI that you have found surprisingly delightful? What are some of the things that you've just really gone, "Wow. That is a cool thing that we can do in that environment that we couldn't do in Excel." Chandoo (01:16:54): This is no longer true, but what I found Power BI to be amazing in is, it has this massive power query layer in the front, that lets you take anything, manipulate it in any which way you want and get the clean cut of data for your analysis. So, that's one thing that I found to be a true game changer. Because, many times when we want to analyze data, when we want to present things, we have this like an 800 meter hurdle to cross, where our data is really shit. The Power Query was like a true game changer. I mean, in a way it was coexisting in both Excel and Power BI worlds, but many times within Excel, you already begin with the data that is pasted into the spreadsheet. So, you're not really deliberately trying to clean it up more. It kind of comes in a semi clean format, whereas Power BI, because there is no holding cell where the data can be pasted. Chandoo (01:17:52): You can kind of technically copy paste, but there's no place where the data is. It has to come from outside, because of the nature of platform. Having Power Query was like one of the big key things. And it is a mental model shift also. We don't really begin in that space. Now, we are beginning in that space where I could potentially get anything and manage it and manipulate it. And the second thing is obviously the power pivot engine, which is really dynamic and amazingly powerful. You could write a simple measure that is just doing a sum or count or whatever, but then the way you present, it really changes the meaning of it and completely presents a different insight, that you would have spent hours and days and sometimes weeks or just might even give up to try to replicate in Excel. Chandoo (01:18:40): But those two are no longer true because of the coexistence of the same ideas in Excel world, but they feel more natural within Power BI. Whereas in Excel, if you can't do something with Power Pivot, you always would be like, "Okay. I'll get as far as I need. And then I'll use the cell structure and the relative references and whatnot to calculate the rest. Whereas power BI doesn't offer that. Everything has to be done with these two preset engines. So, you will be forced to go in and achieve more with them. And the more you get it right at those two stages, everything else becomes like a piece of cake really when it comes to visualizing and analyzing, There are other aspects too, and we kind of grown a customer trait now. So we don't no longer find that amazing to be honest, but the first time you see it, you will always be like, wow, this is good. Chandoo (01:19:30): That kind of cross filtering and interaction is a game changer. The ability to have tool tips that are rich and informative is a good thing. That idea of using bookmarks to change the state or the display of the visual, so that you don't have to put multiple things and still get what you want. Or even some of the simple things like having a separate mobile view, where I can show different visuals or structure them differently, depending on what the audience is, looking at it from. All of those are some of those fundamental things that really, for me, at least helped that platform elevate to a different level from what is previously possible in Excel. There's many other things like even some of the simplest things like the ability to show pictures from a URL, it is nearly impossible to do that in Excel. Whereas in Power BI just happens with the simple switch of the setting. And now suddenly you have something that is so much more beautiful than a simple table. Rob Collie (01:20:31): Yeah. I mean that composability of Excel, the network effect of all of its features, sort of interacting in the grid, leads to an environment in which, like you mentioned in the very beginning with that first blog post, you felt like you'd either invented or discovered something, that in your own silo, felt like the first light bulb, like first time anyone ever discovered fire. I know how that feels for sure. Big time. You're wiser now, you know that when you discover or invent some- Rob Collie (01:21:03): You're wiser now. You know that when you discover or invent something, you're probably not the first. As I am also now wiser. But there's still that moment where for yourself, you invented something using things that were not necessarily ... no one ever anticipated that particular usage. Rob Collie (01:21:18): So let me give you an example from my life in Power BI, is I quote-unquote, Invented an American football passing chart that shows where the quarterback likes to throw the ball and where he's successful at it and all that kind of stuff, right? It's just a Power BI 2D scatterplot. Chandoo (01:21:38): Yeah. Rob Collie (01:21:39): With a football field image behind it. Chandoo (01:21:42): Mm-hmm (affirmative). Yeah. Rob Collie (01:21:42): That's it. It's just the background image. Now, but then I had to go to your blog accidentally one time and read about jittering. Chandoo (01:21:50): Yeah. Rob Collie (01:21:51): The concept of jittering. And I remember you had a really great article on jittering in Excel. Chandoo (01:21:55): Yeah. Rob Collie (01:21:55): And I was like, oh, I need that in my passing chart because all my dots are going to be on top of each other. Chandoo (01:22:00): Yeah. Rob Collie (01:22:01): And then I did a really, really, really complicated, it was really labor intensive, to achieve jittering in my chart, took a combination of M and DAX that I pity the fool that has to go back and understand what I did. I wrote a long article about it. But if you just stumbled upon my workbook and tried to figure out how I did it, oh God. And so here I am, I have invented a pass chart. You know, we forget about it. We set it aside. And then years later, a football coach from Texas, a high school in Texas, calls us up and says, "Hey, do you know that that football thing you invented is probably the best football passing chart that's available on the internet?" Rob Collie (01:22:37): I'm like, "No". And so we went and did this thing called Cover Hawk. If you go to coverhawk.app, it's very, very, very much like an MVP exploratory thing where we're trying to see if there's some traction to this. Maybe there's a product worth developing. We even went to a conference two weeks ago in Texas. Conference of, nothing but football coaches, nothing but high school football coaches and we're just standing there like the data nerds. Two booths down from us is a bunch of tackling dummies. They're trying to sell tackling dummies and helmets and things like that and we're there as the data nerds with Cover Hawk. Rob Collie (01:23:15): So have you had any of those that you're comfortable sharing? Have you invented anything? Chandoo (01:23:20): I don't want to call, there've been mentions, because I feel like, like you said, we're all wiser now. Rob Collie (01:23:28): Right. Invented in air quotes. Chandoo (01:23:31): Yes. So there are, you could call them as hacks because the way Power BI is evolving, certain things are not possible at certain points of time. They may not be true anymore. Like now, you could do it. But at that point in time, that feature is not available or it's not as refined. Chandoo (01:23:47): If people are finding it useful, you might invent something fun, but if nobody has the use for it, then it's just a novelty, right? Like the way you did, the football coach said it is a useful thing. Based on the YouTube videos that I have on Power BI and the comments or the views that I get, one video that I have is a budget versus actual with variance information as a combined graph. So the idea is in the world of finance, we do a lot of comparisons with what's my actual, what's my budget for a bunch of projects or products or whatever. Chandoo (01:24:19): And many times you may also want to, okay, 100 is my budget, 120 is my actual, so we got 20 variance or 20% variance. And I want to see the variance information right there, but because Power BI yet doesn't have any customizable data labels, so you get both 100 and 120, but you can't really get 20 or 20% in there. And even if you could get it somehow, it's not visually represented so it's hard for people to see that and instantly make a mental picture of what that 20 is, which direction it is going. Chandoo (01:24:52): So I made a graph where it's nothing but a column or a bar chart and another column or bar chart. And the sort order is also maintained on both depending on what interactions are happening and whatnot as a visual and I made a YouTube video about it. And from time to time, people tell me that, "Oh, this is a good one we've been looking." Like you know, there are custom visuals that can do this for you. But sometimes custom visuals, people try to shy away from them, just like Excel add ins in a way. Maybe because they're not compatible or there's maybe security issues or whatever. Chandoo (01:25:25): Having a native visual is useful. That's one thing. And there are other fun projects I do to just demo certain things. But I think this is one. There is another one that I had an article and I think even a YouTube video where we are using the what-if parameter option of Power BI to user input and then build a calculator that will do some projections on what if you save $500 a month that, I don't know, some sort of investment that will give you 6%, how much money it would create in future, right? So some sort of projections. And again, at that point, Power BI didn't have the functions like FV or future value calculation things. So you had to do it in an arithmetic way using the formulas. But now it is, I believe they have added some more finance functions now. Rob Collie (01:26:15): I remember that with PRODUCTX. Chandoo (01:26:17): Yeah. Rob Collie (01:26:17): They didn't have the PRODUCTX function. And so Amir told me, "Oh no, no problem. You just convert it to exponents". Like you do something with the E-function or something, right. Chandoo (01:26:29): Yeah. Rob Collie (01:26:30): And then you do a sum-x of that. And then you raise that answer to the power and now you're, and I'm like, "Oh God". And I actually did a few things like that. And then PRODUCTX came along and I'm like, "Oh, thank goodness". Yeah. I mean, because it's exactly that scenario, right? The future projection scenario. If your growth rate is the same every year, then you can use the power function, right? But as soon as there's variables happening at each year, it's all over, you need other functions. So you had to re-implement the FV function before it existed. Chandoo (01:27:04): Yeah. So there are some of those things that I had fun. And again, what I find tricky within the Power BI space especially is because the platform is shaky and it is always moving, the goal posts are changing. What you find it useful in a hack kind of a thing may not be relevant tomorrow or 10 months down the line. But yeah, certain ideas, and I think because I don't view them as innovations, but those are good things, good practice in a way I think, just try and showing people how to do and things will eventually ... that will be normalized and then the next things will happen. Rob Collie (01:27:40): Yeah. Well, I find those moments of hack, or invent, or work around, or whatever you want to call them where you create something that didn't quite seem possible when you got started, it was like on the fringe of possible. Those are some of the most satisfying achievements. Those feel really good. I think in a way it's like, that's almost what it's all about is that act of creation. Chandoo (01:28:04): Yep. Rob Collie (01:28:04): Even if you stayed a hundred percent within the known and traditional bounds of the product and you haven't invented some new technique, by the time you're done with a good report, a good dashboard, whatever noun you want to use and then the customer, in our case, the client, right? That's a piece of software they're going to now go use to revolutionize their business. You still get that feeling of creation. Chandoo (01:28:31): Yep. Rob Collie (01:28:31): And I think it's one of those addictions with no downside, it's a positive addiction. Chandoo (01:28:38): Yeah totally. And I feel like in that sense, the amount of satisfaction I derive within the Power BI space, more of it comes from Power Query and Power Pivot side of things, because that's where you are really tinkering with the language that is available to you. Whereas, the visual space is still hacky. You have all these bookmarks and selection pane and toll tips and stuff like that. That'll let you go in and make it to the next level. But because there is so much click action happening, you feel like the sequence of clicks is the real source there. Whereas in Power Query or another place, what you are describing, can we replicate it with a different situation, different data and different scenario, more easily. Chandoo (01:29:18): I think another example that people find useful, the ones that are blogged are using Power Query to look up the whole thing. But it's basically like a word search, but more, not simple words, but keyword search. So you got your X values in a column, and then you have your key phrases that you want to go and see in that and either remove them or delimit them based on any of those. So then there is no direct function. Maybe there is now, but when I was doing it, there was no direct function so I had to use the accumulate function or the list thing, which is basically like, it'll go through and it'll run some. It's a crazy concoction, but it works. Rob Collie (01:29:59): You want to see crazy concoctions, go look at my jittering, jittering techniques. I'm not very sophisticated. It's always like a street fight with me with this stuff. You know, it's like throwing dirt. Chandoo (01:30:10): Yeah. Rob Collie (01:30:11): It's not an Olympic fencing tournament with me. I find that very interesting that the two engines under the hood are the ones that have spoken to you the most. Coming in, I know that you can formula. You're R1C1 on Twitter. Like that's a reputation to live up to. But again, just sort of always thinking about you as the so incredibly differentiated on the visuals layer and Excel, I would have expected you to be just sort of fully enamored with the toolkit that they've given us, despite the fact that it's clicky. Chandoo (01:30:47): Yeah. Rob Collie (01:30:47): It's clicky to set it up. It's not clicky for the end user. It is very clicky for the developer. Chandoo (01:30:53): Yeah. I do create a lot of visuals in Power BI and I share them on the course as well as on my blog and YouTube. So there is quite a few visuals that I'm very proud of them and people do tell me that this is really good and well-composed. It's just that I find that you can't really get there unless you have a very good data and calc engine behind. So 70% of the time really goes there, right? The other 30% is really choosing the visual, putting it and coloring it. That's pretty much it. That's why I mentioned those because without that, you're really not able to deliver the visual. Rob Collie (01:31:29): And once I heard you say it, that's where I am. It makes total sense to me. Yeah, the visuals are a very, very, very important last mile. They're only the representation of the metrics coming out of your data model. They can't be better than that. Chandoo (01:31:46): Exactly. Rob Collie (01:31:47): And oftentimes, when you want to achieve a particular visual effect, you actually need to go do something in the data model to power it. For example, the dashboard we use to track the podcast statistics. I index all of those, every podcast episode to day one, the time sequence, the time access across the bottom of the chart. It doesn't make sense for that to be the calendar because then I just get all of these curves coming off from different places off of the zero. Chandoo (01:32:18): Yeah. Rob Collie (01:32:18): I need them to all start in the same place. So I can say, "Hey, look, this one's doing better at day five of it's life then this other one was doing at day five of its life". And that's really interesting. So it's this really cool spray chart where everything's spraying out like a sprinkler. Well, I had to do DAX. I had to do DAX to shift everything and the chart looks the way it does because of the DAX. And that interplay is again, one of those just like incredibly satisfying and gratifying things to do. Chandoo (01:32:46): Yeah, I totally agree. And then you can't really get there without the measure, uplifting work. I remember doing this massive gender pay gap dashboard for one of the ministries here in New Zealand. And people look at that and they'll go, "Wow, this is amazing!" But then I know that it takes me literally 10 minutes to make the visuals, but the measures behind that, the ones that drive the calculations and then tell them what actions they need to take. And that's where a lot of hard work is, right? But writing those measures was amazingly satisfactory, like learning how to use some of the, especially, variables to break down complex calculations. And I remember the times when there were no variable, so it would be pretty much donkey work to get things done. Rob Collie (01:33:34): Yeah. Back in my day, we didn't even have the sort by column feature. Our slicers had to have zero one dash January in them. Chandoo (01:33:45): Exactly. Rob Collie (01:33:45): So that we could get good sort order. What do you think the next 10 years looks like for Chandoo? Chandoo (01:33:53): Well, that's a very interesting question. And I keep thinking about it more now than back in 2010, 13 or 2016, to be honest, because I'm also trying to visualize what's going to happen, how I can be helpful to others and how I can share my craft and how I can feel that passion going forward. Because now that I'm more aware of what could happen if I don't carefully nurture it. So one thing that I'm trying to do is I'm trying to find myself in a situation where there is enough challenge, but not too challenged that it feels like stress and then you will give up halfway thinking, "Oh, this is not worth my effort, I don't need it" kind of thing. So a positive challenge that is happening for the last year and half is that shift to video-based content production versus blog articles. Chandoo (01:34:48): Again, purely because there is that unknown factor of how do I make videos? How do I make them engaging? How do I make them just as fun and quirky as a blog? How do I make them useful? So that is something that I'm finding interesting and learning. And hopefully that is the path that I want to be on for the next few years. I don't really know, 10 years, a really long time to commit to anything. But next few years, this is where I want to be, make more videos, change, or help people through the video platform while simultaneously maintaining blog and providing some support material there from time to time. If there is no global pandemic or anything like that, personally, we might be doing a lot more travel in 10 years because kids would be what, 21, 22 by then they might be finishing with their uni or starting their work. So they'll probably be not with us at that time. And we will find the same place as you are like an empty-nester. And I would imagine if we don't have anyone within our house, then we may want to go to India or parts of Europe, or maybe even visit the US again and spend some more time with some of the friends and family that we lost connection all these years. Chandoo (01:36:01): That's more on a personal side, but yeah, at gender.org, I think it will live. I'll keep blabbering about one thing or another. It could be Excel, it could be Power BI, it could be some other new technology or whatever. I enjoy tech. I feel like I would enjoy it for the rest of my life and I enjoy talking about it. So that's what I would do. I may not launch courses or sell stuff that far down the line because there is an expectation when you are selling stuff that you also care for your customers and provide service and all of that. It is fun to get money and provide service, but not as fun as producing content, to be honest for me. Rob Collie (01:36:38): Yeah. Chandoo (01:36:38): You could outsource that and all that, but because I'm not trying to grow an empire here, I feel like it might be an unnecessary thing for me to take up that challenge. And there is also a thought that I keep toying with, which is to probably get into full-time teaching or something like that. Wherein I might go and try and get a PhD or start teaching at a university or something like that. I don't really know. I mean, from outside, it looks like a romantic thing, but then the moment I try to read up more about it. I'm like, "Ah, I couldn't be bothered". Rob Collie (01:37:11): Yeah, it's tricky. I'm just sort of imagining this world in which you became a full-time teacher and someone just accidentally lucks into you as their instructor and their whole life would be different afterwards. Not everyone is reachable, right? So a bunch of people would go through that class and be like, "Oh, that guy was weird", but that handful would just absolutely light up. Chandoo (01:37:31): But there are other things too. For example, these days I'm drawing a lot more inspiration because I'm trying to be on the video platform more. I'm drawing inspiration from other YouTubers who have been in the journey for quite a bit of their life. Some of the people that I enjoy watching content are, there's one person called VZ waiter. It started off as a blog, but it's basically everything about life. But because it has been around for as long as I have my website, 13, 14 years, you could see that evolution in where he's leading the trajectory, then there's Electro Boom and few other places. Chandoo (01:38:07): When I watch them, I'm thinking they are in the life stages that I have been, or they are going through the journey and they're still maintaining that creativity and that inspiration and that fun factor because I believe in the content place, you need to have ample amount of fun. Otherwise, it would just drain you out and you lose that spark. That's where I'm looking. And I'm trying to draw inspiration from those people and try to get those values into what I do every day. Rob Collie (01:38:35): Yeah. I was going to ask you if you tended to draw inspiration from other YouTubers, like within our space, or if you primarily looked sort of broader? Chandoo (01:38:47): I'm looking more broader, because our space is good, but also there is this unwanted or a needed thing of comparison and jealousy and other negative feelings that you'll get. So I watch our space to learn, that's for sure. I watch our space to understand what sort of content is resonating with audience and what sort of reactions they are getting, how certain things are done, just so that I can better my craft. But when it comes to getting inspiration or setting a goal post or whatever, I'm trying to look at it more from a holistic point of view. If I want to be a content creator like X, who that X is, and that person is funny, that person is engaging, that person is awesome, and that person is also a genuinely nice person. So that's the person that I'm looking for. And there are plenty of them around the world. So there's enough content to inspire you. And that will also change from time to time, right? Chandoo (01:39:42): There's one other thing that I thought I'll mention. So when I saw raw data in the podcast, I remember because I had a podcast back in the day and I slowly wound it down. So I no longer have episodes, but all the old episodes are still there. I think early last year I was talking to a friend that, maybe I should start another podcast or maybe a video cast or a blog or something like that. I wanted to call this as data dump simply because it's just me venting things out or not trying to be an angry, middle-aged man or whatever. It's just that was the name that I went with. I was kind of in two minds, whether to call it a podcast or maybe make it a 30 minute YouTube segment or a series of videos on my channel or whatever. But I never went with that idea because I have been through the podcast journey. I know that it is quite a bit of effort to put out podcasts every week or month or whatever time period you choose. And I didn't really want to commit myself to something that'll just become a block on my creativity. Chandoo (01:40:46): Already, I'm doing YouTube. I have a blog and I have courses so I felt like this is not something that I want to put on my plate right now. But when I saw Raw Data come out, I was like, "Oh, that's a good name as well". Like, you know her obviously doing this, but it is good that you are doing this and reaching out to people that have stories to tell and people need to hear certain things. And the podcast is a very powerful medium. I hear podcasts all the time when I go on a walk or when I'm driving and it's a good companion to have. And it's a very solid way to build the bond with your audience as well, because you're connected at an audio level in their ears. Many times people hear this in their headphones, right? So you're literally, they're just whispering in their ears. Rob Collie (01:41:29): Yeah. We even had recently, we hired a consultant at our company who heard of us through the podcast. Chandoo (01:41:36): Yeah. Rob Collie (01:41:37): That's a first. Yeah, podcasts are a lot of work, especially if you want to do a good job on them and you've got that high quality bar. Chandoo (01:41:44): Yeah. Rob Collie (01:41:45): And I do too. You need Luke. There's no way. In fact, you need more than a Luke. And there are many people now that have their hands in one way or another, to varying degrees, with every episode, in addition to just the host and the guest. For example, Luke, I know what Chandoo is, you get to make a choice. I'm going to give you a choose your own adventure here. We make a gif for every guest, for every episode. Now, your gif is going to be this. It's going to be you standing there from the waist up and a table with relatively normal hair, like what you have today. Okay. But sitting next to you as a Van Der Graaf Generator, like one of those steel ball things, right? Chandoo (01:42:27): Yeah. Rob Collie (01:42:27): And you're going to put your hand on it in the gif and your hair is going to stand up and look like the Chandoo icon. Okay. Chandoo (01:42:34): Yeah. Rob Collie (01:42:34): All right. Now, the question is, and this is an important choice. The Van Der Graaf Generator is going to have an icon on it. Do you want it to be the Excel icon or the Power BI icon? Chandoo (01:42:48): I feel like you should say Power BI because it has got the power. Rob Collie (01:42:51): Okay. All right. So it's going to be the Power BI icon. It's going to be the best of all gifs. Chandoo (01:43:04): Those gifs were like an amazing thing as well. They add so much more quirkiness to the podcast, especially when you put them out on Twitter. Like, oh, it was fun. I liked the one with the [inaudible 01:43:18] on his monkey dancing stuff. Rob Collie (01:43:25): I mean, you can imagine, you have this idea, and this is something I learned again, writing a book. And I learned it again when I was making the videos, you have this idea, this idea hits you about what would be perfect and then you can't un-see the idea. You can't forget the idea, but now you have to go execute the idea. And it ends up being a lot of work. A gif a week turns out to be a reasonably intense pace because the idea isn't even always clear to us. What are we going to do? What are we going to do with this one? With you, thanks to the signature icon. We know what we're going to do. Rob Collie (01:44:03): So hey, sincerely, a real pleasure to get to talk with you for such a long time and to see you. And I really appreciate you taking the time to join us and to do the show with us. So now we've done the bio, right? We've done the history of Chandoo, right? That doesn't mean you can't come back. This podcast is in part, just a professional excuse to talk to cool people. Rob Collie (01:44:29): Chandoo thank you for coming and seriously, just thanks for being you. I've learned so much from you directly and indirectly over the years, be a good borg. Absorb things from other good people. I've definitely borged some good Chandoo content. Chandoo (01:44:45): Thank you so much for having me and I really appreciate what you're doing here with the podcast. And it is a really positive and fun way to get the message out to people and then tell good stories and connect to your audience. I wish you great success with the podcast. Rob Collie (01:45:01): The podcast is only as good as the guests. Powered by guests. Announcer (01:45:05): Thanks for listening to the Raw Data by P3 Adaptive Podcast. Let the experts at P3 Adaptive help your business. Just go to p3adaptive.com. Have a data day.  

We dive into the deep end of the Data Lake on this episode with our guest, Senior Staff Developer Advocate at Databricks, Denny Lee. Denny knows so much about Delta Lake, Apache Spark, Data Lakes, Data Warehouses, and all of the tech that is involved.  At one point Rob’s mind gets so blown by something that Denny talks about, his jaw may still be on the floor!   Episode Transcript: Rob Collie (00:00:00): Hello friends. Today's guest is Denny Lee. Absolutely one of the most friendly, outgoing, and happiest people that you'll ever hear on this show, that you'll ever hear anywhere, for that matter. And it's a little bit different. This episode, we spend a lot of time focused on technology, which we very often don't do. But the reason why we did is because Denny represents an amazing opportunity for us to explore what's happening like in this parallel universe of data. Most of us are up to our eyeballs in the Microsoft data platform and not even the entire Microsoft data platform, but very specific portions of it like centered on Power BI, for instance. Rob Collie (00:00:45): In the meantime, there's this entire universe of technology running on Linux in the cloud, that if you watch the show Silicon Valley, these are the tools being used by those people on that show. And also by the way, in the real Silicon valley, and to know someone like Denny, who has walked in the same world as us, because he's fully entrenched in this other world, I couldn't resist the opportunity to have him translate a lot of the things from that world for the rest of us. Rob Collie (00:01:16): In the course of this conversation, there's an absolutely jaw dropping realization that hits me that I was completely unaware of. I couldn't believe, I still can't believe that there was an ongoing flaw, an ongoing weakness in Data Lake technology that's only recently being addressed. And by the time we were done recording this, it was clear to me that we need to do it again, because there are so many things left to explore in that same day. We could have at least one more episode like this. So we're definitely going to have him back. So buckle up for a journey into the world of Apache Spark and Hadoop and Data Lakes and Lake Houses and Delta Lakes. Jason and the Argonauts make an appearance. We talk about photon, but most importantly, we talk about why this would be ever relevant to the Power BI crowd. So let's define that world, Denny's world as it, and then let's get into it. Announcer (00:02:22): This is The Raw Data by P3 Adaptive podcast with your host, Rod Collins and your cohost, Thomas Larock. Find out what the experts at P3 Adaptive can do for your business. Just go to P3Adaptive.com. Raw data by P3 Adaptive is data with the human element. Rob Collie (00:02:45): Welcome to the show. Denny Lee, I haven't spoken with you in, gosh, it's coming up probably on 10 years, right? Like it's getting close. Denny Lee (00:02:55): Yeah. It's been a long while. It's been a long time. You ran to the Midwest and I just wanted nothing to do with you. Rob Collie (00:03:01): That's right. I mean, that's the natural reaction from the Seattle tribes. Denny Lee (00:03:05): Oh, absolutely. Yeah, we're bad like that. Yeah. Rob Collie (00:03:08): It's like that scene in Goodfellas where the boss of [inaudible 00:03:11] says, and now I got to turn my back. Denny Lee (00:03:13): Exactly. Yeah. You went over the cascades. I'm done. Rob Collie (00:03:17): That's right. So, but you also seem to have left the other family, the Microsoft family. Denny Lee (00:03:24): That's very true. I did do that, didn't I. It was a long time ago. Rob Collie (00:03:28): This is like one outcast speaking with another. Denny Lee (00:03:31): That's true. That's true. We are outcasts. That's fair. But I mean, I don't think that had necessarily to do with leaving the big ship either though. I think we were just outcasts in general. Rob Collie (00:03:38): That was our role. Denny Lee (00:03:39): It was, yeah. It was all brand for us. Rob Collie (00:03:43): We're not going to spend too much time on history here, but well, we can, but there are a number of things that I do want to know about your origin story. You and I met basically over the internet, even though we were both Microsoft employees at the time- Denny Lee (00:03:58): And on the same campus. Rob Collie (00:03:59): ... And you showed up on my radar when Project Gemini and Power Pivot was actually getting close to like beta and stuff. Right. Denny Lee (00:04:07): That's right. That's right. Rob Collie (00:04:08): And you just materialized. It was like, now it's time to talk about these things publicly. And there was Denny. Denny Lee (00:04:15): Yes. Yes. [inaudible 00:04:17] loud. Rob Collie (00:04:17): Well, look who you're talking to. Denny Lee (00:04:21): Fair enough. Fair enough. Mind you, this is a podcast that I don't think anybody can see anything by the way. You do know that, right? Rob Collie (00:04:25): Yeah, I know. Yeah. They're not recording the video. Denny Lee (00:04:28): Thank you. Rob Collie (00:04:29): So what was your role back then? What got you associated with Power Pivot Project Gemini? Denny Lee (00:04:35): I'll be honest. What associated with, because I was going, "Why in expletive were we doing this?" In fact, because before this, I was on the SQL customer advisory team, I was the BIDW lead. BIDW. I know big, big words and the reason I bring that up is only because we had just announced maybe what, nine months prior, the largest analysis services cube on the planet, which was the Yahoo Cube so that was 20... At the time that was back in what, 2010, 24 terabyte queue built on top of like, I want to say two perabyte, 5,000, [inaudible 00:05:14] cluster. And so at the time that was a pretty big thing. So it's probably even bigger thing now. So whatever, but still the point being like, especially back in 2010, that's pretty huge. And so I'm going like, "Okay. So I just helped build the largest cube on the planet." And so now we're going to focus on this thing, which is this two gigabyte model. And basically my jaw dropped to the floor. Denny Lee (00:05:34): I'm going, "I just helped build the largest cube on the planet and you want me to help build a two gigabyte model? You sure you didn't mix up the T and the G here? Like what, wait, what's going on here? So that's how I got involved. But suffice to say, after talking to you, after talking to Kamala, after talking to some of the other folks, I realized, "Oh, I'm missing the point." I actually missed the whole point altogether about this concept of self-serve BI because, of course, everything before was very much IT based BI. So yes, it makes sense for an IT team to go ahead and build the 24 terabytes. Actually. No, it doesn't. But nevertheless, you don't want to ask your domain expert to basically build a 24 terabyte cubes. That seems like a really bad idea. So yes. Yeah. But that's how you and I connected because I was going like, "Wait, why are we doing this?" And then after being educated by you, realized, "Oh, okay, cool. This is actually a lot cooler than I thought it would be." Rob Collie (00:06:33): It's really interesting to think about it. The irony, right, is that I was thinking about Power Pivot in light of like, holy cow, look at all this data capacity we can put into Excel now. This is just like orders and orders of magnitude explosion in the amount of data that can be addressed by an Excel Pivot Table. To me, it was like science fiction level increase and you're going, "That's it." Denny Lee (00:07:01): Exactly. [crosstalk 00:07:03]. Rob Collie (00:07:05): Now, in fairness, I mean the compression does turn that two gigabytes and that's... The two gigabytes was the limit for file sizes in Office, but more specifically in SharePoint, right? I think it was the SharePoint limits. I wonder if that's even relevant today, but at the time it was super relevant, but yeah, the two gigabyte file size limit, even when compressed, might've been the equivalent of a 30 or 40 gigabyte normal cube, but you were still dealing with a different terabyte model. That's neat. Wait, this is so small. No, no. It's huge, trust us. Yeah. So you are one of the people who could write the old MDX. Denny Lee (00:07:48): That's right. Now we're hitting on the fact that Rob, Tom and I are old people. We're not talking about markdown extensions, all [crosstalk 00:07:55] framework. We're actually talking about MDX as the multi-dimensional expression. Does anybody still use it? Rob Collie (00:08:03): I think it's still used. Yeah. Denny Lee (00:08:04): Okay. Cool. Actually I have no idea- Rob Collie (00:08:06): But it's definitely been, in terms of popularity, it has been radically eclipsed by DAX. I mean even most of, if not all, but most of the former MDX celebrities now spend more time as DAX celebrities. Denny Lee (00:08:22): Do you want to mention names? Rob Collie (00:08:24): We've had some of them on the show, right? We've had Chris Webb, right? Okay. We haven't had the Italians. Denny Lee (00:08:29): Why not? Rob Collie (00:08:30): Well, I think we- Denny Lee (00:08:30): Alberto... Those guys are awesome. Rob Collie (00:08:33): I think we're going to, for sure. I mean, that's going to happen. Our goal is to eventually have like all 10 people who could write MDX back in the day, have them all on the show. We've had plenty of guests on the show where we talk about the origin stories of Power BI and all of that. Not that we couldn't talk about that. We absolutely could, but I think you represent an opportunity to talk about some incredibly different things that are happening today, things that are especially, I think a lot of people listening from the Microsoft data platform crowd might have less experience with a number of the things that you're deeply familiar with these days. And some of them do have experience with it. It's a very interesting landscape these days, in terms of technology like dogs and cats living together, mass hysteria, like from the Ghostbusters. Rob Collie (00:09:18): It's crazy how much overlap there is between different platforms. You can be a Microsoft centric shop and still utilize tons of Linux-based cloud services. And so I know what you're working on today, but let's pretend that I don't. Where are you working today, Danny? What are you doing? What are you up to? Denny Lee (00:09:38): I'm actually a developer advocate at Databricks and so the company Databricks itself was created by the founders of the Apache Spark project, which we're obviously still very active in. The folks behind projects like... And including Apache Spark and the recently [QUADS 00:09:57] project, which was to include the pandas API directly into Spark, but also things like MLflow for machine learning for Delta Lake, which is to bring transactions actually into a Data Lake, projects like that. That's what I've been involved with actually after all these years. Denny Lee (00:10:10): And just to put a little bit of the origin story back in just for a second, this actually all started because of that Yahoo cube. So what happened was that after I helped build the largest cube on the planet with Dave [Mariani 00:10:23] and Co just a shout out to Dave, what happened was that afterwards I was having regular conversations still as a Microsoft guy, right with Yahoo but we invariably went to like, "Wait, we don't want to process this cube anymore," because it would take the entire weekend to process a quarters worth of data. And if we ever need to reprocess all of it, that was literally two weeks offline. Sort of sucky. That's the technical term. Rob Collie (00:10:46): Had to be a technical term. Denny Lee (00:10:49): [crosstalk 00:10:49]Yeah, suck it. So what happened was that we were thinking, "What can we do to solve this problem?" And so we ended up both separately coming to the same conclusion that we were using Spark. So everything in terms of what I did afterwards, which I was part of the nine person team that created what now is known as HDInsight for project Isotope and then eventually joined Databricks was actually all from the fact that I was doing Spark back then, shortly after helping create the largest cube on the planet because of the fact we're going, "We don't want to move that much data anymore." Rob Collie (00:11:21): All right. Let's back up. That was a lot of names of technologies that went by there. It was a blur. Okay. So the founders of Databricks originally created Apache Spark? Denny Lee (00:11:31): Correct. Rob Collie (00:11:32): All right. What is Apache Spark? Denny Lee (00:11:35): Apache Spark is a distributed framework, in memory framework that allows you to process massively large amounts of data across multiple notes, notes servers, computers, things of that nature in a nutshell. Yeah. Rob Collie (00:11:49): So, that's what it does, but in terms of the market need, what market need did it fill? You had this kind of problem and then Apache Spark came along and you're like, "Oh my God." Denny Lee (00:11:58): The concept is, especially back then, the idea of web analytics. It was started with the idea that I needed to understand massive amounts of web data. Initially it was just understanding things like events and stuff. But then of course it quickly dovetailed to advertising, right? I need to understand were my advertising campaigns effective, except I have these massive amount of logs to deal with. In fact, that's what the Yahoo cube was. It was basically the display hats within Yahoo. Could they actually build solid campaigns for display ads on the Yahoo website? Well then what invariably happens is that this isn't just a Yahoo problem. This this is anybody that's doing anything remotely online that they had the same problem. And so what why became what now is considered the de facto big data platform is because of the fact that lots of companies, whether we're talking to the internet scale companies, or even what now are traditional companies, traditional enterprises, when it came down to that, they had a lot of data that was semi-structured or unstructured, as opposed to beautiful flat [crosstalk 00:13:03]. Denny Lee (00:13:03): What you actually had was basically the semi-structured, unstructured key value pairs, [jfonts 00:13:08], all these other myriad of data formats that you actually had to process. And so, because you're trying to process all this data, what it came down to is you need a framework that was flexible enough to figure out how to process all that data. And so in the past, we would to say, "Hey, just check into the database or I mean we share the database and we could [inaudible 00:13:27] it. But the data itself wasn't even in a format that was easy to structure in first place. Rob Collie (00:13:32): Let's start there. The semi-structured and unstructured data revolution. We already had this problem before the internet, right before everybody went digital. But it really made it mainstream. The most obvious example is people like being in Google and Yahoo, these search engines, right? Them going out and indexing and essentially attempting to store the entire contents of the internet so that they can have a search engine. Imagine trying to decompose the contents of a webpage on a website, into a database friendly format. You could spend years on it just to get like one or two websites schema designed to fit... It would absorb one or two websites. And then the world's third website comes along and it doesn't fit. Denny Lee (00:14:19): Exactly. Rob Collie (00:14:19): ... what you designed. And so I've actually, in terms of storage, the whole Hadoop style revolution of storage, I think is awesome, without any reservation. The whole notion of data warehousing, if you just take the words at face value, don't lose anything. And it was very, very, very expensive to use SQL as your way of not losing anything. And so these semi-structured stores were much more like, om, nom, nom, garbage cans. You could just feed them anything and they would keep it. Denny Lee (00:14:49): That's right. Rob Collie (00:14:50): So then we get to the point where, oh, we actually want to go read some of that stuff. Denny Lee (00:14:54): Ah. There we go. Yes. Rob Collie (00:14:55): We don't want someone to store it. We want to be able to like, I don't know, maybe access it again later and do something with it- Denny Lee (00:15:00): Even get some actual value out of it. Rob Collie (00:15:02): Yeah. I mean, is that where Spark comes in? Is at that point? Denny Lee (00:15:06): Absolutely. So we would first start with Hadoop and so the context, if you want to think of it this way, is that because exactly to your point, I could build a functional program with MapReduce to allow me to have the maximum flexibility to basically... Because everybody likes writing in Java, I'm being very sarcastic by the way. Okay. Very, very sarcastic. You can't tell if you don't know me well, but yes. So I want to really call that out. So you love writing Java. You want to write the MapReduce code, but it does give you a massive amount of flexibility on how to parcel these logs and it's a solid framework. So while you wouldn't get the query back quickly, you could get the query back. And so the context is more like, if you're talking about like, especially at the time, you're talking about terabytes of data, right? Denny Lee (00:15:52): The time it would take for me to structure it, put into a database, organize it, realize the Sparks call wasn't working, I realized that I forgot variable A, B, C, and all these other things that you would iterate, especially with the classic waterfall model. By the time we did it, it's like 6, 7, 8 months later, if we're lucky. And then if you had a large enough server. There's all of these ifs. And so what happened with the concept of [inaudible 00:16:15] was like, "Okay, I don't care. It's distributed across these 8, 10, 20 commodity nodes. I didn't need any special server. I run the query, might take two weeks but the query would come back and I would get the result. And the people were like, "Well, why would you want to wait two weeks?" I'm like, "Well, we'll think about it from two ways. One, do I want to wait two weeks or do I want to wait eight months? Number one. Denny Lee (00:16:40): Number two. Yes, it's two weeks but then I can even figure out whether I even needed the data in the first place, right? Rob Collie (00:16:46): That's right. Denny Lee (00:16:47): And so how Spark gets added on top of this is saying, well, we can take the same concept as Hadoop, except do it on any storage, number one, you work specifically to Hadoop, you could do it on any storage, number one. And number two, it will be significantly faster because we could actually put a lot of the stuff... Stuff, a technical term into memory. So I could process the data in memory, as opposed to just going ahead and basically being limited four or eight gigabyte, especially with the older JVMs, right, limited to that much memory to do the processing. Rob Collie (00:17:20): I have a hyper simplified view of all of this, which took me a long time to come around to, which is the old way, was to look at a bunch of data and figure out how to store it in rectangles. And that's very, very, very, very difficult, labor intensive. That's the six, seven months, if you're lucky. And by the way, rule number two is you're never lucky. Denny Lee (00:17:41): Exactly. Rob Collie (00:17:43): So six or seven months is what this project is specked to be at the beginning. And it never comes in on time. Denny Lee (00:17:48): We don't hit that target anyways. Exactly. Rob Collie (00:17:50): And then the world changes and all of your rectangular storage needs to be rethought. Right? Okay. So pre-rectangle-ing the data to store it ends up being just a losing battle. Now analysis, I like to think of analysis. Analysis is always rectangle based. When you go to analyze something, you're going to be looking at the things that these different websites have in common. You always are going to be extracting rectangles to perform your analysis. I loved your description of like, okay, we can take six, seven months wink, wink to store it as rectangles. And then we get fast rectangle retrieval. We think we hope, Denny Lee (00:18:26): We hope. Rob Collie (00:18:26): We probably did not anticipate the right rectangles or we can delay that. We can delay the rectangularization. Store it really easily and cheaply by the way, quickly, easily cheaply. And later when we're fully informed about what rectangles we need, that's when the rectangle work happens and even in the old days, when all we had was Hadoop's original storage engine, two weeks, yeah, I can see that leaving a mark at runtime. That's also what we call a sucky. We call that slow in the tech universe, but it's still better than the six or seven actually 14, 20 months from before. Okay. So Spark walks into that and through some incredibly hocus-pocus magic, just brings fast queries, essentially fast rectangle extraction to that world where you still don't have to pre-rectangle. You still get all the benefits of the unstructured cheap commodity storage, but now you don't have to wait two weeks for it to run. Denny Lee (00:19:25): Right. So if you remember our old Hadoop hive days, we would rally around the benefits of schema on read, right and don't get me wrong. I can go on for hours about that's not quite right. Rob Collie (00:19:34): You're giving me way too much credit right now. I've never touched Hadoop. I'm aware of it and we use Data Lake storage at our company and all that kind of stuff. Think of it this way. I have a historians point of view, a technical historian's point of view on this stuff. But like you start talking about, "Yeah, you remember back in our days and we were like sling..." No, I can play along, but it would feel inauthentic. Denny Lee (00:19:56): No, fair enough. So from the audience perspective, the idea of schema on read in essence using Rob's analogy would be like, you store a bunch of circles, triangles, stars. That's what's in your cheap storage and the idea of schema on read at the time that you run your query at runtime, I will then generate the rectangles based off of the circles and squares and stars that I have in my storage. Rob Collie (00:20:20): Oh, okay. I actually misheard you as well, which I thought you said schema and read. Schema on read. Denny Lee (00:20:27): Yes. Rob Collie (00:20:27): That's awesome. I wasn't even aware of that term, but now that you say it, yeah. That's runtime rectangle. Rectangle when we go to query. Okay, awesome. There was the technical term for- Denny Lee (00:20:37): Yes, there is a technical term for runtime rectangles. That's correct. Rob Collie (00:20:40): I feel so incredibly validated. I will effortlessly slip schema on read into conversation. Denny Lee (00:20:49): Perfect. So, but I'm going to go with the analogy. So in other words, now that we generate runtime rectangles, the whole premise of it was that with Spark we could generate those runtime rectangles significantly faster and get the results to people significantly faster than before with Hadoop. And so that's why Spark became popular. And the irony of it all was that that actually wasn't its original design. Its actual design was actually around machine learning, which is not even runtime rectangles. Now it's runtime erase. So like Vector, so it's a completely different thing, but what ultimately got it to become popular wasn't the runtime arrays of vectors? It was actually the runtime rectangles so people could actually say, "Oh, you mean I don't have to wait two weeks for the query. I can get that query down in a couple of hours?" "Yeah." "Oh, okay. Well then let me go do that." Rob Collie (00:21:39): So that's it, right? Miller time. We're done. That's it. That's the end of the technology story. We don't need [crosstalk 00:21:43] Denny Lee (00:21:43): Yeah, we're done. That's it? There's nothing else there. Thomas Larock (00:21:47): We've hit the peak. Denny Lee (00:21:48): Yeah, that's it. We've hit the peak. Rob Collie (00:21:50): This is the end of history. Right. Denny Lee (00:21:52): Yeah, this is it. Thomas Larock (00:21:54): We're at the end of the universe. Denny Lee (00:21:55): But I'm sure as we all know, the curve, the Gartner hype cycle and everything else for that matter, that's not quite the case. And what's interesting is that especially considering all of us were old database folks or at least very database adjacent, even if we're not database specific. One of the things we realized about this runtime rectangle or the schema on read concept was wait, garbage in, garbage out. Went out of your way to store all of this data. You should do that. In some cases you really do have to leave it in whatever state you get it. Because like whatever the query for the rest API, whatever [inaudible 00:22:36] packet protocol, whatever, whatever format. Sometimes you don't have a choice and that's fine. I'm not against that idea because the context is, especially back in 2011, 2012, we were using statistics like the amount of data generated in the last year was more than the amount of data generated in all of history before that. Thomas Larock (00:22:57): You know that's a bullshit statement, right? Denny Lee (00:22:58): Whether it's a bullshit statement is actually irrelevant to me because the context is not wrong. The reason that statement came up was actually because machines were generating the data now. So since machines are generating the data, the problem is that we don't actually have people in the way of the data being generated. So it's going to be a heck of a lot more than any organic person involved to be able to make sense of that. Rob Collie (00:23:22): I love that exchange there, right? The amount of data generated in this timeframe is more than that timeframe. Tom says, "You know that's a bullshit statement." Then he says, "Hey, do not let the truth get in the way of a good concept." Denny Lee (00:23:33): Exactly. Do not. Do not do not. It's okay. Thomas Larock (00:23:39): That doesn't matter. What I just said doesn't matter. Denny Lee (00:23:42): Exactly to your point though. That's the whole point. It doesn't matter what the statement really is in this case. What really matters is the fact that there is that much generated data. That's what it boils down to. Right? It doesn't matter what the marketing people said. Businesses still had a problem where they had all of this data not structured coming in. And so now the problem is, it's mostly noise. It's mostly garbage and you're going to need time to figure out what's actually useful and what's not. You can automate to your heart's content and so that means, okay, sure, I've processed part of the data to figure that out. But the reality is there's always new devalues, new attributes, new whatever, coming in. At least if you're successful, there's new something coming in. And if there's no something coming in, whatever format that is, you're going to have to figure out what to do with it. Denny Lee (00:24:39): And so at some point, especially when you take into account of things like streaming systems, where basically just data is constantly coming through and it's not like batch oriented at all where basically data's coming through. Multiple streams that ultimately you want to place into a table. What does it imply? It implies that I actually want a rectangle, a structure of some type at the closest point to where the data resides right from the get-go. Because what I want, isn't all of the key value stores. What I want is all the information, but there's a difference between information and data or if you want to go the other way, there's a difference between noise and data. So whatever way you want to phrase it, I'm cool either way with that too. But the fact is the vast majority of what's coming in is noise. You got to extract something out of it. Denny Lee (00:25:32): And then that's where the sticky, coil mining analogies kick in. And again, I'm not going to play with that one, but the point is what it boils down to is that that means I need some structure to what I'm doing, flexibility still. So if I need to change the structure, I can do that, but I need some form of structure. And so what's interesting about this day and age, especially in this day of like we're using Spark as the primary data processing framework, is that there are like, I want to talk about Delta Lake, but as a call-out to other projects, there are other, projects like [inaudible 00:26:01], there's other projects like Apache Iceberg, right? And I'm going to talk about Delta Lake, but we all came roughly the same time with the exact same conclusion. We've been letting so much garbage into our data, we need some form of structure around it so we can actually figure out what's the valuable bits. Rob Collie (00:26:18): It seems like a really hard problem. I mean, even like without technology in the way one person's trash is another person's treasurer. Denny Lee (00:26:25): Exactly. Rob Collie (00:26:26): What's considered trash today is treasure tomorrow. And if I wanted to be really snarky, I'd be like, "Are you saying that we need to go back to rectangular storage in the first place?" So it's back to SQL after all? Denny Lee (00:26:40): Almost. In fact that's exactly what I'm saying. What I'm saying is that I want to get the rectangles as close to the storage, but not actually the storage itself. Okay. The reason I want to get as close to is because of exactly what you said, because maybe today, the stars in my circle, star, square analogy are what's important and then need to be converted into rectangles, but I don't need the circles and I don't need the triangles. Later on, I realize maybe I need the triangles or some of them, not all the triangles. And later on, I may need... You know, forget about the triangles altogether. Let's just put the circles in and whatever else. But the context is exactly that. So you want structured storage as close to the storage, as you can ,without actually going ahead and messing with your input systems. Because typically there's two reasons why you can't mess with it. Denny Lee (00:27:29): One is because you're not controlling it, right? If you're hitting a source system, you're hitting a REST API, it is what it is. That's the source system. So you're going to get whatever it is. And so in order to ensure that you've [inaudible 00:27:42] and you've kept it as close to the original value as possible, your job is basically if it's a REST API call, grab the JSON packet, chuck it down to disk as fast as you can so that way... And validate for that matter, that the JSON's fully formed. So that way, okay, got it. This is the correct thing that also [inaudible 00:27:59] store, but now once you've done that, you can say, "Okay, well out of this JSON packet, I actually only need six of the columns. Rob Collie (00:28:04): Are we talking about dynamic structured caching? Denny Lee (00:28:09): No. Good call, but no, it's not. First of all, there's actually two problems. One is dynamic and one's caching. Thomas Larock (00:28:15): You're right. There's a difference between information and the data. I want to point out that all information is data, but not all data is information, right? [crosstalk 00:28:26] and we can talk about busting your stones about more data's being created because Stephen Hawking would tell you that information is neither created or lost in the universe. It's all around us. Nothing's ever been created or destroyed. No information could be lost, otherwise there's a problem with the universe. So when people talk about what they're really talking about is they're just able to collect more. It's always been there. They just never had the ability to go and get it so easily. Like you're at a buffet and you just can't stop. So here's the problem I see is when you talk about this stuff, I really see two worlds. I said, problem. That's not fair, but there's two worlds. Thomas Larock (00:29:05): The first thing is, you have to know what question you want answered. So there're some executives somewhere and he goes, "I got questions. I need answers." You have to go and collect the data. You have to go and get the information to answer those questions. Denny Lee (00:29:21): Correct. Thomas Larock (00:29:22): What you're really talking about though, is something... It's almost like a deep state R&D project. I'm going to go get all the data. Maybe it's information. I don't know. I'm searching for an extra signal. I'm like SETI and I'm trying to find a signal that we don't know exists yet because I think it might have more information. However, in the meantime, while you're waiting six months for something to finish, the executives like, "I need red, green, yellow." I need to make a decision. I can't wait the six months. So I think there's really two fields here and I don't think people talk enough about the overlap because they always talk about how, I'm just going to fire up 10 more nodes and we're just going to process all this. And we're just going to keep looking for something that might have some value. Who knows? And you're still missing... I just need an actionable dashboard right now. Here's my question. What information will satisfy that question? And I need you to keep ensuring that the quality of the information you're giving me is of the same level. Thomas Larock (00:30:20): Now, if there's something new that you discover later, that's great, but for now I just need something now. Anyway, I just feel that a lot of times people head down the path that you're at, but then you are the edge case, right? You're building Yahoo cubes and all that. And I remember when all that was coming out and it's just wonderful, but you're such an edge case, but I think people want to be that edge case that you're at and they just want to get everything they can and look for that signal. And I'm not sure that that's where people really need to be. I think in a lot of cases, people can have like a little bit of a simpler life, but they should think about the stuff you're doing at Databricks and Spark and all that and think about it more as... It's more research, in my opinion. Denny Lee (00:31:04): Actually, I disagree with that statement, but not for the reasons you think. So, because I actually agree with, I'd say 80% of what you're saying. The reason why I say that is because what typically happens though, is that when the executive or the product manager or whatever recognizes they need that data or something new comes in, by the time they actually need it, and when you start processing it to finally integrate it in the dashboards and the reports and everything else, it's too late. Number one. Thomas Larock (00:31:39): Agree. Denny Lee (00:31:39): Number two, from an ad hoc perspective, more times than not, you don't even know what you know, until you start processing it. Now, saying what you just said, I do actually agree with you on the point, which is, you're not supposed to make this into a deep state research project where you're grabbing everything. I do agree with that wholeheartedly, in fact. Denny Lee (00:32:00): This has nothing to do with structure or anything else. This has to do purely has to do with... Look, you're storing all this stuff. There's actually a couple of really important things to call out. One, you're spending money to store it. If you're storing that much data, it's going to cost money. Do you need to store all this stuff? Number two, do you have privacy concerns by storing all of this data? You can't just store data and for the fun of it and not taking the [inaudible 00:32:28] that you actually have security protocols, you have privacy protocols that you actually have to care about. Sorry, you do. Okay. And this is before I get into HIPAA health act, GRC compliance of CCPA, any of that stuff, right? That's a whole other conversation. So you actually have to care about these things. You're not supposed to just go in and randomly report stuff. Denny Lee (00:32:46): So like I said, I actually agree with the sentiment in what you're talking about. What I think the only difference between... And where the 20% arrives is that when you are a moderately size or larger company, what the concern really is is that you really don't know what you don't know. And if you're going to be processing any data when you're a moderately sized company or larger, you ultimately need to process a lot more data than you want even, in order to get to that actual dashboard. Does it replace the actual dashboard? Quite the opposite. It means you need to create better ones faster. We're actually not that far apart. It's just more that part about saying, okay, ultimately we don't know what we don't know. So if you're a moderate sized company, you're going to have to probably process more data, but you have to respect the data. Thomas Larock (00:33:39): I actually think we're closer than the 80% because I did leave that part out where that data, if you're collecting the right data, it should lead you to ask more questions about the data. I do agree with that. I think my point was when people think about some of these projects, there's not enough structure around it. Like, "Hey, what's the information I need right now for what I can do. And then what's the other stuff that I have to go and look for." And yes, good data should lead to good questions. "Hey, I'm curious. Can we do this other thing too?" Now we're talking now it's going to take four weeks. Now, it's going to take six weeks and that's okay. And that's what I call that research part. But you only get there by asking the exact questions. Denny Lee (00:34:17): Exactly. You should never start with the context of like, "Let me just grab everything," because I'm like, no, no, no, no, no. This is a cost problem. This is a security problem. There's a privacy problem. There's all sorts of problems. And you don't start that way. Anybody that ever starts that way will fail. Unequivocally will fail their project. And I'm going like, "No, no, no, it's newer." I'm like, "No, same problem in the data warehousing world." The same problem. Thomas Larock (00:34:39): But that's a problem for future you [crosstalk 00:34:44] you today, you don't have to worry about that. That's a problem for future [inaudible 00:34:48]. Denny Lee (00:34:48): Yeah. I guess if you follow the idea that I'll just switch jobs every two years and then I can run away. Sure. I guess that's fine, but I would hope that all of us at least, when we're doing this podcast, we're actually trying to advise people who actually want to provide value authorized [crosstalk 00:35:03] Rob Collie (00:35:05): Given the truthiness of more data being created in the last five seconds than in all of human history, Denny's going to have to have me changing jobs more than every two years. Right. Denny's had a larger number of different employers in the last week than most people have in seven lifetimes, just to keep escaping the data storage. So let's get back to the linear progression. Right? We had started with data warehousing, turn everything into rectangles, incredibly expensive, incredibly slow, even just to get it stored. Then we went full semi-structured Hadoop, which has delayed the rectangularization, schema on read. I'm going to just drop that right in there. I'm just practicing. I'm trying it on- Denny Lee (00:35:49): We're there for you, man. Rob Collie (00:35:51): ... But it was a two week query time. So then along comes Spark and now it's a lot faster. And we were starting to turn the corner as to, we need something that resembles structured rectangular style... I don't know. I'm going to use the word storage, but I'm very tentative about that word. We need concept of structure as close to the semi-structured storage as we could possibly get. I don't think we finished that story, but I'm definitely not yet understanding is this where we turn the corner into Delta Lake and Lake House or is it Databricks? What are we- Denny Lee (00:36:28): No, no. It's actually, is the turn of [inaudible 00:36:30] Delta Lake and Databricks and Lake House for that matter, because that's exactly what happened. So at Databricks, the advantage of us using Spark and helped create it, is that we were now working with larger and larger customers that had to process larger and larger amounts of data. And they're so large that basically we have to have multiple people trying to query at the same time. And so you remember old database, the idea of like, do I do dirty reads, do I have snapshots rights, things of that nature. So what invariably happened with almost every single one of my larger customers, was that all of a sudden this idea that your job's failing, and that's normal, right? Your jobs failing, but they'll restart. But how about if they fail, fail? Denny Lee (00:37:14): What ends up happening is that these files are left over in your storage. And this is any distribute system, by the way. This isn't a specific to Spark. This is any distribute system that's doing right. If it's doing a distribute multiple tasks, a job that runs multiple task, that's writing those multiple tasks onto multiple nodes with multiple nodes or writing to disk of some type storage of any type, invariably something fails. And so, because something fails, all of a sudden you're left over with a bunch of orphaned files. Well, anybody that's trying to read it is going to say, "Wait, how do I know these files are actually applicable versus these faults actually need to be thrown away?" You need this concept called a transaction to clarify which files are actually valid in your system. So that's how it all started. It all started with us going backwards in time, recognizing the solution was already in hand, we've been doing it for decades already with database systems, we needed to bring transactions into our Data Lake storage. Rob Collie (00:38:17): Quick, an important historical question and both of your histories, Denny and Tom, have you had experience? Have you performed dirty reads and if so, were they done dirt cheap? All right. Had to do that. Thomas Larock (00:38:34): So in my answer, my answer, Rob is yes. I was young and I needed the work. Rob Collie (00:38:42): I mean, now we need to write the whole song. Right? Thomas Larock (00:38:45): I'm just going to tweet it. [crosstalk 00:38:48] Tweet it so I can [crosstalk 00:38:50]. Denny Lee (00:38:50): At this point we literally could do a trio here without any problems because we know for a fact you qualify. Rob Collie (00:38:55): So, is that ultimately like tagline for Databricks, dirty reads done dirt cheap, but it's not dirty because it's transactional. Denny Lee (00:39:03): Exactly. Rob Collie (00:39:04): I think the world would really respond. Well, the problem is, is that we're now old enough that that song is like the Beatles. Denny Lee (00:39:11): Yeah. Yeah. That's true. We should probably provide context to our audience here. Thomas Larock (00:39:18): Wow. Rob Collie (00:39:19): That is an old AC/DC song. Dirty Deeds Done Dirt Cheap. That's the album title too. Thomas Larock (00:39:23): I think all four Beatles were still alive when that stuff- Denny Lee (00:39:26): Yes, all four Beatles were alive during that song. Rob Collie (00:39:29): John Bonham from Zeppelin might've still been alive. Thomas Larock (00:39:35): So, we're aging ourselves to our entire audience. Thank you very much. Rob Collie (00:39:39): All right. All right. All right. And dad joking to the extreme. Thomas Larock (00:39:42): We've thrown around the term, JSON a lot. Rob Collie (00:39:44): Can we demystify that for the audience? I actually do know what this is but like- Thomas Larock (00:39:49): It's call hipster, XML. Rob Collie (00:39:51): JSON equals hipster XML? Thomas Larock (00:39:53): Yes. Rob Collie (00:39:53): Okay. All right. This sets up another topic later. Denny, would you agree that JSON is hipster XML? Denny Lee (00:39:59): I absolutely would not, even though I love the characterization, but the reason why is because I, [crosstalk 00:40:06] Hey, Rob. Yeah. Yeah. I'm saying it. Rob Collie (00:40:12): Me too. Okay. Thomas Larock (00:40:14): See. Is Jason a subset of XML? Denny Lee (00:40:15): JSON is an efficient way for semi-structured data to actually seem structured when you transmit it. Rob Collie (00:40:25): Okay. So, it's the new CSV, but with spiraling, nesting, curly structures. Denny Lee (00:40:30): Correct because it allows you to put vectors in a raise quite efficiently and allows you to put [Nyssa 00:40:35] structures in efficient. Rob Collie (00:40:36): So it's a text-based data format, right? Denny Lee (00:40:38): Correct. Rob Collie (00:40:38): ... so that it's multi-platform readable- Denny Lee (00:40:41): Exactly. Thomas Larock (00:40:42): Yeah. It's XML. Denny Lee (00:40:44): No. Rob Collie (00:40:45): It's hips and mouth. Denny Lee (00:40:46): So, I believe I have war stories probably because of you Rob about all... Especially when reviewing the XML A. Rob Collie (00:40:57): Oh yeah. Well, listen, I don't have a whole lot to do with that. Denny Lee (00:41:02): [crosstalk 00:41:02] I'm just saying, Tom, you will vouch for me. The insanity of XML, right? You will vouch that. Yes, please. Thank you. I'm supposed to figure out the queue structure with this thing? My VS, visual studio is collapsing on me right now. Rob Collie (00:41:21): Well, hey look that XML A stuff which had nothing to do with creating, is the thing that we talked about with Brian Jones on a recent episode. That's the stuff that was saved in the Power Pivot file as the backup so that we could manually edit that and then deliberately corrupt the item one.data file in the Power Pivot stream and force a bulk update to like formulas and stuff. And yes, it was a pain in the ass. Okay. So JSON, it's advanced CSV, it's XML like, it's the triple IPA of XML or is it the milk stout? Is it the milk stout of XML? Denny Lee (00:41:58): I'm not even partaking in this particular conversation? I'm just- Thomas Larock (00:42:02): Honestly, JSON would be like the west coast hazy IPA. Okay- Denny Lee (00:42:07): Okay. Fine. I will agree to that one. You're right. You're right. West coast hazy IPA. Fair. That's fair. Rob Collie (00:42:12): All right. All right. Okay. Well, I'm glad we I'm glad we did that. I'm glad we went there. And the majority of... I'm going to test this statement... The majority of Data Lake storage is in JSON format? Denny Lee (00:42:23): No. Actually the majority is in the parquet format or ORC format, by the way, depending on it. But which is basically for all intents and purposes a binary representation from JSON into a column store. Rob Collie (00:42:34): I'm just going to pretend I didn't ask that question. I'm going to move on. All right. So is it the notion that when you're reading from Spark, which is of course reading from other places that because things are being updated in real time, it has unreliable reads. Denny Lee (00:42:52): So it's not just Spark. Any system. Rob Collie (00:42:55): Sure. Why did Spark need Databricks? Denny Lee (00:42:58): Fair enough. And I mean, it's more like in this case, honestly, it's why does Spark need Delta Lake? And to be fair to the other systems are out there. And like I said, there's iceberg Hadoop as well, but I'm going to just call out Delta Lake. Right. But the reason was because it's very obvious when you talk about streaming. If I've got two streams that are trying to write to the same table at the same time, and one invariably wants to do an update while another wants to do a deletion, that's a classic database transactional problem. Rob Collie (00:43:25): Even like a Dropbox problem, right? You have multiple users in Dropbox. I get merge conflicts all the time. Denny Lee (00:43:30): Right. So that's exactly the point, right? What it boils down to is that when you get large enough in scale, I don't mean size of your company. I just mean the amount of data you're processing in which you could conceivably have multiple people trying to write or read or both at the same time, you could solve that by two people trying to do the same thing at the same time, and you'd still have the same problem. Why did databases become awkward? That was one of the key facets that we could either succeed or fail. It was very binary. It's succeeded or failed. So we knew whatever went in, it got in, or if it failed, you're given an alert and you're told, "Guess what? It didn't work. You need to try again." Right. Same concept. That's what basically it boils down to. It's like, if you're going to be using your Data Lake as a mechanism to store all of your data, will then do not want transactions to protect that data so that it's clear as daylight what's valid and what's invalid. Denny Lee (00:44:31): So I'm not even trying to do a product pitch at this point. I can do that later, but I'm just saying... It's like, no, just start with that statement. When you have a Data Lake, you want to basically make sure whatever systems processing it, whatever systems reading it. I don't care which one you're using honestly. Obviously I hope you're using Spark, but the fact is I don't care. You use any system at all that's trying to read or write from it. Don't do you not want to make sure that there are some form of consistency, transactional guarantees to ensure what's in there is valid. And then once you've accepted that problem, that this is a problem that you want solved, then invariably that'll lead you to the various different technologies. Denny Lee (00:45:09): Again, I'll be talking about Spark and Delta Lake because I think they're awesome, but the reality is like, that's why the problem, right? That's why we realized this was crucial for most people. Incidentally, that's why we open sourced Delta Lake because we're going like, "No, it's so important that I don't care which platform you're using it on." I don't care if you're using it on Databricks. I don't care if you're using Spark. We just don't care. Denny Lee (00:45:34): Because the whole point is that you got to trust your data first. And if you can't trust your data, forget about machine learning. Forget about BI. Forget about all of these other systems that you want to do downstream. I need to make sure whatever store my Data Lake is actually valid. And then a lot of other people will tell me like, "Oh yeah, well maybe if you could just shove into a database or chuck it over, all this other stuff, I'm like, don't get me wrong. I'm all for databases. Just because I'm a smart guy, doesn't mean I hate databases. Quite the opposite. I've run my own personal Postgres and my SQL still to this day and SQL server. Yes. I have a Window box somewhere in this house so I can- Thomas Larock (00:46:05): See if it runs on Linux. Denny Lee (00:46:08): SQL can run [crosstalk 00:46:11] No, it's true. It's true. But I'm still old school. I actually, no joke, still have a running at least I should say I used to still have a running 2008 R2 instance somewhere. The Thomas Larock (00:46:19): The Lee household, by the way, doesn't actually ever need heat. [crosstalk 00:46:23] It's heat by CPU. Denny Lee (00:46:25): Right. All the stick at CPs and GPS. I can actually cook an egg now. Yeah. Rob Collie (00:46:33): It's like the most expensive power way to cook something. Denny Lee (00:46:35): Exactly. Yeah. Thomas Larock (00:46:37): So he doesn't touch a thermostat. He just spins up like a hundred nodes. There you go. Denny Lee (00:46:41): Yeah, yeah. Rob Collie (00:46:42): That recent heat wave in Seattle was actually when Denny's air conditioning failed and it started pumping all that heat out into the atmosphere. Denny Lee (00:46:50): Yeah. Oops. Sorry about that guys. My bad. Rob Collie (00:46:52): Is it El Nino? No, it's Denny. Denny Lee (00:46:55): Thank you. I'm already in enough trouble as it is Rob. Do you really need to add to the list of things? I mean, I'm in trouble for now. Rob Collie (00:47:02): We're going to blame you for everything. Denny Lee (00:47:03): Oh, fair enough. No, that's all right. That's completely fair. But the concept's basically, it's like, because of its volume, it's sitting in the Data Lake anyways. Do I want to take all of it and move it around or somewhere else? And I'm like, I'm telling you. No, you don't. That's the exactly the Yahoo promo. I know it's sort of funny, but I'm really going right back to Yahoo. When we had this two petabyte system, we were moving hundreds of terabytes in order to get in to that cube. I'm going, "Why, why would I want to do that?" And especially considering we had to basically update it with new dimensions and new everything like every month or so, which meant that we were changing the cube, which meant we're changing the staging databases that was in... Which was basically this massive Oracle rack server and then against your petabytes of data. The whole premise is like, I obviously want dashboards, but I don't want to move that much data to create the dashboards. Rob Collie (00:47:59): Sure. My jaw is on the floor at this point and oh my God, we didn't have the concept of transactions in Data Lakes from the beginning? Denny Lee (00:48:11): That's correct. Rob Collie (00:48:12): Oh my God. Thomas Larock (00:48:14): It's just a lake. Denny Lee (00:48:14): Yeah, it's a lake. [crosstalk 00:48:16]. Why would you bother? All right. Thomas Larock (00:48:19): So the idea, because- Rob Collie (00:48:20): ... multiple people are peeing in that lake at the same time. Denny Lee (00:48:23): Well, there you go. So people were under the perception that people weren't peeing in the lake, so the lake is perfectly fine. And then in reality, not only are people peeing in the lake, you got all sorts of other duties inside there. So yes, I'm using duty. Rob Collie (00:48:39): You've got streams crossing, you've got all kinds of things. So I'm absolutely getting smarter right now. And I'm super, super, super grateful for it. Where do I like to get smarter? That's right in front of an audience. That's the best place to learn. Everyone could go, "Oh my God, you didn't know that." Denny Lee (00:48:54): I think Tom can vouch for me though. The fact that you're getting smarter with me on is not a good testament for you, buddy. Thomas Larock (00:48:59): No. Rob Collie (00:49:00): Ah, no. Come on. You're something else, bud. So Denny Lee (00:49:02): That's right. Something else, but that doesn't mean smarter. Rob Collie (00:49:07): I think your ability to bring things down is next level. Okay. So transactions. I'd heard about Databricks for a while, but in prepping for this, I went and looked at your LinkedIn. Denny Lee (00:49:20): Oh wow. Rob Collie (00:49:20): ... and I saw these other terms, Delta Lake and Lake House, but Databricks has existed longer than those things. Is that true? Denny Lee (00:49:29): That is absolutely true. Rob Collie (00:49:29): And the company is called Databricks. Denny Lee (00:49:31): Correct. Rob Collie (00:49:32): So what is the reason for Databricks to exist? Is it because of this transactional problem or is it something else? Denny Lee (00:49:40): No, actually that's where I can introduce the Lake House. So if I go all marketing fun just for a second, like you use your data bricks to build your lake house. Bam. Rob Collie (00:49:51): Oh no, no. Denny Lee (00:49:55): Yeah. I did that. I did. No, no. Data bricks are what you use to build a data factory and a data warehouse. Now you live in the Data Lake house, right? No. You make the data bricks to build your Data Lake house. Absolutely. I don't need a data factory. I'm building a beautiful Data Lake house. It's right on the water. It's gorgeous. I'm sitting back. I'm listening. Yeah, no- Thomas Larock (00:50:15): Again, no. The Data Lake house is part of your data estate. I get that. Okay. But to me, you're using the Databricks for the data warehouse, the data factory. And do you shop at the data Mart? I get it. Rob Collie (00:50:27): I'm trying to be so deliberate here. So look, we're trying to follow a chronology. We want a number line here. And so first there were Databricks. Why Databrick? Denny Lee (00:50:39): Why Databricks is because we wanted to be able to put some value add initially on top of Apache Spark. Right? So the idea that you can run a Apache Spark with it's open source is great, and lots of companies have been able to either use it themselves without us. Or there are said cloud providers and vendors that are able to take the same open source technology and build services around that and have benefited greatly for doing that. Databricks decided to not go down the traditional vendor route because the decider say, we're going into the cloud right from the get-go. At the time that we did it, it seemed really risky. In hindsight, it makes a lot of sense. But at the time that we did it back in 2013, seemed really risky because they're going like, "We could get lots of money for on-prem customers," because that's where the vast majority of customers work. Denny Lee (00:51:30): But where we saw the value was this concept that with the elasticity of the cloud, there is going to be a whole different set of value add that Spark and the cloud together will bring that you can't get on-prem. The idea that I can automatically spin up nodes when you need them, if you ever log into Databricks, the idea is like we default between two and eight nodes. So you start with two worker nodes and then basically you'll scale up to eight automatically. But just as important scaled out, just as important. So the idea that once you're done, we're not leaving those eight nodes on. If we're seeing no activity, [inaudible 00:52:08] those nodes go back down. You're not running. It's idle after 60 minutes. That's our default. Fine. We're just shutting it down. The idea that instead of you writing lots of code yourself, you've got a nodebook environment. Denny Lee (00:52:22): Not only does it make it easier to write it, but also to run your queries, but also has permissions that also has commenting. But also on top of that, if you're closest to shut down, everything's still safe. You can still see the nodebook. You can see the output of the nodebook still saved for you, but you're not paying for the price of a cluster. Right? So this is what I mean by value added. So that's how we started. Right. And so that's how plenty of people who were into machine learning or to just big data processing. They got interested in us because we're the value add was that now they didn't have to maintain to configure all these Spark clusters. It automatically happened for them. They were automatically optimized right away. Rob Collie (00:53:05): So it's probably the wrong word, we'll get there by depth charging around it a little bit. So was Databricks when it first came out, was it a cloud-based like layer of abstraction and management? Denny Lee (00:53:17): Yeah. Rob Collie (00:53:17): ... across Spark nodes? Denny Lee (00:53:18): Exactly. That's it. Rob Collie (00:53:21): I I hit the sub with the first depth charge. Denny Lee (00:53:23): And all sources were assessed or passed, however you want to phrase it. Rob Collie (00:53:26): All right. So that's Databricks. Denny Lee (00:53:27): Yeah. And so just to give the context, every single technology that we're involved with, whether it's the advancements of Apache Spark to go from, for example, you had to write the stuff initially in Scala, which is how- Rob Collie (00:53:42): [inaudible 00:53:42]. Denny Lee (00:53:42): So credit to Scala... but then you had to write it in Python, right? But then over time we added data frames with the [Smart 2X 00:53:50] line, all of a sudden, now this concept of actually running in SQL, because that makes a lot more sense for everybody. And the Smart 3.0, which includes ability for the Python developers to interact with it better. Or that when we introduced Delta Lake or when we introduced MLflow, all of these different technologies were the realization of as we're working with more and more customers, what were some of the parts that really are needed for the whole community to thrive, irrelevant of whether they're on Databricks or not and which parts are going to be the parts where we will quote unquote, keep for ourselves because we're the value add. Denny Lee (00:54:28): We're going to provide you something valuable on top of said open source technologies, so that way you can still benefit from the learnings. So a [inaudible 00:54:38] and MLflow and Smart, for example, using those technologies, you can still benefit from everything we're saying. It's like, we're still publishing reams of white papers and blogs telling people how to do stuff, because that's the whole point. It's basically a large educational push. We want everybody to grok that there's a lot of value here and here's how to get that value. And then Databricks, if we can say, yeah, but we can make it faster, we'll make it simpler or make it whatever, that's where we will be valuable to you. Now, bring it back to the Databricks to the Lake House concept, if you think about purely from a, why did we use the term Lake House, what it boils down to is the whole value of a data warehouse was the fact that you could protect the data, you had asset transactions. Denny Lee (00:55:21): You could store the data, you could trust what was being stored and you could generate marts off of it to do your business dashboards, whatever else. That's the whole premise, this one central repository. Okay. So where Databricks came in, it was like, well, in a lot of ways, we gave away a cyclotron in the house. We gave away Spark for the processing. We gave away Delta Lake for the transactional protections. On the machine learning side, we're even giving away [inaudible 00:55:46] But the things that there's also all these other technologies like TensorFlow for deep learning, your Pandas, your scikit-learn for machine learning, all these other things, right? Denny Lee (00:55:53): There's all these other frameworks put together. So the premise is that as opposed to, we grew up where it was relatively unfragmented by the time that we got into database systems and SQL server and things of that nature. Right now the system is still massively fragmented with all these different technologies and it'll stay that way for a while not because there's anything wrong, but because we're constantly making advancements. So the value add, what we do is basically saying, "Okay, well, can you make everything I just said, simpler?" Number one. And then for example, now back to the in-memory column store, when you look at Databricks and we said, we have to make Spark faster. So we made Spark as fast as we possibly can. But the problem is that when you're running Spark in general, there's the spin-up time to build up your tasks, spin up time to run the jobs, spin up time to do the task. Denny Lee (00:56:47): There's an overhead. Now that overhead makes a ton of sense for what it's use cases for, which is, in this case, I have a large amount of data. I need to figure out how to process that. But what's a typical BI style? Of course, you already have it structured. You already know more or less what you have to work with. It's just go do it. We can push Spark to a point, but we can't get any faster because at some point literally the JVM becomes a blocker and the fact that we have this flexibility to spin up tasks to analyze the data becomes the blocker. We're not about to remove the flexibility of Spark. That seems silly. So what does Databricks do? And this is one of the many features of quote unquote, the Lake House that Databricks offers, which is, okay, we built a C++ vectorized column store engine. Denny Lee (00:57:36): So how can you do BI crews on your Data Lake? Guess what? We've written in C++. We're going back old school back to our day in order to be able to work around that. We can make some general assumptions about what BI queries are. So since we can make those assumptions, the aggregate step you're making, even distincts though, that's always a hard problem anyways, right? The joints that you're making, the group [eyes 00:57:58] you're making, right? You can make these types of assumptions. If you know the structure of the data, like in this case, because even with Delta, what's Delta? Underlies Parquet. Well, Parquet in the footer has a bunch of statistics. So that tells us basically, what's your min-max range. What is the data types you're working with? So you can allocate memory right from the get-go in terms of how much space you need to take up in order to be able to generate your column store, to do your summaries, to do your subs, your ads, or anything else. Denny Lee (00:58:24): So since you have all that in place, we can simply say, "Let's build a column store vectorized engine in C++ that understands the Spark SQL syntax. So the idea that you haven't changed your Spark at all. If the query you're running because of taking a UTF, because it's hitting whatever, doesn't get the photon engine. It's okay. We'll default right back to Spark and you're good to go and Sparks pretty fast. But if we can use the photon engine, bam, we're going to go ahead and be able to hit that and we can get the results back to you significantly fast. And so for us, this is an example of what I mean by us giving you value add. And the fact is, over time the value adds are going to change. And that's actually what we want. We want an advancement of the technology and we're taking the bet that we will always be able to go ahead and advance the technology further, to make it beneficial for everybody. Then it's worthwhile for you to pay us. Rob Collie (00:59:18): Photon, koala, Panda, Cloud [crosstalk 00:59:25] . You know what, Tom? Thomas Larock (00:59:26): Yeah. Rob Collie (00:59:27): More data platform technologies were invented in the last year than in all of human history. Thomas Larock (00:59:34): That might be true. [crosstalk 00:59:37]. Rob Collie (00:59:36): The real rate of expansion here is the number of technologies. We go decades where we have SQL and then someone comes up with OLAP and then like things like ETL come along. It's like four or five total things over the course of decades. Thomas Larock (00:59:55): They just get renamed. Rob Collie (00:59:56): Yeah. Yeah. A lot of the same problems keep rearing their head, but in a new context. Denny Lee (01:00:01): And exactly to your point, right. Asset transactions came back and rightly so. It rightly so came back. And like I said, what did we build photon on? We're pulling the old [inaudible 01:00:14] Mary Collins store stuff that we did, dude. We're pulling back to that day. Rob Collie (01:00:18): So Photon is that C++ written thing. Denny Lee (01:00:22): Yeah. Rob Collie (01:00:23): Okay. And that is similar to, in a lot of ways like the VertiPaq engine that underlies power- Denny Lee (01:00:30): Yes. Very similar. It's in memory called start engine, dude. Thomas Larock (01:00:35): So I got to drop here in a couple minutes, but I want to just say the benefit of us being old, I mean experienced, is that we recognize all this stuff has already happened. So I read this article a couple of weeks ago. This guy, it was a dev ops article and he's like, "We need a new dev ops database." And I look at that. I go, "What are you talking about? You need a new dev ops database?" And he goes, "Do you know how hard it is to merge changes when you're both trying to update the same database and you have to roll this thing back?" And I'm like, "Dude, this is a problem for 40 years." It's not that you need something new. It's like, you just need a better process for how you're doing your data ops. Your data ops process is failing you and it's not because of the database part of it. It's because of how you've implemented your pipelines and things. Thomas Larock (01:01:27): And I just sat there shaking my head and I go, the kid that wrote this, he's like 22 years old and he's never had this issue like with Denny has. What if you want to update and delete in this Spark group of clusters. He's never had to fight through that yet. So to him, it's all new. Right? And he's like, "Oh yeah, we totally need a new thing." And now he's going to go reinvent hipster, JSON and he's going to say, "Now I've solved this." And bam, now you've got a new standard. Right. And this is why we have so many different data technologies and names. Rob Collie (01:01:57): Just alienated every 22 year old techie on the planet. Do you know how hard it's going to be to get someone to buy you a hazy IPA now? Denny Lee (01:02:06): Yeah, I do. Thomas Larock (01:02:08): I'm okay with that. I'm in the scotch phase right now. Denny Lee (01:02:10): So fair enough. Fair enough. Highland park, by the way. Thomas Larock (01:02:14): Yeah. I'm good. I'm good with all that. So if you're 22 and listening to this, first of all, I'd say that's just not possible. Rob Collie (01:02:24): Yeah. You're probably not, but hey, if you are a 22 year old listening to us, let us know. Denny Lee (01:02:29): Yes, please. Rob Collie (01:02:31): Tom wants to buy you a scotch. Denny Lee (01:02:33): Exactly. There you go. But also I do want to emphasize the fact that for folks that are old-school data based types, the fact is that you're listening to three of us. World school database [inaudible 01:02:48] and a lot of the, maybe not the technology itself, but a lot of the concepts, a lot of the processes are very much applicable today as they were 20 years ago. Just because you're older does not mean you can't keep up with this technology. You just have to recognize where the old processes actually are still very, very, very much in play today. Thomas Larock (01:03:15): I can vouch for that as I've been trying to venture more into data science. My experience as a data professional and my math background as well, has blended together to make it an easy transition where... And I'm looking, I go, this ain't new. Denny Lee (01:03:29): Exactly. Thomas Larock (01:03:30): Kind of the same thing. Rob Collie (01:03:31): Let's change gears for a moment because I'm getting closer to understanding what you're up to. All of this Linux world stuff, that's the world you run in these days. Denny Lee (01:03:42): Yeah, that's what they tell me at least. Rob Collie (01:03:45): For a lot of people listening, I suspect that this sounds like a completely alternate universe. Denny Lee (01:03:52): Fair enough. Rob Collie (01:03:52): A lot of people listening to this are one form or another like Power BI professionals and very, very much up to their eyeballs in the Microsoft stack, which I know isn't mutually exclusive with what you all are working on. That's one of the beauties of this brave new world. However, a lot of people don't have any experience with this even though they're doing very, very, very sophisticated work in data modeling, DAX, M. They're hardcore professionals. And yet a lot of this stuff seems like, again, like a foreign land. So where are the places where these two worlds can combine? What would be the relevance or like the earliest wave of relevance. What's the killer app for the Power BI crowd? Denny Lee (01:04:38): Yeah, so the Power BI crowd, what it boils down to is, whatever's your source of data. If you're talking about a traditional, like hitting a SQL server database, yeah, you can... Well, 99.9% chance you can trust it, right? There was transactional protection. The data wasn't there. You're not getting dirty reads unless you, for some reason, want them you're good to go. But what ends up happening to any Power BI user, is that you're not just asked to query data against your database anymore. You're asked to query against other stores and you're asked to query against your Data Lake. The Data Lake is the one that contains the vast majority of your data. There's no maybes here. It is the one that contains the most of your data. So the problem underlying which isn't obvious, and I'll give you just a specific example to provide context and I'll simplify. Denny Lee (01:05:30): Let's just pretend you've got a hundred rows of data. And then somebody decides to run an update statement. Now, in the case before Delta, how you run an update statement, let's just say the update affects, let's just say 20 many rows. You actually have to grab all 100 rows, rewrite them down, rewrite 80 of them. Take the 20 rows that you need to update, rewrite them. They're into a new table. Once you validate that it works, delete the original table, rename the new table back to the old table. And now you have the correct number of rows you want with the 20 rows as an update. So far so good. What happens if you're trying to query that at the exact same time when it's mid-flight? Thomas Larock (01:06:21): What happens? Denny Lee (01:06:22): Exactly or what happens if you are trying to go ahead and create when multiple users are trying to do the same thing, because for sake of argument, two people are trying to run the same 20 row update at the exact same time and because it's in` mid-flight, neither one knows which one is the primary. So that means the data you get, you can't trust if there's anything that's done to it. Maybe a deletion happened. And also now there's only 80 rows. Right? And they're going, "Okay, so which one is it?" Or the worst scenario. You take the hundred that you had, it fails, mid-flight. It wrote the 80 rows down into the same folder. So what you end up having when Power BI is trying to query it, it's not getting a hundred rows. First of all, it's getting a hundred of the old rows, it's getting 180. So now your numbers are completely off. And so that's the context that when you had a database the transactions, you didn't have to worry about that. So what's the value of the Data Lake, having Delta Lake? Denny Lee (01:07:32): That's exactly it. Having that ability to protect the data. So even if it was mid-flight writing and it failed, it doesn't matter. There's a transaction log that states very clearly, "Here are the 100 rows that you care about. And it's really the files by the way, that's in the transaction log. But let's just for sake of argument, here's the five files that have the 100 rows you care about and that's it. The only reason I'm calling that stuff out is because this is also very important for clout. So when you go ahead, whether it's Windows write DIR or Lytics you write LS, right, that's a pretty fast operation. When you run that same command, LS on a cloud object store, it's actually not a command line operation that you're used to. What it is, is actually translation to a bunch of REST API calls. And so the REST API calls basically underneath the covers is basically a distributed set of threads that go out to the different parts of the storage and return that information back to you. Denny Lee (01:08:34): Now for five files probably it doesn't really matter, but if you're talking about hundreds or thousands of files, just listing the files takes time. So just running the LS, isn't going to come back in seconds. It will take many seconds to minutes just to get that query back. So how Delta Lakes solves that problem, it says, okay, wait, no, it's okay. In the transaction log, here are the five or 100 files that you need. So there's never a listing. So whether it's Spark or any other system that's querying Delta Lake, the transaction logs telling you, "No, here's the five/100 whatever number of files that you actually need. Go grab them directly." Please don't forget. A cloud object store itself is not a traditional obstacle. This idea of bucketing this idea of folders. The folders don't actually exist. It's just something that you have to parse. It's just one gigantic blob of crap. That's all it is. So what happens is they actually have to basically parse the name of the files in order to return that to you and then claim there's a folder in there. Rob Collie (01:09:36): It's kind of like schema on read, right? We're going to give you this notion of directories, but it's created from thin air. Denny Lee (01:09:41): Exactly. It literally is. Yeah. So because of that, then the whole premise of saying, "Okay, well now I can return that stuff to you that much quicker." And then there're other aspects of Delta Lake, for example, like schema evolution and schema enforcement. The idea that if you've already declared that this is the schema, like as in, I've got two wins in a string column, let's just say. If you try to insert, update into that table, and it's not two string, maybe it's two [inaudible 01:10:09] strings, let's just say, it'll say, "No, I'm not going to let you do it because I'm enforcing the schema," just like a traditional database staple would. But also we allow for schema evolution, which is, if you then specify, no, no. In this case, allow for evolution, go ahead and do it. Right. So it gives you the flexibility while at the same time giving that structure. Rob Collie (01:10:32): It's almost like even... There's a really simple parallel here, which is like an Excel validation. You can say, "Here's the list of things you can choose from," but then there's also allow people to write in new values or no? Denny Lee (01:10:44): Right. Rob Collie (01:10:46): So there's that nature of flexibility. It's either hard enforce schema or evolvable. Denny Lee (01:10:52): Exactly. No. And that's exactly right. So there are many factors like that. I can go on about streaming and batch and all these other things, but the key aspect, what it boils down to, for any Power BI user, any Power BI professional is this notion that the Data Lake, without asset transactions, without things like schema force, but without inherently is an untrustworthy system for various reasons. Rob Collie (01:11:19): Totally. Yeah. I mean, again, this is the jaw dropping thing for me. It's like really? That's been okay ever? Even for five minutes, that was okay? It's really hard to imagine. Denny Lee (01:11:29): Well, I mean, the context, don't forget is because you're at the tail end of what has happened to that data. The people that made this decision were on the other end, which is, I have so much data coming into me at such a disgustingly fast rate. I just need some way to get it down. Otherwise, I will lose it. Rob Collie (01:11:54): Yeah. It's a real immediate problem. And earlier you very graciously grouped me in with yourself and Tom, when you said, we're all old database people, but I was never really a database person. I've always been a lot closer to the faucet than the plumbing. Denny Lee (01:12:10): No, fair. Rob Collie (01:12:11): And so for the Power BI crew that's listening, which is again, closer to the faucet, we could say, "Hey, this is not our problem. It's not actually something that most Power BI people are going to be dealing with." It's going to be an infrastructural decision made. It is very much an IT style decision as opposed to this new hybrid model of what we used to call self-service. But it's really like hybrid model of Agile business driven faucets. But if my IT organization decides that the Data Lakes that I've been using to power some of my beautiful Power BI, it might be that I've been quietly... And this is the scary part, unknowingly suffering the consequences of crunchy rights that are conflicting with one another. Rob Collie (01:13:01): I might've been dealing with bad data and not known it, but if my IT organization decides to roll out something like Delta Lakes, do I notice? I mean, other than the fact that I won't have bad data anymore, will I need to do anything differently? Denny Lee (01:13:17): No. Rob Collie (01:13:18): Do I need to query it differently through- Denny Lee (01:13:20): Nope. Nope. Rob Collie (01:13:21): Or is it just the people who are doing the rights that have to play ball? Denny Lee (01:13:25): The way I would phrase it is this way. It's the traditional Power BI reporting problem. Why you have to care, which is the problem isn't so much that you're supposed to tell infrastructure what to do. The problem is you're going to get blamed when the numbers are wrong. Rob Collie (01:13:42): Sure. Denny Lee (01:13:43): Right. And you're the first line of people that will be attacked. Rob Collie (01:13:51): [crosstalk 01:13:51] comes out of the faucet. Right? Denny Lee (01:13:53): Yep. Rob Collie (01:13:53): I give cups of that to the rest of the team. They're going to say, "Hey, you gave me bad water." Denny Lee (01:14:01): That's right. Rob Collie (01:14:02): And I'm not going to be able to talk about the plumbing that they don't see because it's all behind the wall. It's just the faucet. Denny Lee (01:14:08): Exactly. But at the same time, exactly to your point. When you're running that query, for example, using Spark or for that matter, anything that table will talk to Delta Lake, no. Nothing changed. The only thing that changed, which is a benefit and not a con, is if you want to go back and look at historical data, Delta Lake includes time travel as well. So- Rob Collie (01:14:34): Snapshots. Denny Lee (01:14:34): Yeah. Snapshots of the previous time. So if you want to go, you can just append, like for example, you run a smart SQL statement, which is very close to SQL tables. Select column A, B whatever from table A. So then you can basically that's what your normal Power BI source query would be. Well now, just for sake, however you want to look at the snapshots, select star column from table A version, as of whatever version you want to look at. Rob Collie (01:15:01): This is another old familiar problem. So for example, our Salesforce instance, it does have some history to it, but like it's not... Or let's take an even easier example, like QuickBooks. QuickBooks is almost inherently a system that's about what's true right now and to do any trending analysis against your QuickBooks data, it's hard, right? You've got to be doing snapshots somehow. And so to run our business, we have multiple line of business systems that are crucial to our business and we're pushing snapshots into, most of the time, I think Azure SQL in order for us to be able to keep track of where we're trending and all that kind of stuff. So you're talking about that Delta Lake Lake House, give me this, snapshotting against my stores. And I don't think it's just Sparks stores. Right? It's all kinds of stuff, isn't it? Denny Lee (01:15:59): Spark is the processing engine. Right? Sort of the query agent. Delta Lake is the storage layer, basically the storage layer on top of typically Parquet. So that's the context. So the idea is that you're basically reading the Parquet file. We have other copies of the Parquet files that allow you to basically go through the snapshot. The transaction to log tells you which files correspond to which version of the data you have. Rob Collie (01:16:23): So the snapshotting, the time travel thing, right, that's a benefit that I could gain and really use as a Power BI. Yeah. That would be a noticeable difference, right? Have you kept up with Power BI very much at all. I'm wondering if in your world, if I'm using Power BI and a lot of the data that I need is stored in a Denny style world rather than SQL- Denny Lee (01:16:49): And the SQL [inaudible 01:16:50]. Rob Collie (01:16:50): Is your expectation that the import and cache mode of Power BI is still very much relevant in your world or would it only be considered cool if I was using direct query? Denny Lee (01:17:01): Oh no, no, no, no. Whether I'm using direct query, whether I'm using import, it's always going to be a product of what is your business need. For example, if SQL server suffices for everything you're doing because of the size, because whatever else, I'm the last person to tell you to go do something else. Wait, come on. We're SQL server guys. Right? So no, I'm not going to do that. That's ridiculous. Right. What I'm talking about is very much into, no, you have a Data Lake or you need one. How to get the maximum out of it. That's literally where my conversation is. In other words, for sake of argument, IT had structured it such that the results of the Data Lake go into your SQL server and then you can query your SQL server to your heart's content. Cool. I'm not saying you're supposed to go to Delta Lake directly. Denny Lee (01:17:48): I'm saying, whatever makes the most sense, because for example, I'm making this up scenario up obviously, but direct query would make sense, for example, if I have constant streaming data. I want to see at that point in time, what the change was from even a second ago or even a minute ago. Okay. Well, Delta Lake has transactional productions such that when the data is written at that point in time when you execute that... I'm using Spark SQL, as example, as the Spark SQL statement, we know from the transaction log what files have in fact been written. We will grab the files as at that point in time. So even if they're half-written files in there, it's not going to get included because it wasn't written. So then direct query for your Power BI to go grab that data. No problem at all. By the same token you turn off. I was like, "Yeah, but I don't need streaming. I just need to go ahead and augment my existing SQL table with another fact table or with a dimension table." Cool. Hit that. Rob Collie (01:18:43): Am I going to benefit from the photon engine if I'm using Power BI? Would my direct queries run faster? Denny Lee (01:18:50): Absolutely. Rob Collie (01:18:50): ... as a result? Denny Lee (01:18:50): And that's exactly the context, at least from a Databricks perspective, that's the whole point. You can take your Power BI queries and you can go ahead and run them directly, get your Data Lake with the same Spark SQL statements that you were originally running. Except now they're faster because they're using the photon engine. Rob Collie (01:19:06): Awesome. Even if I'm in import mode in Power BI, like the data refresh could also- Denny Lee (01:19:11): Exactly [crosstalk 01:19:12] would be significantly faster because now I can get the data to you significantly faster than before. Rob Collie (01:19:17): I think this has been an amazing tour of this. I would love to have you come back. Maybe we do a series, which is Denny explains it all, right? And when I say it all, I mean the domain of the Linux cool kids, Denny Lee (01:19:33): I thought we were going to talk about coffee. Rob Collie (01:19:37): Well, so I was actually... That's written down next, is espresso. Denny Lee (01:19:41): Good. Rob Collie (01:19:41): I wanted to get into that. So there's a scene in Breaking Bad where Walter White meets Gil and Gil has been like this master chemist. Gil has been working on the perfect coffee. And Walter is really obsessed about really getting into this giant lab and building and making his blue perfect mix at an industrial scale. He couldn't be more excited and yet he stops and goes, "Oh my God, this coffee, why are we making meth?" Denny Lee (01:20:12): Exactly. Yeah. Yes. Thomas Larock (01:20:15): So, yeah. I agree with you wholeheartedly. Rob Collie (01:20:18): You seem like the Walter White that decides, "Nah, you know what, it's going to be [crosstalk 01:20:24] because I watch even your latte art game. I've watched it evolve over the years. You were in like when Casper moved to Redmond, I remember him like touching base with you and getting the official espresso, like [Kit 01:20:39] so, here I am in Indiana, I'm years behind you in the espresso game. And so we just splurged for one of the automated, like espresso and latte makers from DeLonghi or whatever. And every time I tell that thing to make me a latte and it ends up like this white foam with a vampire bite where the two spouts of espresso came into it. Every time I do that, I think about you and these works of art that are crafted with... Like you got to use the word like artisanal and handcrafted and I'm pushing a button and I'm getting this monstrosity. I just go, "Maybe Denny 15 years ago would have been okay with us. But Danny of today would be very, very, very upset." Denny Lee (01:21:22): That's true. I mean, I am from Seattle. So you have to admit many of us here are very OCD. So that's why I fit in very well, for starters and saying, well, you do know this is like, again, for those of you who may not know, Seattle is a very much a coffee town to put it rather lightly. Rob Collie (01:21:42): Anytime you have a people that live under a one mile thick cloud blanket, nine months out of the year... Overcast days, don't even talk about overcast days. Like supposedly they use this metric for cities across the U.S. like how many overcast days per year? But they do not grade the overcast days by intensity. Right? And so supposedly like Cleveland has like as many overcast days, whatever is Seattle. No, no. That is bullshit when you're on the ground. So yeah, when you live under that oppressive blanket, you're going to need as much caffeine as you can lay hands on. And this is why Seattle is a [crosstalk 01:22:19]. Denny Lee (01:22:19): Oh, yeah. Well that and also don't forget, we are also known for being a rainy city, but actually there are sections in Nevada that actually received more rain than us. Rob Collie (01:22:28): Well, in terms of inches of rain. Again, like I grew up in Orlando, there's more rain there too. Right. It's just that in Seattle, it falls in a constant mist forever. Denny Lee (01:22:37): But don't forget. That's why people like me love it because you got a Gore-Tex jacket, eh, whatever. We don't care, we just don't care. But back to the coffee thing, because you know, I'm going to OCD on you. Yes. I'm glad to, at any point in time, dive into all the particulars. Rob Collie (01:22:58): We want to have you back on for sure. And we're going to make time to talk about your coffee... What am I going to call it? Like your rig? We need to talk about what your set up is like we're talking to The Edge from U2 about his guitar affects, right? We need to know. Denny Lee (01:23:16): You got it, dude. You know full well it'll be pretty easy to convince me to talk about data or coffee. So that's what we're going to do. Rob Collie (01:23:24): Seriously. If you're open to it, I'd love to do this together because there's so many things we talked about that we didn't have a chance to like really explore. Denny Lee (01:23:30): We left the wording about dynamic structured cache. I still haven't addressed that. The reason why I'm saying I agree with structures because that's the whole point. The data that comes in may not be structured, but just to your point, when I want to create, when I want to make sense, I want to see those rectangles. I don't want to see a bunch of circles, stars, triangles. I need those rectangles so I can actually do something with that data. That's what it boils down to. Whether it's Coriant processing ETL, machine learning. I don't care. I just need to be able to do some of that. There has to be a structure to that data first before I can do anything with it. So the structure agree. The reason I'm saying cache, I don't agree with completely is because the point is, cache is meant as when you hit a final state and you're trying to improve performance. Denny Lee (01:24:20): This is where cache is actually extremely beneficial. I'm not against caches, by the way, I'm just saying, but the reason why I'm saying it can't be a cache is because you're going to do something with that data from its original state to getting to a structured state. And then in fact, you may go ahead and do more things, standard part of ETL process. If you still remember our old data warehousing days where we have the old TP transactional database that goes into a staging database that goes into data warehouse before it goes into [NOLA 01:24:49] queue, right? It's analogous to this concept of data quality, right? The data is dirtier in the bating, or at least not structured the way for purpose of analysis at the beginning and overtime from [inaudible 01:25:01] TP to staging to data warehousing, it gets closer to a format or structure that is beneficial for people to query and to ask questions, right? Denny Lee (01:25:09): The same thing for a Data Lake. Often we talk about it as the Delta medallion architecture, but it's a data quality framework. The bronze silver gold concept. Bronze is when the data is garbage in garbage out silver's when I do the augmentation and transformation of it, gold is when it's proper feature engineering for machine learning or aggregate format for BI queries. Okay. But irrelevant of how I define or what wording I use, that's how to cache. I have to put that data down in state for the same reason I had to do it with LTP staging data, where I was the biggest. How about if there had to be a change upstream? If there's a change to the [LTP 01:25:44] database, I need to reflect that to staging and data warehousing. If I don't have the original LTP data, I can't go ahead and reflect that into the staging and data warehousing. Denny Lee (01:25:52): If I need to change the business logic, I need to go back to the original TP source so I can change it downstream into your staging, into your data warehouse. Same concept with the Data Lake. I'm going to need to go back to the original data based on new business requirements and reflect that change. So that's why it's not a cache because I need it stateful so I can do something with it. Ultimately, you want to make sure as the Power BI pro, the data that you're showing to your end users is as correct as you could possibly be. So whether it's technology or process, you're going to need both to ensure that, and this is what we've been discussing today is the ability for your Data Lakes to have that now. Rob Collie (01:26:40): All right, this is so good. I'm glad we had this [inaudible 01:26:43] I'm glad we made the time. Thank you so much. Denny Lee (01:26:44): Thanks buddy. Announcer (01:26:45): Thanks for listening to The Raw Data by P3 Adaptive podcast. Let the experts at P3 Adaptive help your business. Just go to P3adaptive.com. Have a data day.

It's not every day that you can hear a great conversation with the Head of Product of Excel. Brian Jones sits down with us and talks about the past, present, and very promising future of Excel. Rob and Brian go way back, and the stories and laughs abound!   Check out this cool World Orca Day Excel template for kids!   Episode Timeline: 4:00 - Brian's lofty title is Head of Product at Excel, The importance and magic of Excel, and people's a-ha moments with Excel 20:25 - The difficulty of not seeing your projects' impact on the world and how the heck does Bluetooth fit into the story?!, Rob and Brian reminisce with some funny conference stories 32:00 - The XML file format and some very neat XML tricks that everyone should know about 51:25 - The birth of the Excel Web App and Rob can't believe some of the things that Brian's team has done with Excel 1:05:00 - How to onboard the Excel, VLOOKUP, and Pivot crowd into data modeling and Power BI, and the future of Excel most certainly includes the Lambda function (maybe!) Episode Transcript: Rob Collie (00:00:00): Hello, friends. Today's guest, Brian Jones, head of product for this thing you might've heard of called Microsoft Excel. Brian and I go back a long way. We were both youngsters at Microsoft at the same time, and we both worked on some early features of Office apps, and we're friends. Really, really have sincerely warm feelings about this guy, as you often do with people that you essentially grew up with. And that's what we did. When Brian and I first worked together, he was working on Word and I was working on Excel. But even though Brian was on Word at the time, he was already working on what we would today call citizen developer type of functionality in the Word application. So even though we were essentially on different sides of the aisle within the Office organization, we were already finding ourselves able to connect over this affinity for the citizen developer. Rob Collie (00:00:55): Now we have some laughs during this conversation about how in hindsight, the things he and I were working on at the time didn't turn out to be as significant as we thought they were in the moment. But those experiences were very valuable in shaping both of us for the initiatives that came later. Rob Collie (00:01:11): Like almost everyone at Microsoft, Brian has moved around a bit. He's worked on file formats for the entire Office suite, which ended up enabling Power Pivot version one to actually function the way that it should. He's worked on Office-wide extensibility and programmability, back to that citizen developer thing again. And in that light, it's only natural that Excel's gravity reeled him in. And in that light, it's only natural that someone like that, someone like Brian, found his way to Excel, and it really is a match made in heaven. And if you permit me the Excel joke, that turned out to be a great match. Rob Collie (00:01:50): We took the obligatory and entertaining, I hope, walk down memory lane. We spent a lot more time than I expected talking about file format. And the reason why is that file formats are actually a fascinating topic when you really get into it. Lot of history there, a lot of very interesting history and challenges we walked through. And of course, we do get around to talking about Excel, its current state, where it's headed, and also the amazing revelation for me that monthly releases actually mean a longer attention span for a product and how we ended up getting functionality now as a result of the monthly release cycle that would have never fit into the old multi-year release cycle. We were super grateful to have him on the show. And as usual, we learned things. I learned things. I have a different view of the world after having this conversation than I did before it, which is a huge gift. And I hope that you get the same sort of thing out of it. So let's get into it. Announcer (00:02:56): Ladies and gentlemen, may I have your attention please? Announcer (00:03:03): This is the Raw Data by P3 Adaptive podcast, with your host Rob Collie and your cohost Thomas Larock. Find out what the experts at P3 Adaptive can do for your business. Just go to p3adaptive.com. Raw Data by P3 Adaptive is data with the human element. Rob Collie (00:03:26): Welcome to the show. Brian Jones, how are you, sir? Brian Jones (00:03:30): I am fantastic. Thank you for having me, Rob. I'm excited. Rob Collie (00:03:33): So let's start here today. Well, you and I go way back, but today, what's your job title and what are your responsibilities? Brian Jones (00:03:42): So today, my job is I'm the head of product for the Excel team. So I lead the team of product managers that are tasked with or given the honor of deciding the future of Excel, where we go with Excel, what are the set of things that we go and build Rob Collie (00:03:59): Head of product. That's a title that we didn't have back when I was still at Microsoft. We did at one point have something called a product unit manager. Is it similar to that? How does that relate? Brian Jones (00:04:11): That's a good question. So we're continuing to evolve the way that we use titles internally. So internally, we have titles that still for most folks externally don't make any sense, like program manager, group program manager, program manager manager, director of program manager. And so for externally, whenever I'm on LinkedIn or if I do PR interviews, things like that, I use the term head of product. Internally, we don't have the term head of product. Rob Collie (00:04:37): Okay. All right. So that's a translation for us. Brian Jones (00:04:40): Yes, exactly. Trying to translate the Microsoft internal org chart to something that makes more sense to folks. Rob Collie (00:04:49): Yeah. So things like, if we use the word orthogonal, what we're really saying is that's not relevant. Brian Jones (00:04:53): Exactly. Rob Collie (00:04:54): That kind of decoder ring. Brian Jones (00:04:57): I didn't realize orthogonal [inaudible 00:04:59] until you said it and I'm like, " Oh yeah, no. Of course, that is completely a ridiculous term to use." Rob Collie (00:05:03): Or I don't know if they still do this, but an old joke that Dave [Gayner 00:05:07] and I used to have, it was all his joke at the time. It was big bet. Do we still talk about big bet? We're going to place a big bet. Brian Jones (00:05:14): Yep. Big bet or big rocks. Big rocks. You know the- Rob Collie (00:05:17): Big rocks. Whoa. Brian Jones (00:05:18): Yeah. It's kind of an analogy. You've got a jar and you want to fill it with the big rocks first, and then you let the sand fill in the rest of the space. So what are the big rocks? Rob Collie (00:05:26): Okay. Yeah. But big bet was one that we used to always make fun of. Brian Jones (00:05:31): Especially when there'd be, "Here are the big bets," and there's 20 of them. Rob Collie (00:05:34): Yeah. The joke I think we used to make was we would call something a big bet when we really didn't have any good reason for doing what we were doing. Anyway, all right. So you're head of product for Excel. That is a pretty heady job. That's pretty awesome. Brian Jones (00:05:52): It's a pretty fun job. Absolutely. Rob Collie (00:05:54): I mean, you're not lacking for eyeballs in that business, are you? We're all friends here. We're all on the same side of this story. I mean, it is the lingua franca of business, Excel. It is the business programming tool. People don't necessarily think of it as programming, but formulas are a programming language. To be head of product for the platform, you could call it an application, but really it's probably more accurate to call it a platform that is, I think, is the single most critical platform to business in the world. That's pretty amazing. Brian Jones (00:06:30): Absolutely. And that's usually the way that we talk about it internally. It depends on who your audience is externally when you're talking about it. But yeah, Excel is a programming language. I remember even before, back when I was on the Word team, but I would go and meet with PJ, who ran program manager for Office all up. And he'd always referred to Excel more as an IDE. And that didn't totally resonate with me at the time because to me, Excel was just a list app, an app for just tracking things. I didn't totally understand what he meant by that, but I'd nod cause he was super important and smart. And it wasn't really until I started working on the team that I was like, "Oh, I totally understand all these things that PJ used to reference." Rob Collie (00:07:06): This one of the things I had been dying to ask you is when you and I first met, I was working on the Excel team, but still had... Gosh, this was year 2000 maybe, maybe 2001. And even though I was nominally part of the Excel team at that point, I still didn't really know Excel, and you were working on Word. So the thing we both had in common at that point is that we didn't know Excel. So I wanted to get your perspective. I know that you've done some things other than Word, but we were already sort of teasing this. So let's just get into it. What's it like to come from "outside" Excel and how's that transition? How do you view Excel differently today versus what you did before? We already started talking about that. The list keeper. That's very common way for people to view it. Brian Jones (00:07:53): When I first started, yeah, I was on Word, although I was working on more kind of end user developer type of pieces of Word. That's how you and I first interacted because we were talking about XML. The first feature I owned was a feature called easy data binding to Excel. And the whole idea was when you could easily bring content from Excel into Word, but then create a link back so that the content in Word would stay live. And a lot of this stuff that I did while I was on Word was all about trying to make Word a little bit more of a structured tool so that people could actually program against it because Word is completely unstructured. It's just free-flowing text. So trying to write a solution against that is almost impossible because you can't predict anything. So we did a lot of work to add structure, whereas Excel out of the gate has all that structure. So it's just much easier to go and program. Brian Jones (00:08:39): If I had gone straight from Word to Excel, it would have been a little bit more of a shock, but I actually had about eight years in between where I was running our extensibility team. So a lot of the work we would do was revving the add-in model and extensibility for Excel. So I got some exposure there. When we did all of the file format stuff and the whole file format campaign, That was a couple of years where I was working really closely with a bunch of folks in Excel, like Dan [Badigan 00:09:06] and folks like that. So I had a bit of exposure, but I'll tell you when I first joined, I had a similar job, but it was for the Access team and we were building up some new tech. Brian Jones (00:09:17): Some of it still is there today. Office Forms came out of some of the investments that we were doing in Access. But when I showed up into Excel, I was very much in that mode of, "Why don't the Excel folks, get it? Everything should be a table with column headings." And like, "That's the model. And why do they stick with this grid? Clearly word of it is eventually going to go away from the printed page as the key medium. Excel's got to go away from the grid. And they've got to understand that this should just be all tables that can be related." And thankfully, I was responsible when I joined and didn't try and act like I knew everything. So I took some time to go and learn. Brian Jones (00:09:52): And it didn't take me long. We have some crazy financial modeling experts on the team and stuff like that, where I'd say it was maybe six months in that it clicked for me where I understood those two key pieces. The grid and formulas are really the soul and the IP of Excel. The fact that you can lay out information really easily on a grid, you have formulas that are your logic, and you can do this step-by-step set of processes where each cell is almost like another little debug point for you. [Cal captain sub 00:10:20] second, and it's the easiest way to go and learn logic and how to build logic. Brian Jones (00:10:25): I didn't get any of that at that time, but you pick it up pretty quickly when you start to look at all the solutions that people are building. And now, obviously, I've been on the team now for five years, so I'm super sold around it. But I'd say it took me a little while and I'm still learning. It takes a while to learn the whole thing. Rob Collie (00:10:41): Yeah. It's funny. Like you said, Word's completely unstructured. You're looking in from the outside and you're like, "Well, Excel is completely structured." Then you get close to it. You're like, "Oh no. And it's not, really." Brian Jones (00:10:52): No. Not at all. Rob Collie (00:10:53): I mean, it's got the cells. Rows and columns. You can't avoid those. But within that landscape, is it kind of deliberately wild west? You can do whatever you need to. You're right. Okay. So tables, yes. Tables are still very important. But you've got these parameters and assumptions and inputs. And what do you do with those? I mean, they're not make a table for those. Brian Jones (00:11:19): Yep. Absolutely. I think that the thing that I started to get really quickly was the beauty of that. Like you said, it's unstructured. You have nice reference points. So if you're trying to build logic, formulas, you can reference things. But there's no rule about whether or not things go horizontally, vertically, diagonally, whatever. You can take whatever's in your mind that you're trying to make a decision around and use that flexible grid to lay it out. It's like a mind map. If you think about the beauty, the flexibility of a mind map, that's what the grid is. You can go and lay out all the information however it makes the most sense to you. Brian Jones (00:11:53): Really, that's what makes Excel still so relevant today. If you think about the way business is evolving, people are getting more and more data, change is just more constant, business processes are changing all the time. So there are certain processes where people can say, "This thing is always going to work the same way." And so you can go and get a vertical railed solution. That's why we use the term rail. That's kind of like if I always know I'm going to take this cargo from LA to San Francisco, I can go and build some rails, and I got a train, it'll always go there and do the same thing. But if business is constantly changing, those rails are quickly going to break and you're going to have to go off the rails. Excel is more like a car than a train. You can go anywhere with it. And so as the business processes change, the people who are using Excel are the same people who are the ones changing those business processes. Those are the business folks. And so they can go and evolve and adapt it and they don't have to go and find another ISV to go and build them another solution based on that new process that's probably going to change again in six months. Thomas Larock (00:12:52): So Brian's been in charge for five years of Excel, and he's sitting there telling us how there's still more to learn. And two weeks ago, we all got renewed as MVPs. And so I was on the MVP website, and I'm going through all the DLs I can join because that's all a manual process these days. I'm like, "Oh, there's the Excel MVP DL. I don't know why I haven't joined this yet." So I click. I'm immediately flooded with 100 emails a day. 100 emails a day. Now, I don't believe I am a novice when it comes to Excel. I don't. I know I'm not on you all's level at all when it comes to it. You build and work and live the product. But I know my way around enough that I can explain things to others when they say, "I'm trying to do this thing." "Oh, I think it's possible." Thomas Larock (00:13:40): But I read these passionate MVPs that you have and the stuff that they highlight, and it's not complex stuff. It's like, "Hey, this title bar seems to be wider in this." And I'm like, I might not even notice this stuff. And I see these features that aren't a complex feature, but I'm like, "I didn't even know that was there. I didn't even know you could do that. Oh, you can do that too." There's so much. And like you said, it's a programming language. It's an IDE. It's all these things. As [Sinopski 00:14:10] said, "It's the killer app for Windows." To have the head of product say that, there's just so much. He really means it. There is a lot to it. And it is something that is malleable and usable by hundreds of millions of people a day. Brian Jones (00:14:25): Yeah. Rob Collie (00:14:26): My old joke is, if you want to know how good someone is at Excel, just ask them, "How good are you at Excel?" And then take their answer and invert it. Brian Jones (00:14:37): That's absolutely true. Rob Collie (00:14:38): If someone says, "Yeah, I'm really good at it," You know they don't have any clue because they haven't glimpsed the depth of that particular mine shaft. And once someone has been to the show, they know better than to oversell their knowledge because they know they can't know everything. Rob Collie (00:14:54): You say you're good at Excel. And then the very next question is one that you're not going to be able to answer. So you got to be careful. [inaudible 00:15:00] person views Excel as Word with a grid. And that's not obviously what it is, but that's the oversimplification for... I don't know... maybe 80% of humanity. Brian Jones (00:15:10): Yeah. And the thing is, there's a lot more that we're doing in the app now to try and make it, one, more approachable, because there's a set of folks that just find it really intimidating, for sure. You open it up and it's this huge, dense grid. Like, "Hey, where do I start? What should I go and do? I've never even heard of this thing before." In the past, a lot of stuff that we would do, we never really thought about those first steps of using the app because we were always like, "Well, everybody knows our app. We're going to go and do the things for everybody that knows our app." And I think we're doing a better job now trying to think, "Well, there's a bunch of people who don't know about our app. Let's go and figure out what the experience should be like for them." Brian Jones (00:15:43): But we've done a lot with AI where we're trying to get a little bit better about... We look at your data. Recommend things to you. So we'll say, "If you've got a table of data, hey, here's a pivot table." You may not have even heard of the pivot table before. So really more like, "Hey, here's a summary of your data." You want to go and insert that. Brian Jones (00:16:00): In fact, those tests are always fun because then we get to work with people who've really haven't ever used a pivot table. So it's always fun to hear the words that they use to describe what a pivot table is. It's like, "Oh wow, you grouped my data for me." Or stuff like that like, "Wow. That's a nice name for it too." So we're trying to do more of that to expose people to really those higher-end things. But those things where for those of us that use it, once you discover that stuff, you're even more hooked on the product. You're like, man, that first experience of somebody built a pivot table for you and you realize, "Oh my God, I didn't know I could do this with my data. Look how much easier it is for me to see what's going on," and trying to get more people to experience that kind of magical moment. Thomas Larock (00:16:39): Now imagine being me and only knowing pivot through T-SQL and that magical day when you meet Rob and he's like, "You just pivot table [inaudible 00:16:49]." And you're like, "How many hours have I wasted? Why didn't someone tell me?" Brian Jones (00:16:56): Yeah. We get that a lot when we'll go and show stuff. Oftentimes, the reaction is more frustration. "I can't believe I didn't know about this for the past five years." Rob Collie (00:17:05): We get that all the time now with Power Pivot and Power Query and Power BI in general. The target audience for that stuff hasn't been really effectively addressed by Microsoft marketing. But even back, just regular pivot tables, such a powerful tool, and so poorly named. You weren't around on the Excel team, Brian, when I waged a six-month campaign to try to rename pivot table to summary table. Brian Jones (00:17:31): Oh really? Rob Collie (00:17:31): Yeah. Brian Jones (00:17:31): How long ago was that? Rob Collie (00:17:33): Oh, well, it was a long time ago. I mean [crosstalk 00:17:35]- Brian Jones (00:17:36): Pivot tables had already been out for quite a while. Rob Collie (00:17:37): Oh God. Yeah. I mean, they were long established. They were in the product. I didn't even know what they did. Believe it or not, I worked on the Excel team for probably about a year before I actually figured what pivot tables could do. People would just throw it around all the time on the team like, "Well, once you have the data, then you can chart it. You can pivot it," blah, blah, blah, blah, blah. And so I would fit in- Brian Jones (00:17:58): You would nod? Rob Collie (00:17:59): I would fit in... I would also author sentences like that, that had the word pivot in it. It was a pretty safe thing to do. There was no downside to it. But believe it or not, the time that I discovered what pivot tables are for... you'll love this... I was trying to figure out how to skill balance the four different fantasy football leagues that I had organized within the Excel and Access team. I wanted to spread it out. Levels of experience. I've got this table of data with the person's name and their level of experience and my tentative league assignment. And just this light bulb went on. I'm like, "Oh my God, I bet this is what pivot tables are for." Total expertise by league. Like, "Oh, look at that. It's totally it." That was a big change for me. That was during the release, Brian, where you and I were working together. Brian Jones (00:18:54): I think I played on one of those fantasy football leagues. Rob Collie (00:18:56): You might have. Brian Jones (00:18:57): I was one of the people with zero experience. I remember going into the draft not knowing... I knew football, but I didn't know anything about fantasy football. Rob Collie (00:19:03): That's right. We did loop you in. So let's do that way back machine for a moment. That release when you and I met was the first release on Excel. I was the lead at that point. It was my first time being a lead. It was the first time I was in charge of a feature set, and it was really my baby, this XML thing we were doing. And the reason for that was because no one was paying any attention. That was this weird release. For a whole release, Office went and tried to do cloud services without having any idea what that really was going to mean. And so we stripped all of the applications down to skeleton crews. And this is really the only reason why on the Excel side, some youngster like me was allowed to be a lead and come up with a feature, because no one cared. No one was paying any attention. There was no one minding the store. Rob Collie (00:19:48): I remember being so wild-eyed enthusiastic about how much this was going to change the world, this XML import export future. And I mean, you might as well just take it out. I can't imagine it's being used hardly at all today. I bet Power View is used more often than the XML import export feature. You all have done a pretty good job of hiding it. So kudos. But it was a good thing to cut my teeth on. I learned a lot of valuable lessons on that release. Rob Collie (00:20:24): How do you feel about the XML structure document work that you were doing in Word at the time? Do you kind of have the same feeling looking back at it that I do? Brian Jones (00:20:33): It was a similar thing. In fact, we did rip it out a couple of years later. I think that when you and I would talk about it, we would talk about these scenarios that were super righteous and great. And then we just start geeking out on tech. And then we would get way too excited about the tech and we kind of forget about those initial scenarios. We wouldn't stop and think, "Wait a minute. These users we're talking about, are they actually going to go and create XML files?" Because you need one of those to start with before any of this stuff makes sense. And no, of course, they're not. But for me, a lot of it started from that. Like I said, one of my first features was that easy data binding to Excel feature. And we thought, "Hey, maybe XML would be a good tech for us to use as a way of having Word and Excel talk to each other," because clearly they have different views on what formatting is and how to present information, but the underlying semantic information, that could be shared. Brian Jones (00:21:20): And so I could have a set of products show up in Excel as a table. And when they come into Word, they look more like a catalog of products. That totally makes sense. And we just did a lot of assumptions that people would make, do all the glue that was really necessary. And of course, they didn't. So I had the exact same experience. The other big thing that was different back then for us was we would plan something, meet with customers for six months, but then it'd take three years to go and build it. We had no way of validating that stuff with customers because we couldn't get them any of the builds. And then even after we shipped it, they weren't actually going to deploy it for another three-plus years. And so the reality is from when you had the idea to where you actually can see that it's actually not working and people aren't using it is probably about six years. So you've probably moved on to something else by then. Brian Jones (00:22:04): The only way you really as a PM got validation that your feature was great was whether or not leadership and maybe press got excited about your thing, but you didn't get a whole lot of signal from actual customers whether or not the thing was working, which is obviously completely different now, thank goodness. Rob Collie (00:22:18): Yeah. That Is true. It took some of the fun out of being done too, now looking back at it, like the day of the ship party, when we were done with the three-year release. "Okay, fine." We'd dunk each other in fountains and there'd be hijinks and stuff. But the world did not experience us being done. That was purely just us feeling done. And then it was like you take a week off maybe, and then the next week, you're right back to the grind at the very beginning. You never got the payoff. Even if you built something really good, by the time the world discovered it and it was actually really helping people at any significant scale, you're no longer even working on that product. Brian Jones (00:22:57): Yeah. You're doing something completely different. Rob Collie (00:22:59): You might be in a different division, both finding out the things in real time that Rob Collie (00:23:03): [inaudible 00:23:00] Both finding out the things, sort of in real time, that aren't working. That's the obvious advantage, right? But there's also this other emotional thing. Like you never got the satisfaction when you actually did succeed. Brian Jones (00:23:11): Right. You didn't see it actually get picked up, adopted. Millions and millions of people using it, which is what the team gets now. We no longer pick a project and say, "Okay, how many people and how long is this going to take?" You really just try and figure out what's critical mass for that project. And then you just let them run. And you'd be really clear around what are the goals and outcomes they're trying to drive. And they just keep going until they actually achieve that. Or we realized that we were wrong, right? And we say, "Hey, we thought people are going to be excited about this. It's not even an implementation thing. We were just wrong. We misread what people really were trying to do. Let's stop. Let's kind of figure out a way of moving off of that and go and figure out what the next thing is we should go and do." Rob Collie (00:23:50): That era that we're talking about right now. The 2003 release of Office. I was still very much a computer science graduate and amateur human. That's exactly backwards, it turns out, if you're trying to design a tool that's going to be used by humanity. Brian Jones (00:24:08): Well, it's what leads you to get really Excited about XML? Rob Collie (00:24:12): That's right. Yeah. That's right. Tech used to have such a power in my life. I'm exactly the opposite now. Every time I hear about some new tech, I'm like, "Yeah, prove it." I am not going to believe in this new radical thing until it actually changes the world around me. I'm not going to be trying to catch that wave. But XML did that to me. It was almost a threat. If we don't take this seriously, we're going to get outflanked. It got really egregious. Rob Collie (00:24:42): I had a coworker one time in that same release in the middle of one of my presentations asked me. This guy wasn't particularly, in the final analysis, looking back, not one of the stronger members of the team, but he had a lot of sibling rivalry essentially in his DNA. And he'd asked me in front of his crowded room, "Well, what are you going to do about Bluetooth?" And, we didn't know what Bluetooth was yet, right? It was like, unless I had an answer for what we were going to do about Bluetooth and Excel, right? Then I was not up on things. You know, the thing that we use to connect our headphones. At the time, Bluetooth was one of those things that might just disrupt everything. Brian Jones (00:25:29): It was funny. It was at that same time, I was asked to give a presentation to the Word team about Bluetooth. We were all assigned things to go and research as part of planning and that was one of the ones I was asked. And I gave a presentation that was just very factual. Here's what it is. And I was given really bad feedback that like, "Hey, I wasn't actually talking about it strategically and how it was going to affect Word. I was just being very factual." And I was like, "I don't understand. I don't understand what success looks like in this task." Right. Rob Collie (00:25:59): I remember going, a couple of years later, going into an offsite, those offsite big, I don't know if you all still do those things, big offsite, blue sky brainstorming sessions. There was this really senior development lead that was there with me. And he and I were kind of buddies. At one point, halfway through the day, he just leans over to me and says, "Hey, I'm going to the restroom and I'm not coming back." And I looked at him in horror, almost like "Thou dost dishonor the offsite!?" And he's like, "Yeah, you know, I've never really believed that much in this particular phase of the product cycle. It's never really meant anything to me. It's all just BS." It was just devastating. I just knew it was right. He was... Brian Jones (00:26:46): But you didn't want to, you didn't want to believe that. Rob Collie (00:26:52): I mean, I felt so special. I was invited to the offsite, the big wigs and everything. Brian Jones (00:26:57): They have nice catering too, Rob Collie (00:26:59): Yeah and he was totally right to leave. Brian Jones (00:27:04): I always remember getting super nervous to present stuff for those. Once it was actually, it was one of our XML ones where I was trying to convince, it was my attempt to get us to create an XML file format, which actually ended up, obviously, happening. But I got an engineer to go to work with and we had Word through an add-in, start to write to XML. And it was just a basic XML format. And then I built all of these... it was like asp.net tools that would go and then create an HTML version of the Word doc that was editable. And it also even created, I think it was called WHAP, I don't remember, like a tech for phones. It was back when you didn't have the rich feature phones, but these basic ones. Brian Jones (00:27:41): And so I created this thing that was almost like a SharePoint site. So you could take all your Word docs, go through this add-in, and then you could actually get an HTML view of them to edit it and a phone view of them to go and edit it. Brian Jones (00:27:51): I think it was probably 2002 or 2001, but I was so excited to go and show that at the offsite because I was like, "Okay, this is where I make it, man. Everybody's going to be so excited about me." But I don't know. I think everybody was excited about Bluetooth at that point or something. Yeah. Rob Collie (00:28:05): Oh yeah Bluetooth, WHAP was so 15 minutes ago. So there's a few, irresistibly funny or interesting things I want to zero in on from that era before we come back to present, and we're definitely going to come back to present, for sure. Rob Collie (00:28:21): First of all, we went to a conference like some W3C sponsor. I don't think it was necessarily W3C affiliated, but it was the XML conference. Brian Jones (00:28:31): The one in Baltimore? Rob Collie (00:28:32): Yes. Rob Collie (00:28:33): Okay. Now two very, very, very memorable things happened at that conference. I bet you already know one of them. But the other one was, and we're just going to make this all this anonymous person's fault. Okay. We're not going to abdicate any responsibility. And we're just going to talk about our one coworker from Eastern Europe who brought his wife and they had vodka in their hotel fridge, or freezer, or something like that. And every day I would wake up and say, "I am not going to get suckered into that again." Rob Collie (00:29:12): And then the next day I would wake up and say the same thing. That was a tough trip. Brian Jones (00:29:16): I definitely remember that. Rob Collie (00:29:18): Even on my young, relatively young, body at the time that... Trying to keep up with that, that was difficult. But the single most outstanding memory from that conference, and we will also leave this person anonymous. But there was an executive at Microsoft who was hotter on XML than either you or I, which is hard to believe, right. And we ended up with the sponsored after hours session at this conference. You remember this? You see... Brian Jones (00:29:45): I do. Rob Collie (00:29:46): You know where we're going. Okay. So this was a 30 minute sponsored by Dell or something. Right. It was a 30 minute session, at 5:00 PM, at the end of a conference day where everyone's trying to go back and get to the bars or whatever, right.? But, it's a Microsoft executive, it's Dell sponsored, we'll show up. And the plan was at the end of this 30 minute talk given by this executive, he was going to bring all of us up on the stage to show everyone the team that had done all of this, right? Great plan. Except it was the worst presentation in history. I remember it running for two hours. It was so bad that we started off with 200 people in the room and at the end of it, and I'm just like an agony the whole time cause like I'm associated with this, right? Rob Collie (00:30:31): At the end of these two hours, or what felt like two hours anyway, it was easily 90 minutes. There's five people left in this room of 200 and it's not like the presentation is adapted to the fact that it's a smaller audience. It's just continued to drone on exactly as if everyone was there, right? And I'm sitting here thinking, "Okay, he's not going to call us all up on this stage. There's been more people on the stage than in the audience. If he does this, he's clearly not going to do that." And then he did and we all had to parade up there and stand there like the biggest dodos. I've never been more professionally embarrassed I don't think, than that moment. Rob Collie (00:31:14): And we're all looking at each other as we get up out of our seats like, "Oh my god." Brian Jones (00:31:19): I definitely remember this. Rob Collie (00:31:22): I don't see how you could have forgotten. Brian Jones (00:31:23): Well, yeah. And the person that we're talking about is actually one of my favorite people on the planet. I totally... I love this guy. I view him as like a mentor and everything, but... Which makes me remember it even more. Brian Jones (00:31:34): I think it was just, there was so much excitement. There'd been so much build up to this and this was like a kind of crescendo right? Of bringing this stuff. We probably should have had it a little bit shorter. Rob Collie (00:31:46): I mean when it reaches the point where clueless, mid twenties, Rob Collie is going, "Oh no, this is not the emotional, this not the move." You don't do it. Brian Jones (00:31:58): I'm no longer excited about being called up. Rob Collie (00:32:04): So from my perspective, you kind of parlayed that experience of the XML and all that kind of stuff. I think you did a really fantastic job of everything you guys did on that product. Again, it was the relevance that ultimately fell flat for both of us right. I guess in the end, the excitement with XML wasn't really all that appreciably different from the excitement about Bluetooth. I mean, it's everywhere, right? XML is everywhere. Bluetooth is everywhere and neither one of them really changed things in terms of what Excel or Word should be doing. It seemed like you played that into this file format second act. And I think very, very, very effectively, actually there was a little bit of controversy. Rob Collie (00:32:43): Let's set the stage for people. This was the 2007 release of Office where all the file formats got radically overhauled. This is when the extra X appeared on the end of all the file names, right? Brian Jones (00:32:58): Yeah. Rob Collie (00:32:58): There was a controversy internally. Kind of starting with Bill actually. That we shouldn't make well-documented transparent file format specs, right. There was this belief that the opaque file formats of the previous decades was in some sense, some big moat against competition. And of course, a lot of our competitors agreed. Tailor out in the public saying, "Yes, this is a barrier to competition. It's a monopolistic, blah, blah, blah." We, Microsoft had just gotten its ass kicked in the Anna Truss case. So it was really interesting. I credit Brian, your crew, with really advocating this very effectively. That's a difficult ship to turn. First of all, you got all these teams to buy into all this extra work, which no one wants to do. But when it's not even clear whether you have top level executive support, in fact, you might actually have C-suite antagonism towards an idea. To get it done. That's a career making achievement. I'm sure you remember all of that. Right. But what are your reactions to that controversy? Do you remember being in the midst of that? Brian Jones (00:34:12): I do. It was definitely a long running project. It evolved over quite a number of years. The beginning of it was, in that previous release, the XML stuff you and I were talking about was more about what we called "Custom XML". Right? So people would go and create for themselves. But in that same release, we had Word, we outputted an XML format that was our definition, which we called "Word ML" and Excel did a similar thing. Words' we try to make full fidelity. So you could save any word document in the XML format. Excel's was kind of a tailored down, it was less about formatting, it was more, "Hey, here's like..." It's almost like, "Here's a better version of CSV, right. But we're going to do it as XML." And so we already had a little bit of that. Brian Jones (00:34:53): And the whole reason we were looking at that was, on the Word side, for instance, a lot of the customer issues that we'd get where people would have corrupt files, they were corrupt because they there'd be some add-in that they had running or some third party app that was reading and writing word files. The files were fairly brittle and complex. The binary format... The binary format was written back in the days of floppy disks, right? So the top priority was how quickly can you write to a floppy disk and read from a floppy disc, right? It wasn't about, how easy is this for other people to go and read and write? Not because it was on purpose, make it hard. It was just the primary bid is let's get this thing so it's really easy to read and write from floppy, right? Brian Jones (00:35:31): And so in Word, we were like, "Wow, I think that there's a bigger opportunity here for an ecosystem around Word if we make it easy for people to read and write Word docs and build solutions around them." And so then the next release, the Excel team was looking at doing some big changes around a lot of the limitations, like how many rows you could have in columns, right. Lengths of like formulas and things like that. Right. And so there was this thing where the Excel team was like, "We are going to need to create a new file format." And on the word side, we thought this XML thing was great. We want to move to that as our new format. Brian Jones (00:36:01): And so everything kind of came together and it was clear. Hey, this is going to be the release that we are going to go and rev our file format, which we hadn't done in a while. This is also the release of the ribbon. So there were two really big major changes in that product, right? It was the new file format and the ribbon. It's funny. I still refer to it as the new file format, even though it's 15 years old. Rob Collie (00:36:23): Yeah. It's the new file format it's still new, yeah. Brian Jones (00:36:25): I still call it that, which is kind of nuts. But I think that the controversies you were talking about was really more of a... Boy, this is a really big deal for the product. We had changed file formats before in the past and not necessarily gotten it right. And there were a lot of challenges around compatibility and stuff. And so there was just a lot of worry of let's make sure you all have your stuff together here, right? Like let's make sure that this doesn't in any way break, stop people from wanting to upgrade to the new version. But it went really well. The whole goal of it was let's get something that we think third parties can go and read and write, and this is going to help build an ecosystem. And a new ecosystem run Office. Office already had big ecosystem with VBA and COMM add-ons and stuff like that, right.? But we won't have this new ecosystem around our file formats as a thing. That's why we chose... There's a packaging layer, which is all zip based. So if people haven't played around with it that XLSX, you can just put a .zip at the end and double click it. And it's just a zip file. And you can see a whole bunch of stuff inside of it. Right? Rob Collie (00:37:23): Yeah. If you're listening, you haven't done that go right now, run don't walk, grab an Excel file or a Word file, whatever. Go and rename the XLSX or BPTX, go ahead and rename it so that it ends in .zip and then open it up and you'll be blown away. Thomas Larock (00:37:38): PowerPoint is my favorite when I have to find some unknown setting that I need and I can just search through the whole thing. Yep. Rob Collie (00:37:45): Or all the images. You want to get all the images out of the PowerPoint file. It's just a zip file that has a bunch of images in it. Right. Brian Jones (00:37:50): So I also did this for backpack. It's the same thing. You can crack open the backpack by renaming a zip file... Thomas Larock (00:37:58): An actual physical backpack? What are we... what are we talking about here? Brian Jones (00:38:03): Ah yeah. Rob Collie (00:38:03): This is the digital acetate that is over the top of the entire physical world that you aren't aware of. Thomas Larock (00:38:08): Digital acetate, that's it? That's it. That's where the podcast peaks. Right? Those two words. We're all going home now. Brian Jones (00:38:19): Yeah. No. A SQL server, there's DAC pack, which is just the, say database schema. Then there's a backpack which has the data and the schema combined. But you can, if you rename them . zip, you can crack them open to see the XML that makes up those forms. So it's not just office products. Rob Collie (00:38:37): We ended up standardizing the entire thing, but that packaging format, it was called OPC, Open Packaging Convention, or something like that. It was something that we did in partnership with a Windows team. It's part of the final ISO standard for our file format. And then there were a lot of other folks that went and used that exact same standard. Because it's a really easy way of you have a zip package. You can have a whole bunch of pieces inside of it, which are XML. And then there's this convention for how you can do relationships between the different pieces. So I can have a slide. That's an XML and it can declare relationships to all the images that it uses. And that way it's really quick, easy to know, okay, here's all the content I need to grab if I want to move pieces of it outside of the file. Rob Collie (00:39:16): So the single coolest thing I've ever done with, we'll just call it your file format Brian. We'll just pretend that it was only you working on that. Brian Jones (00:39:23): Just me yeah, I was pretty busy, but yeah. Rob Collie (00:39:27): So the very, very first version of Power Pivot, first of all, your file format, the new file format made Power Pivot possible. We needed to go and add this gigantic binary stream of compressed data and everything, everything about Power Pivot needed to be saved in the file. At the beginning of the project, everyone was saying, "Oh, no, we're going to save it as two separate files." And I'm like, "Are you guys kidding?" The Pivot cache, for instance, is saved in the same file. You can't throw a multi file solution at people and expect it to... This was actually like Manhattan project, just to get that stream saved into the same file. It was pretty crazy. However, when it was done, there was something really awesome I wasn't aware of until the very end, which was, first of all, you could open up a zip file and just tunnel down and you would find a file in there called item one.data. Rob Collie (00:40:21): Okay. That was the Power Pivot blob. That was everything about the Power Pivot thing. And it was by far the biggest thing in the file, like it was like 99% of the file size was what was there. However, as this backup, someone had decided, I had nothing to do with this, to save all of the instructions. I think it's called XML for analysis XMLA. All of the instructions that would be required to rebuild exactly that file, but without any of the actual binary data in it. So it was a very, very small amount of XML. Okay. So here's what we would do because there were no good automation, no interfaces, no APIs. If we needed to add like 500 formulas to a Power Pivot file, you could go through the UI and write those 500 formulas, type, click, type, click, type, click. Rob Collie (00:41:08): Okay. So what we would do, and my first job outside of Microsoft, is we would go in there and we would edit that XML backup and add all the formulas we wanted in it. And by the way, I would use Excel to write these formulas. I would use string concatenation and all of that kind of stuff to write these things. It was very, very, very sensitive, one character out of place in the whole thing fails. So you make those changes. You save the file, reopen it, nothing happens because it's just the backup. Okay. So then you've got to go and you've got to create a zero byte item one.data file on your desktop and you copy it into the zip file and overwrite the real item one.data, therefore deliberately corrupting the primary copy. So when you reopen the file it triggers the backup process and it rehydrates with all of your stuff, it was awesome. Rob Collie (00:41:57): And then a couple of releases of Power Pivot later, suddenly that didn't work anymore and I was really pissed. But it just really shows you, it opens up so many opportunities that you never would have expected. And even a hack like that, that's not the kind that you'd be really looking for, but the fact that something like that even happens as a result of this is really indicative of what a success it was. Brian Jones (00:42:19): Yeah. I mean, there's a lot of those things where, I love building platforms, like that's my favorite part of the job. It's all those things that you see people do that you never would have predicted. Right? That's just so exciting. PowerPoint had this huge group of folks that would go and build things like doc assembly stuff, right. Where they go and automatically build PowerPoint decks on demand, right? Based on who you're going to go and present to cause they've just shredded the thing. In fact, when we did the ISO standardization, it was a 6,000 page doc that we had to go. And we built and reviewed with a standards body and we did it over about a year. Which sounds nuts, a 6,000 page doc in about a year. And the way that we were able to do that is there was never really a 6,000 page doc. There's a database where there's a row for every single element and attribute in this, in the whole schema, that would then have the column which is the description, which would just be the word XML. Brian Jones (00:43:09): And so we could, on demand, at any point, generate whatever view or part of the doc we wanted. So we'd say, "Hey, we're going to go in now, review everything that has to do with formatting across Word, Excel and PowerPoint." And so we just click a couple buttons and the database would spit out a Word doc that was just that part. Everybody could go and edit it cause we were using the structured elements we'd added to Word, which is called content controls, which was the next version of that XML stuff that we had to deprecate. And then the process, as soon as you'd finish editing that Word doc, we just submit it back. The process would go back and shred that Word doc again and put it all back in the database. And so we really used the file format to bootstrap documenting the file format. Rob Collie (00:43:48): And then when you dump a 6,000 page document on someone, they have no choice. But to just say, yep, it looks good to us. Brian Jones (00:43:55): Well, there was a pretty, incredibly thorough review still. It was just pretty impressive. The final vote that we had in Geneva, the process leading up to that, the amount of feedback that we got. Cause basically the ISO, you can kind of think of it like the UN, you go and show up and every country has a seat, right? I mean, not everybody participates, but anybody that wants to can. And so yeah, we had to respond to thousands of comments around different pieces, things that people wanted to see changed. Rob Collie (00:44:22): Yeah. I can imagine, right. Think about it. You just said at the final vote in Geneva. That's a heavy moment man. Thomas Larock (00:44:29): Yeah. That threw me off for a second. I thought, for sure, you were talking Switzerland, but now thinking that was just a code name. Rob Collie (00:44:38): No, I think, I think he was actually in Switzerland. Brian Jones (00:44:40): In Switzerland. Rob Collie (00:44:41): Have you seen the chamber where they do these votes? It looks just like the Senate from episode one of Star Wars. It's just like that. It's pretty heavy. Brian Jones (00:44:51): The little levitating... Rob Collie (00:44:53): The floating lift. Yeah. I think they call that digital acetate. I think that's what they call that. By the way on the Excel team, the way I came to look at the new file format and the open architecture of it, again, this this will show you how quickly I had turned into the more cynic side of things. Well, okay. We're going to be changing file formats. And we're doing that for our benefit because we didn't have enough bits allocated in the 1980s version of the file format that was saved to floppy disc, as you pointed out, right. Who could ever imagine having more than 64,000 rows, it's just inconceivable or 250 columns or whatever, right.? Because we hadn't allocated that. We'd made an engineering mistake, essentially, we hadn't future-proofed. So we need to make a file format change for our benefit, right. To undo one of our mistakes. And the way I looked at it was, "Ooh, all this open file format stuff, that'll be like the 'Look, squirrel!'" To distract people and to sort of justify, while we went and did this other thing, which, ultimately it actually went pretty well. The transition for the customers actually wasn't nearly as bad, because we actually Took it seriously. Rob Collie (00:46:03): The transition for the customers actually wasn't nearly as bad because we actually took it seriously. We didn't cut any corners. We did all the right things. Brian Jones (00:46:07): Well, there were several benefits too. We were talking about all the kind of ecosystem development benefits, but the fact that the file was zipped and compressed right, it meant that the thing was smaller. And that was all of a sudden, it was no longer about floppy discs. People are sharing files on networks. And so actually being able to go and have a file that's easier to share, send over network because it's smaller was a thing. Brian Jones (00:46:26): There were a couple of things that we were able to go and highlight. There's also a pretty nice thing where it was actually more robust because it was XML, and we split it into multiple pieces of XML. It meant that even if you had bit rot, you would only lose one little piece of the file, whereas with old the binary format, you had some bit rot and the whole thing is impossible to open up.There are a couple of things that were in user benefits too, which helped. Rob Collie (00:46:50): And ultimately, on the Excel side, the user got a million row spreadsheet format and the ability to use a hell of a lot more than like 14 colors that could be used in a single spreadsheet or something. It was .like a power of two minus two, so many bizarre things. Like Excel had more colors than that, but you couldn't use more than a certain subset in a- Brian Jones (00:47:10): At a time, yeah. Rob Collie (00:47:10): -In a single file. So yeah, there were a lot of benefits. They just weren't- Brian Jones (00:47:15): It's not like it's an explicit choice. It's just that at the time somebody is implementing something, you're right in a way, assuming, "Oh, this is fine. This is enough. I'll never have to worry with this issue." Rob Collie (00:47:25): Why waste the whole byte on that? When you can cram four different settings into a single byte. If you read the old stories about Gates and Allen programming up at Harvard, they had these vicious head-to-head competitions to see who could write the compiler or the section of basic in the fewest bytes possible. This was still very much hanging over Microsoft, even the vestiges of it were still kind of hanging over us even when I arrived. But certainly in the '80s when the Excel file format was being designed for that rev, it was still very much like, "Why waste all those bits in a byte?" "Let's cap it at four bits". Thomas Larock (00:48:05): In that blog series from Sinofsky, he talks a lot about that at the early start. And I'm at a point now where he's talking a lot about the code reuse because the Excel team, the Word team, I guess PowerPoint, but all these other teams, were all dealing with, say, text. And they were all doing their own code for how that text would be displayed and shown. And Bill would be the one being like, " This is ridiculous". "We should be able to reuse the code between these products". And to me, that would just be common sense. But these groups, Microsoft just grew so rapidly so quickly, they were off on their own, and they have to ship. I ain't got time to wait around for this, for somebody to build an API, things like that. I'll just write it myself. Brian Jones (00:48:50): It's a general thing that you get as you get larger where the person in charge that can oversee everything is like, "Well, these are all my resources", and, "Wow, I don't want three groups all building the same thing". But then when you get down, there's also a reality of we're just going to have a very different view on text and text layout than Excel. And Excel is not going to say, "I want all of that code that Word uses to lay out all of their content to be running for every single cell". Right? That's just suboptimal. And so it's always this fun conversation back and forth around where do you have shared code and reuse and where do you say it's okay for this specific app to have this more optimized thing that might look the same, but in reality, it's not really the same. Rob Collie (00:49:33): Brian, do you remember the ... I'm sure you do, but I don't remember what company they were from. But at one point in this file format effort, these really high priced consultants showed up and went around and interviewed us a couple of times. Do you remember that phase? It was like- Brian Jones (00:49:51): Was that towards the end? There was a couple summary stories that were pulled together just to talk about the overall processes. It was actually after the standardization. Rob Collie (00:49:58): I remember this being at the point in time where it was still kind of a question. whether we should do it. Brian Jones (00:50:02): I don't remember that. Rob Collie (00:50:04): The thing I remember really vividly is a statement that Chris Pratley would make over and over again, this encapsulated it for me. I came around to seeing it his way, which was the file format isn't the thing. That's not the moat. The thing that makes Office unique is the behaviors of the application. It's not the noun of the file format. It's the verb of what happens in the app. It's instructed to think that even if you took exactly the Excel team today, every single person that's already worked on it, and said, "Hey, you have to go rebuild Excel exactly". There's no way that version of Excel would be compatible with the one we have now. It would drift so much. Rob Collie (00:50:43): You could even have access to all the same specs. We would even cheat and say, "Look, you can have access to every single spec ever written". So? It was clearly someone had thought it was time to bring in like a McKinsey. They were all well dressed. They were all attractive. They were all a little too young to be the ones sort of making these decisions. It was just really weird to have them show up, three people in your office. Like, "Okay, I'll tell you what's going on". Brian Jones (00:51:11): I can totally imagine. It's funny I don't remember that. There were several rounds of analysis on how we were doing it, what we're doing and making sure we were doing it the right way. But yeah, Chris is spot on. I mean, your point about rebuilding it, that's essentially what we've been going through for the past five plus years around our web app. It's a lot of work. Unfortunately, we can't let it drift. The expectation from everybody is, "Hey, I learned the Wind 32 version. When I go to the web, I want it to feel the same. I don't want to feel like I'm now using some different app." Rob Collie (00:51:44): What an amazing, again, like a Manhattan project type of thing, this notion of rewriting Excel to run on the web and be compatible. Brian Jones (00:51:55): Yeah, with 30 years of innovation. Rob Collie (00:51:56): Yeah. That started in the 2007 release. Excel services, the first release of Excel services was 2007. And this whole thing about shared code, like what features, what functions of Excel, what pieces of it were going to be rewritten to be quote unquote "shared code"? And shared code meant it was actually server safe, which none of regular desktop Excel written in the early '80s, still carrying around assembly in certain places, assembly code of all things, right? Excel was not server safe. It was about as far from server safe as you could get. And so to rewrite this so ambitious without breaking anything. Oh my God. What a massive ... This dates back, gosh, more than 15 years. Brian Jones (00:52:45): Yeah. I'd say like the first goals around it were a bit different, right? It wasn't a web version of Excel. It was like BI scenarios and how can we have dashboards and Excel playing a role in dashboards. But yeah, I'd say since I joined, it was probably maybe a half a year or a year into when I joined, we just made the decision to shift a huge chunk of our funding to the web app. It was just clear that we need to make even more rapid progress. If you go, we have a site where you can go and see all the features that are rolling out there. It's incredible. And it's just because of the depth of the product. "Wow that's so many features you've done. You must be almost done". But then you look at everything else that's still isn't done yet. Brian Jones (00:53:23): Now thankfully, we're getting to the point where we can look at telemetry and say, "Hey, we've got most people covered." Most users, when we look at what they do in Windows, they could use the web app and shouldn't notice a difference. But there still is a set of things that we're going to keep churning through. So that'll continue to be a huge, huge investment for us. But yeah, the shared code strategy, we have an iPhone version, an iPad version an Android version. We've got Excel across all platforms. And because of the shared code, when we add new features, the feature crew that's working on that, they need to have a plan for how they're going to roll out across all those platforms, clearly levered shared code. But they also need to think through user experience and stuff like that too. Clearly a feature on a phone is going to behave differently than it's going to behave on a desktop. Rob Collie (00:54:05): Part of me, just like, kind of wants to just say, "I don't even believe that you've pulled that off, there's no way". It's kind of like, I've never looked at the Android version, and until I look at the Android version, I'm just going to assume it's not real. This is why it's one of the hardest things imaginable to have a single code base with all these different user experience, just fundamental paradigms of difference between these platforms. Like really? Come on. Brian Jones (00:54:34): It was a massively ambitious project. Mac shifted over maybe three years ago. And that's when, all of a sudden, in addition to a bunch of just features that people have been asking for that we'd never been able to get to, the massive one there was we were able to roll out the co-authoring multiplayer mode for Excel. Rob Collie (00:54:50): Multiplayer. Brian Jones (00:54:52): That's the term I like for co-authoring. It's more fun. Rob Collie (00:54:55): Yeah. It's like MMO for spreadsheets. Brian Jones (00:54:57): Yes. We were able to get that for the Mac. I mean, all of our platforms. One of us can be on an iPad, an iPhone, the web app, and we'll all see what we're doing in real time, making edits and all of that stuff. That alone, if you want to talk about massive projects, 30 years of features and innovation, basically that means we had to go and teach Excel how to communicate to another version of Excel and be very specific about, "This is what I did." "Here's the action I took." And that is massive. There are thousands and thousands of things you can do in the product. So getting it so that all of those versions are in sync the entire time, and so we're all seeing the exact same results of calc and all of that. That itself was a huge, massive project. Rob Collie (00:55:37): Take this as the highest form of praise when I say I don't buy it. I can't believe it. Brian Jones (00:55:44): I hope everybody's okay that we just talked for like an hour on just like listening to somebody at a high school reunion, I think, or something. Is this like me talking about how great I played in that one game? And you're like, "Yeah, that was a great basket". Rob Collie (00:55:54): Yeah. "Man, my jumper was on". the thing that's hard to appreciate, I think, is that you got to come back to the fact that we're talking about the tools that everyone in the world uses every day, that we rely on. And I think being gone from Microsoft for the last 12 years, I'm able to better appreciate that sense of wonder. This isn't just you and I catching up, I don't think. People enjoy, for good reason I think, hearing the stories of how these things came to be. People don't know by default how hard it was to get to a million rows in the file format. If you're like a robot, you're like, "I don't care how I got here. I just care what it is", then you're not listening to this show. We call it data with a human element. Robots can exit stage left. I think you should feel zero guilt. This isn't just self-indulgence. Brian Jones (00:56:55): Well, on the off chance everybody else ... I've listened to a lot of Rob's other podcasts, and they're awesome. So if you're bored with this one, it's okay. Go check out some of the other ones. They're great. Rob Collie (00:57:06): Imposter syndrome rears its head. This guy, Brian, all he does is run the mission for Excel. I mean, he's, he's not good enough to be on our show. Brian Jones (00:57:17): And talk about his past, I guess, all the time. Rob Collie (00:57:21): If you struggle with imposter syndrome, right there should be your antidote. Right? We just heard ... Trust me, Brian, you're okay. Brian Jones (00:57:31): Thank you, Rob. Rob Collie (00:57:32): We just talked about all this gargantuan effort, intelligent, but also heavily resource and effort intensive. You can't just be smart about this and then suddenly it's done in a month, you've got to be smart every single day on every single little microscopic detail while grinding for years to do these sorts of things. And at the same time, sort of at the same time, there's also been this JavaScript API. Holy hell. So file this also "under things that can't be done", a version of the office object model and API that can be used for local desktop automation, like VBA has forever, but also to be used for server automation of the cloud-based web based apps. Brian Jones (00:58:22): Ooh. Yeah. Rob Collie (00:58:23): And I haven't had the time or the energy to get into this. I have not written a single line of code against this new API. But when I started to hear things like ... Yeah, we had little tiny things like this even back when I was at Microsoft, there were a couple of APIs that would work on the server, but they were completely different from the desktop version. And they were an egregiously tiny subset, just the least ambitious things possible. And so I kept using the criteria of, until you give me a range object equivalent in this fancy blah-blah-blah API that you're working on, Microsoft, it ain't real. It certainly sounds like y'all have that now, which is nuts. Brian Jones (00:59:11): Yeah. It's funny. This one could actually be a long history discussion because we started this project actually 10 years ago, basically around when my daughter was born, where the first step was really looking more pro devs. But it was the idea of we're moving cross plat. It was still early days for the web app. We weren't really working on mobile yet, but it was clear, mobile apps were going to start to become a thing. And so it was like, hey, VBA and COM APIS just aren't going to be across plat, so we need to figure something out here. And I would go and talk to a bunch of developers. I'd go to build conferences and stuff like that. And they'd talk about how challenging it was to specialize in office development because so much of the tech that they would learn was not applicable in other contexts. Brian Jones (00:59:54): And so we start thinking, "Okay, well what are the texts that they know?" We were looking at things like that .net. And we eventually started looking at well, there's obviously a ton of people doing web development, HTML, JavaScript. So what if we just said extending Office was just like writing a webpage or building a web app? And I'd say the first round of it, it was a little bit more like data visualization type of scenarios. We would go and kind of I-frame in your web app into Excel. You could write some HTML. But that was the starting point to start to build out an API. And it just, back to the same thing we were talking about the web app, 30 years of API innovation, it takes a long time to get to where we've got an API that covers most of the things you can do in the product. Brian Jones (01:00:38): And so we stayed really focused on ProDevs for a long time, because to do an end user, kind of like the VBA, equivalent where you can do Mac recording and stuff like that, you've got to cover everything somebody can do. Because it's kind of weird to say record macro, and then only 20% of the things you do actually get recorded. And so we reached that point about a year or so ago. And so I think it was last spring, we went and released the beta version of this Office scripts. And then we just GA-ed the spring. Right now, the main platform we run on for the macro recording piece is all on the web. But of course, the plan is to go across plat and desktop and everything too. But it's kind of neat. It's an example of, there are certain cases now where going out first on the web where if you went back like four years ago, everything was Windows first and then eventually it would show up, hopefully, on web. Rob Collie (01:01:29): First of all, I'm crazy impressed that this has happened. I just tuned out of this whole conversation many years ago because I just felt like you're never going to get there. And then you just sneaked up on me. It was like the tortoise and the hair. The other thing that's really impressive to me is sort of the organizational discipline and willingness to make those long commitments, the things that aren't going to necessarily pay off in 6 to 12 months, or even 18 months. To be on an initiative that does take many years to deliver, that's first of all, a non-Microsoft mindset. I typically associate that with Microsoft as an outside observer. And secondly, you're in this world now where you're releasing on a much faster tempo. It would seem like it would even reduce the chances- Brian Jones (01:02:20): Yeah, it's going to be harder- Rob Collie (01:02:21): Further. That you could keep your eye on a project that honestly, like you say, it honestly doesn't have any payoff at 90% done. You've got to get to a hundred before you can reap any of the benefits. It just seems like such delayed gratification, many, many, many years of it. Hats off. I was on some sort of usergroup virtual meeting, and someone from your team, someone from the Excel team was there. I know them, but I just forget who it was. And I was laughing about my wheel of inquisition, which is the pie chart that I spin with VBA, at a random distance to pick who has to answer the next question or who has to ask the next question on team meetings. I just said, "Look, this is an example of why you still need desktop Excel". And your team member interjected and said, "No, I actually think you can do this in the Office script API now". And I'm like, "Get out of town. Just shut up". That was the moment that I kind of started to pay attention. Brian Jones (01:03:16): Yeah. It's gotten a lot more real over the past few years. We had a similar experience with a lot of the MVPS where it was kind of like, okay, I have a set of things that I use come in VBA for, I don't see you solving those yet. But now it's gotten a lot more real. I don't know how much you worked with Tristan, but this is all ... Tristan Davis runs this team. And they're just doing a phenomenal job. Rob Collie (01:03:36): Tristan wasn't really around when I was there. I left the Excel team in 2006, late 2006, to go work on fantasy football over in- Brian Jones (01:03:45): Oh yeah, that wasn't just a hobby. You actually went to a team. I forgot about that. Rob Collie (01:03:47): I actually did, yeah, the team that existed for six months and then evaporated. And I got folded into being, which was terrible. But then I ended up on Power Pivot. So all is, well that ends well. But by the way, the last time I saw the vice president who's left Microsoft, but the VP who was in charge of Excel for many years, the last time I saw him, I was working on Power Pivot. And he already announced he was leaving. And he just sort of like, "oh, so friendly, good to see you", you know, whatever, just in the hallway. And he's like, "Hey, you know, I still use you, Rob, as a cautionary tale to people on how to not mess up their career. Brian Jones (01:04:20): Wow. Oh my God. That's great. Rob Collie (01:04:22): He wasn't expecting it to land is a bad thing. He had no idea. And I'm like, "Oh, that's great". I'm slapping him on the back. "That is the best thing ever". He had no idea. Actually, in the end it was fantastic for my career, that whole path. I wouldn't have it any other way. Brian Jones (01:04:39): But that wasn't the point he was making at the time. Rob Collie (01:04:41): No, that Was not the point he was making. He didn't bother to find out whether it was working out for me. So I just buttered you up. I just told you how awesome it is that y'all have the discipline and the patience to set a multi-year goal and deliver on it. I got another one for you that's in this category, which is, let's get serious about onboarding the VLOOKUP and Pivot crowd into data modeling. Let's do that. By the way, observation about Microsoft not being willing to invest in anything that's not six months out, that impression of mine is primarily built from watching this problem for the last decade, decade plus actually. There's never an answer that's six months away. It's a hard problem. It's just fundamentally a really hard problem. And it's not one that you can just set aside like a two person team and have them take care of it. Rob Collie (01:05:39): But if we ... I'm going to put myself on the same side of the table because I understand, I think, a lot of the trade offs. If we collectively had been doing this even five years ago, it would have been over by now. It's like when you're on a diet, "Oh, if I'd only started six months ago, I'd already be skinny". I am encouraged by the stepped up cooperation between the Power BI and Excel teams. And yet I still think there's something massive being missed here, which is that all of the truly valuable data model builders for Power BI, all of the future builders, it just rounds to 100%. They're all Excel if you look up in Pivot people today. There's this sort of myth that it's an IT thing. It's not, it's not an IT thing. It's the citizen developer type embedded in the business. Rob Collie (01:06:34): Even just emotionally like we were talking about earlier about the people who, when they see something, they get frustrated that they didn't know about it. I mean, this is our life. We're not just a training company. In fact, we do a lot more project consulting now than we training. Training is becoming less and less of an important part of our business. But it's still sort of part of our DNA. So many times people would just like scream from the back of the room. Like, "No! What?" I can go to a finance conference, like an FP&A conference, and show this off and be positive that 90% of the room plus has never seen any of it. It's life-changing to them. Rob Collie (01:07:09): It's almost like we owe it to them. And everyone that listens to the show knows that this is my soap box issue. I would be so inauthentic if I didn't bring it up with you. I don't expect you to go, "Hey Rob. Yeah, that was it. This conversation was the thing we needed. Now we're going to go and do it". But I can't omit this one. So anytime I can lobby to add this particular topic, maybe that multi-year payoff gene. Oh my God. Let's do it. Brian Jones (01:07:39): Yeah. And just for folks that for the listening audience, this isn't the first time I've heard this from Rob. He's only six months into my job. And Dave, Dave Gayner, who's my boss, he had had my job. And then he moved up and became a big wig. He said, "Hey, one of the things you should probably do is go and just get Rob Collie's training because you'll kind of see a bunch of customers and how they use the product, and you'll also learn from him". And it was neat because your next training was in London, so I got to go travel to London to go and get the training. So that was kind of a nice perk too. Rob Collie (01:08:08): Yes, it was. Brian Jones (01:08:10): Well, and yeah, I learned a ton. Same thing. I was like, "Wow, this is pretty cool. I didn't know my product could do this", because I'd been on the job for a few months. In fact, I remember one of the people that was in the session was sitting next to me, and you'd introduced me and who I was during the sessions. Rob Collie (01:08:25): Of course, we got to put the target on you. Brian Jones (01:08:26): Yeah, of course. And so I was making some comments, and the guy was just ... I felt like he was just really offended that I didn't know. He's like, "How can you be in charge of this product and not know this stuff?" And so I wanted to go into the background of like, "Well, I just joined, and that's how it works". But, of course, I got super defensive. Rob Collie (01:08:44): You're welcome, Brian. Brian Jones (01:08:45): But I can't remember ... I think you and I are having beers or something afterwards. And you're just like, "If you just went to every airport and put up posters that had a V lookup with a question mark and then Power Pivot below, that would do it. Right? It's resonating. I still remember that. Rob Collie (01:09:02): That's good. That's long been one of my- Rob Collie (01:09:03): Yeah. Brian Jones (01:09:03): It's resonating. I still remember that. Rob Collie (01:09:03): That's good. That's long been one of my favorite things. Brian Jones (01:09:05): There's a lot of stuff that we've done over time to try to... like, for instance, we've done things to try and make people more aware of Power BI. People have a Power BI license and they're doing stuff in Excel, we'll go and say, "Hey, did you know about power BI?", and try and go and take them off to discover that, because they might be able to benefit from it. Right? And over the long run, we are working with the Power BI team, we want to make sure that we've got better compatibility with the engine and the data models that are run. Right? So you can, whatever you build in Power BI, you can bring into Excel and vice versa. And so there's clearly like a north star we have there around the user not having to think, oh, am I using Power Pivot or am I using Power BI? And those are two different decisions they have to make. That just kind of feels weird. It's basically you're doing the same job. Brian Jones (01:09:49): And as you know, Rob, like you working on the product, you can't just put a big button in the ribbon and all of a sudden everybody's going to discover it. Rob Collie (01:09:56): No. I- Brian Jones (01:09:56): Right? Nobody sees it. So that's where the trick is trying to figure out those points in time where it's like, hey, we think someone would benefit from this higher level of feature. How do we go and educate them about that? And that's where we're starting some of the stuff I talked about with AI. We're doing that, but at a lower level, like you've got a table of data and we say, hey, did you know you can summarize this with a Pivot table? And we're going to iterate on and see how that works. But this would like... Educating people on Power BI will be a thing that we'll continue to look to do based on the activities that folks are doing. And some of the triggers you've thrown out are great ones, right? Rob Collie (01:10:31): Talking to someone from Microsoft recently on this show, I made kind of exactly that recommendation. It Was like, forget it. Even if it was just an awareness thing, the really hard problem is to look at what they're doing and go, oh, this particular file that you're working with right now, we can turn it into a Power Pivot file or a data model. That's crazy. That is really, really difficult. But identifying the people who are good candidates to build data models and therefore would change their lives in the process, right? Brian Jones (01:11:05): Yeah. Rob Collie (01:11:06): Everybody wins here. Identifying those people by their behavior, that shouldn't be hard. And then if it's just an on-ramp, "Hey, you are one of us," right? You are one of this chosen few, and here's the URL. It just takes you to like a YouTube video that's five minutes long, it doesn't try to teach anything, it just shows what they're missing, God, that would be so profound. And that I wouldn't care about Power Pivot in Excel. That wouldn't matter so much. Brian Jones (01:11:37): Yeah. It's probably a thing that you haven't seen because you probably wouldn't have gotten the notification, but there already is in the product. I can't remember the exact triggers, but we went and looked at some data to see who are the people that would benefit from Power BI and what are the behaviors in the app? I don't remember if it was using VLOOKUPs, or it also had to do with what kind of things they were sharing because it turned out if they were sharing then they had more likelihood that Power BI would be super useful because they could go and publish a thing. But there was a set of folks that we would actually go and notify. We'd have a little toast that would come up saying, hey, do you know about Power BI? Here's what it'll do for you. Click here to go and learn more. And that was super effective. It ended up being that the logic that we picked for who we'd share that with the click-through rate was super high. And the Power BI team saw that not only did they go through, but then they actually became engaged Power BI users. Brian Jones (01:12:21): But those are the things that Arun's team and my team. So for folks that don't know, Arun runs the Power BI team. We already look at that and work on that. And then there's a ton of stuff that we've announced around silly things that just felt more like bugs, but actually were a good amount of work, like our web app wouldn't refresh pivot tables that were connected to Power BI data sets, right? We went and fixed that. So now if you want to go and publish your pivot table into Power BI, you can. And so that's another way that those users might first start with just Excel content that they built, but they go into Power BI start, to see some of the benefits that Power BI brings and start to onboard into other things. Brian Jones (01:12:57): We're also looking at how to do a better job if you just, if you happen to have those cases where Excel is your source of truth for data, you have a big table of data in Excel. Getting that to be something you push up into Power BI, they have so much cool intelligence now, right? Where you go and push that data in Power BI and they'll automatically generate dashboards for you and stuff like that. Right? So I'd say view this as it's a long running thing, it's going to be slow, and so you're not going to see like everything solved right away, but you should notice every six months that there's that next thing that we've done and just know that that's not just us doing little small piecemeal things, there is a longer north star that we're going towards. But it's going. It's slow. Some of the stuff that we're doing under the hood is actually... some of it's a little bit heavier lifting. Rob Collie (01:13:39): I'm actually incredibly relieved to hear that. I've been waiting 11 years, so a slow and steady approach is fine with me. It's the possibility that nothing might be happening that has been really killing me, but any- Brian Jones (01:13:54): We are definitely working on it. Rob Collie (01:13:56): And look at the success that you've had on long running initiatives. We just talked about them. If this gets a similar type of mindset, the ghost of Rob Collie can finally rest easy. Brian Jones (01:14:07): You can run... I mean, he's still going to give us a hard time about it. It's one that I... I should just talk to you every once in a while about them. Right? Because it'll be... Every time we announced one of these, I should just kind of ping you just to see, do you see it as a continued progress or did that seem like just kind of a swing and a miss, right? Rob Collie (01:14:22): Well, I had no idea, for instance about the toaster thing that you talked about. No idea. Boy, the pivot table refreshing. That's a huge deal. Brian Jones (01:14:30): Yeah. Rob Collie (01:14:31): It doesn't do anything really to address growing the author audience, but it doesn't matter, that one was just too big, and I know that it was a hard problem, so. Brian Jones (01:14:39): It just clearly... I had another clear sign of, there were some just gaps where we would... like the two things just didn't work together properly. Right? And clearly we need the systems to work together seamlessly. Rob Collie (01:14:50): All right. What's next for Excel? What's the overall roadmap other than taking care of this Power BI authoring problem, which is clearly top of the stack. Brian Jones (01:15:00): All of that stuff I talked about, the multiplayer mode of Excel, super critical there. We're going to keep innovating, making that better matter. For us, web is a tier one experience. It's got to be just as good as Win32 and Mac. So we're going to continue to innovate really heavily there and invest really heavily there. The whole area around intelligence, just trying to find ways of making the application easier to use, easier to learn, more approachable. If you look at, even in schools, people are trying to teach data literacy. Excel should be a key tool for that. And so it's really important that we go and make Excel a thing that teachers feel like they can use to go and teach kids how to work with data. And so there's a big opportunity there, but there's a lot of work we need to do there to just make the application feel more approachable for that audience. Rob Collie (01:15:46): I completely agree. So many times if I had more time and energy, thought about it so much, like going and volunteering when my kids were still little. Can you imagine showing conditional formatting to a room full of like eight year olds? Brian Jones (01:15:59): Absolutely. Rob Collie (01:16:00): They would just jump out of their seats at the insights that leap off the page. Like you'd show them a sea of numbers. Who did best? No, I don't know. No. Okay. Look, voom! Conditional format. And everyone just knows. Brian Jones (01:16:15): Yeah, you should go and look, we put a couple of templates. We did one back in November. It was a thing we did with NASA where it was to get kids excited about looking at space data or data on rockets and things like that, so we built a bunch of templates. We've got one that we just launched about orcas. It was World Orca Day. And so we've got these templates that let you go and analyze the different pods of orcas that exist. It's using the Wolfram data types, so you can actually have cells that are each orca and you can click on them to see all the data about them and start to do analysis. So there's a lot of fun stuff we're doing there. But probably the really big one, back to your point, big bet. The other big bet is you were calling it citizen developer, but really there's people that like to build and create and make stuff, right? Brian Jones (01:16:59): You've got people that are heavy data and want to do an analysis, but there's another side of Excel, which is really more just like solution building or modeling or things like that. Right? Like back to the whole, I talked about the beginning, the soul of the product is the grid and calc. Rob Collie (01:17:14): Yep. Brian Jones (01:17:15): And there's a set of investments that, this is a long running project that we have rolled out in pieces, and I don't talk about them as being one combined thing, but really they are one combined vision, which is really upleveling Excel as a programming language. We're working really closely with researchers in Cambridge, UK who specialize in functional programming. They're experts in functional programming. We've been working with them for a while. Rob Collie (01:17:40): Is this where we get to mention, namedrop, Simon Peyton Jones? Brian Jones (01:17:44): Yes, it is. Simon Peyton Jones, who is just amazing. He's been heavily involved in helping us develop that. And again, Excel, worlds must be the language, people don't usually think that. You think of things like JavaScript, but those are like in the maybe 10 million developers. Excel is literally, there are hundreds of millions of people who know how to write formulas, right? It's by far the world's most popular programming language. Brian Jones (01:18:08): And there are certain things though that the product was always lacking. Some of the stuff we announced recently, things like Lambda and let, those were just basic primitives you'd expect to have in a programming language, right? The ability to have code reuse. So if you write a formula that's complex, do you have to rewrite it every time? And then if you realize you made an error, do you have to then go back and fix it everywhere? Or you use Lambda where you can just define it once, give it a name, use that name everywhere, and if you need to fix it you go to one place and you go and fix it. Lambda also added a whole bunch of really powerful stuff around like you can do recursion, it made Excel Turing complete. In theory, Excel was Turing complete because of VBA. But with Lambda, it means just the grid and calc and formulas is Turing complete, take all the VBA and all that stuff out of it. Rob Collie (01:18:52): The thing I really like about that feature is the name, because you hear it and you know immediately what it does for you. I mean, it just jumps off the page, right? Like I say, Lambda, and everyone's like, yeah, totally, I know what I'd do with that. I couldn't resist. Brian Jones (01:19:07): Yeah. Rob Collie (01:19:08): It's like, it's not reusable formulas. It's not. Brian Jones (01:19:11): No. There's a reason. If there's anything that you should take away, the things that we do or are doing are really long running bets. Lambda is... it's a geeky, underlying enabling tech that a lot of end user-facing experiences will start to show up on top of over time. And yeah, if you even look at like our blog posts, we were targeting a specific audience with Lambda that would actually get it, right? Over time we will add more. The other big one there is data types. I don't know how much you've looked at data types, but this will actually resonate with you from back in your XML days. If you think about all the innovation that's happened in Excel, tons of innovation around things like pivot table, conditional formatting you mentioned, all that stuff, formulas then stuff like XLOOKUP and crazy fast calc, there's been so much innovation, yet what you can type into Excel is no more interesting than like a notepad. Right? It is the most basic thing. You could write a string, a number or a formula. What the heck? Brian Jones (01:20:10): It's been around for 30 years and we haven't innovated in what can go into a cell. That's what data types is. We talk about it with just things like stocks, but underneath we have plumbed Excel so that a cell can be a deep, rich value, completely jagged data. If you look at like the Wolfram data types, it's hierarchical data that's all stored in that one cell, but it still works with calc. You can now have a cell value can be an image. You can compare two images with each other. You can say, does this equal that? You get true/false. It's kind of mind blowing what data types actually does. Rob Collie (01:20:46): I can do a VLOOKUP or an XLOOKUP or whatever against a table of images now? Brian Jones (01:20:50): Yes. Rob Collie (01:20:50): And I won't need like the camera tool. Brian Jones (01:20:53): Right. Now, the only way right now to get images is if you have a data types that have images and then you can pull that out through a reference. But of course eventually we'll just have where you can just go and insert your own image, right? But one of the partnerships that we did with Power BI is you can have Power BI data come into Excel as a data type. So I could have a set of data defined up in Power BI, bring that in as a data type, I can look at that data, I can hover over it, I see a card, it tells me where that came from. So now if I'm like, oh shoot, is that really what the product price is? Did somebody just copy paste this? Where did it come from? I know where it came from my hover over it, I see a card. It came from that Power BI source, right? Brian Jones (01:21:31): So it means Power BI can be your master data manager. Right? You can say, okay, I've got all my master data here, I bring it into Excel, and Excel I actually see where that data came from, I can trust it, and now I can go and do a whole bunch of calc around it. So the data type stuff, it's just, again, it's kind of scratching the surface a little bit in terms of the way that we go and talk about it, but it's some deep, deep plumbing. Brian Jones (01:21:53): And then the last piece, which is also about just new types of values is arrays. Right now I can go and have it where I can have a formula that returns an array of values, right? So I could have stock dot history and that one formula will return a huge set of data that goes into the grid, and then I can write another formula that references that result and I can have that formula apply to the entire thing in the array, right? So I can have an array of data that I can pass through the calc chain and do manipulations on that array of data. And so I can do that step by step by step process, manipulating that full array, where in the past, that was the beauty of Excel, is I do a step by step by step, but it was always with that single value, like I get a number and I'm going to go do a few operations on it to get the final result. Brian Jones (01:22:41): Now I can have a data type like a person and do multiple operations on it using the programming language, or I can have an array of data that I can do multiple steps on it. So it's the beauty of the grid and calc, but now I'm working against the world's data, I'm working against all sorts of objects, not just numbers and strings. Rob Collie (01:22:59): So I'm going to give you all the disclaimers. It's one person's opinion and it's an unsolicited opinion. I know that. Brian Jones (01:23:05): Yeah. Rob Collie (01:23:06): Right. But what better time to give you that opinion than when we're recording for the world to hear? So I think that of all the features you just mentioned that are pushing the envelope, like the state-of-the-art of Excel forward, I have this instinct that the one that's going to become most common, sort of like most ubiquitous, is Lambda. And so I want to project that sooner or later you're going to rename it, because I think it's probably more applicable to a wider audience. You know the book, or the concept, crossing the chasm? I think for early adopters, Lambda is the right name. It's not the right name for early majority. And I think there is one for Lambda. It's just so fundamental, right? Rob Collie (01:23:52): This is a problem. The lack of formula reuse is a problem that basically plagues every single formula writer ever. I can imagine living my whole life without, in a particular niche, without ever needing to manipulate arrays as if they were a single value. Now, there's plenty of cases where I can imagine that I would, right? It's not that I'm saying that it's a bad feature. I think it's an awesome feature, but man, Lambda, I just see Lambda as just like incredibly mainstream. I could be completely wrong about that, of course. But I think I'd have a better chance teaching Lambda to a broader audience than I would arrays for instance, even the new dynamic ones. So anyway, end of rant. It's not even really a rant, it's- Brian Jones (01:24:37): That's no rant. Rob Collie (01:24:37): As rants go, it's pretty polite. Thomas Larock (01:24:39): That wasn't a rant. Brian Jones (01:24:39): That's just an opinion. To me, I'm like, hey, tyranny either or. Rob Collie (01:24:43): I know. Friends like, I hear your business book concept and I have raised you my business book concept. Brian Jones (01:24:50): All the time, man. Rob Collie (01:24:52): You've read the same best 100 business book's cliff notes that I have. Brian Jones (01:24:57): I haven't thought about this from a crossing the chasm, early adopter thing. I think of it more like in terms of user persona, it's more closer to being truly like would identify as a developer. And a lot of the stuff that we've done here so far is at that layer. The place that Lambda's exposed as like a name is in the formula itself you build. But clearly as we go and build a UI on top of it to make it easier to go and create those, we wouldn't go and say like have a big button that says, generate my own Lambda. That's not really the way that it'd be implemented. But then the data types in the array stuff, I think you'll be surprised over time. I get that you're not... like it wouldn't be a thing that you'd say, "Hey, I'm going to go and teach somebody arrays," but I think that what you'll find is it'll just almost naturally start to become how you work. Brian Jones (01:25:42): It's just like data types is almost one of those things where you say, I don't have a class right now where I say I'm going to teach you about cells, let's talk about cells and what goes into a cell. And so this is really just saying we're just upleveling what can go into a cell. And so eventually nobody really even thinks about it. It just will seem natural. It's like, well, of course you could put that stuff in there. Rob Collie (01:25:59): Well, so it's not about... It might've come across this way. It's not about a lack of belief in these other things. It's more like, I just think the bar size, the pie is bigger. And the example that I can't get out of my head, it's something that I wasn't exposed to until I come outside of Microsoft, all these income statement forecast spreadsheets, giant spreadsheets that cover acres of screen real estate of this like projecting forward month, month, month, month, month with different variables being fed in and different growth rates and different attrition rates and all of that, and all of that intermediate calculation, all of that real estate is just to produce a very small handful of outputs. So it's like I start with a small number of assumptions and some rules, and then I expand that, blow it out across this whole spreadsheet that thankfully can be more than 256 columns wide now, right? And then all I do is harvest those three values at the end. That's all I really care about. Or what's my break even month, how long before I break even? There's just a handful of outputs. Rob Collie (01:26:59): And I end up with this gargantuan device and I can't iterate on it. It's like I changed the inputs and then I want to save off the outputs. I just want to turn that whole thing into... and then just turn it into a table on another sheet, hide that machinery behind the scenes. And I just think that's just so incredibly mainstream, like everyone has this problem. And so that's why I just can't get this one off my brain. So it's really an obsession with the Lambda feature as opposed to a lack of interest in the others. Because I've already used the others, too. Brian Jones (01:27:29): Yeah. The idea around reusable logic and even going further, right, like Lambda's that I could go and create and then call from outside of Excel, a Lambda as a web service you could publish, right? There's a long way that we'll be able to go with this. And so I think that it's a huge, huge opportunity for us. Right now what we've released is just kind of the beginning underpinnings of a lot of innovation that will be coming. Rob Collie (01:27:58): I have to say as a long-time observer of all this, former insider on exactly that product, I think it's really truly exciting. It's real innovation. It's not keeping up with the Joneses. It's not like making sure you have parody with your competitors. And it's also not the old disease of, wouldn't it be cool if? It's not the old stuff like the things that we were doing with XML, right? It's grounded in true, actual value for the customer base, for the people of the world, and is innovative. That's hard to do. It's doable, it turns out, but there's a lot of discipline to it and I salute the direction that y'all been headed in. It really is exciting. Brian Jones (01:28:38): Yeah. Thanks. The thing that's been fun about that project is the combination of, you know them, there's a bunch of people on our team who their background is like financial modeling. Right? And so in addition to us researching the customer, we hired the customer. Right? Which, is great. And then we've combined that with the folks in, like Simon Peyton Jones in MSR and Andy Gordon, this combination of computer science experts. Right? And then people who deeply understand the customer problem. And like you said, it's really easy to go and see, what are those challenges? Well, one is when I'm looking at my model, can I trust the data, where'd the data come from? Right? And so this partnership with Power BI around data types helps a ton with that. Can I go and write logic that's reusable, so it's not fragile? Lambda's going to be huge for that. Brian Jones (01:29:26): And then even our collab stuff, we announced that in our web app now, and it'll come to the desktop, I can now go and write... if I see a value and I don't know where the hell that came from or why did it change, I can right click and say, show changes, and it will actually show you the edit history of that. Rob Collie (01:29:39): Wow. Brian Jones (01:29:40): So I can see, oh, Rob edited this two days ago, I can now go and contact Rob and say, "Hey man, what's going on? Why did you change my formula to a hard-coded value? That's not good practice, man." Rob Collie (01:29:49): For yucks. Brian Jones (01:29:50): Yes. Rob Collie (01:29:51): I just wanted to see if- Brian Jones (01:29:52): Just for fun. Rob Collie (01:29:52): ... you're paying attention. Yeah. Brian Jones (01:29:54): I wanted to get the output I was hoping for. Rob Collie (01:29:56): Yep. That's what we do. We cook the books. Well, I can't keep up, both because you're doing so many things... It used to be a lot easier to keep up. You're doing too many things for me to keep up, but also it's weird, Excel has now reached the point where it's not the app that I spend the most time in on my computer. I'm in Power BI, I'm in Outlook, I'm in PowerPoint a lot. I'm a mascot. The people who work at our company are so much better at the tool. So I'm not even really in Power BI that much. Brian, thank you so much. Brian Jones (01:30:29): Yeah. This was fun, man. It was great catching up. Rob Collie (01:30:31): I think people are going to enjoy this, and if they don't, we won't know. Brian Jones (01:30:34): All right. Thomas Larock (01:30:35): Yeah, right. Brian Jones (01:30:38): It's like how we used to ship software, right? Rob Collie (01:30:39): Yeah. That's right. Actually- Brian Jones (01:30:40): Throw it out there and move on. Rob Collie (01:30:42): Actually, I hate to tell you, we have detailed files. We have an amazing Power BI report that uses robotic process automation, courtesy of Ash, to go export the data from the unfriendly portal. And then we power query it. We'll be able to tell you next week how you're... how you're trending. Brian Jones (01:30:59): Oh great. Rob Collie (01:31:00): Are you trending ahead of Ken Paul's or are you behind Ken Paul's. If you want to know where you rank in the world we'll tell you. Brian Jones (01:31:07): Please don't tell me where I'm ranking. Ignorance will be bliss. Rob Collie (01:31:12): Yeah. Well, just seriously, it was a real pleasure. How often do you get to talk to someone who's in your role and how often do you get to listen to someone who's in your role? I don't think very often. So I certainly appreciate it. I think our listeners will as well. Brian Jones (01:31:25): Well, I had fun, man. It was great catching up. Announcer (01:31:27): Thanks for listening to the Raw Data by P3 Adaptive podcast. Let the experts at P3 Adaptive help your business. Just go to p3adaptive.com. Have a data day.