Raw Data with Rob Collie

He's known as the Man of a Thousand Jobs and he's done it all. From horse farming and training, to security at Rupp Arena, to using dynamite to blow up mines, to analytical chemistry and everything in between. Kevin currently works at Eli Lilly and Company and his path to Power BI was anything but a typical data journey. He's a fine Southern Gentleman with a mind for data, he's Kevin Overstreet. Episode Timeline: 2:07 - Kevin's Data Salvation, his impressive pedigree, and his amazing career path 9:17 - Data challenges at Eli Lily and how they've overcome them using the right tools 19:11 - The Wild West that is Sharepoint 24:00 - Rob launches one of his World Famous Monologues, this time it's about new branding for BI 30:13 - Does solving a problem have to be complex? Kevin discusses a great work mindset to overcome certain obstacles 33:00 - The Impostor Syndrome 36:30 - Dataflows and how to keep data secure 39:45 - Whose problem is it-The Business side or the IT side? 45:30 - The Power of the new BI Tools 48:36 - The story of how Kevin met Rob reveals a character trait that is valuable Episode Transcript: Rob Collie (00:00): Hello, again, data people. Today, we welcome Kevin Overstreet to the show. Now, Lon Chaney was known as the man of a thousand faces. Well, Kevin Overstreet, I call him the man of a thousand jobs because he has had almost every job you can imagine from horse trainer, to analytical chemists, to dynamiting mining, making roads construction guy, and all points in between. But for the past six years, Kevin has been a Power BI and Power Platform professional. He works at Eli Lilly, and he provides a tremendous amount of essentially like anything glue, to make everything run. Rob Collie (00:37): We crossed paths a long time ago when he came to one of our Power BI classes, one that I taught in Cleveland, but he's not a one trick pony. He is really up to his eyeballs in the entire Microsoft stack, especially in the Power Platform, and he is a really entertaining conversation, and I hope you enjoy it. Let's get after it. Announcer (00:58): Ladies and gentlemen, may I have your attention please? Announcer (00:59): This is the Raw Data by P3 Podcast with your host, Rob Collie and your co-host, Thomas LaRock. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Raw Data by P3 is data with the human element. Rob Collie (01:17): All right. Welcome to the show. Kevin Overstreet. Kevin Overstreet (01:21): Hello. Rob Collie (01:22): How are you doing, Kevin? Kevin Overstreet (01:23): Good. Rob Collie (01:24): As usual, you're joined by myself and Tom, or Thomas LaRock, depending upon, whether you use the professional version or not. Well, Kevin, let's get right to it. I'm really excited to have you on the podcast. I think it's going to be a real treat. Let's start here. Why do we know each other? What do you do these days that is relevant to data and the Microsoft data platform and business application platform? Just give us a quick summary there. Kevin Overstreet (01:53): I work at Eli Lilly. Although I work at Lilly, all my opinions are my own, but one of the things that we do is we deal with massive amounts of data every day. Previously, years ago I was doing that in Excel and then one day I was searching for how to figure out a distinct count of items in a pivot table, and that was where I stumbled upon the Power Pivot Pro blog. Rob Collie (02:19): Oh yeah. What year roughly was that? Kevin Overstreet (02:23): I think that was around 2010, 2011. It wasn't very long after Power Pivot was first released and you had started the blog that we kind of gravitated and found each other. At least I found you at that point. Rob Collie (02:37): That's been a long road, hasn't it? We like to joke about, and you like to joke about, you being patient zero at Eli Lilly for this stuff. There's a lot of people who started out in Excel and found their way to this stuff. That's no accident. You're not the usual Excel story. What's your trained discipline. If you're at a pharmaceutical conference, how are you going to introduce yourself? What's your background? Kevin Overstreet (03:04): At one point, I was an analytical chemist. Now, before that, I was actually in the department of immunology at the University of Kentucky and we were doing brain tumor research there. Technically, I'm a biologist/immunologist. Rob Collie (03:18): Yeah. Some really lightweight stuff going from immunology and cancer research to analytical chemistry, which I don't even really know how that differs from chemistry, but it's got a cool adjective on the front of it, doesn't it? Yeah. Kevin Overstreet (03:34): It does. Rob Collie (03:35): But as a personal matter of interest, I call you the man of thousand jobs. Kevin Overstreet (03:40): That's true. Rob Collie (03:41): You've already dropped some serious interest there, but everyone's got a different path to this stuff. You probably have a unique path. What other jobs have you had Kevin? Just like a softball of a question. Kevin Overstreet (03:51): I did not have an IT background, so I grew up on a horse farm in Kentucky where we had about 300 horses when I was growing up. I think my first job was probably hauling hay and cleaning out horse stalls for my dad at 50 cents a day, which he docked us on if we didn't think he had done enough. That started at about 12-years-old. A friend of ours had a pizza parlor. So, I was in charge of the big, giant, deep fryer, this high pressure deep fryers, which you probably couldn't do today. Rob Collie (04:25): Every pizza parlor needs a gigantic deep fryer, yes? Kevin Overstreet (04:29): That's true. Rob Collie (04:30): Because you need what? Fried pizza. Kevin Overstreet (04:32): Whether anybody remembers this is definitely me dating me, it was the shaky pizza parlor. Not only did they have pizza, but they also had fried chicken. Rob Collie (04:41): Oh, okay. Kevin Overstreet (04:42): So, I dipped my hands down into the hot grease to fried chicken at 12-years-old. Then around 14, we built a racetrack on the farm and started training horses. So, I was training horses there for several years. We've had about three world champions come off of that farm. There, I went, while I was in college, I was a security guard at Rupp Arena. We did security for all the concerts and everything there. I hung out with a Rat one night, if anybody remembers Rat. During time off from college, I worked for a communications company that was installing all new phone system at Auburn University. Kevin Overstreet (05:23): I worked for a construction company where we were installing a water main in the middle of winter, which was a little chilly. The great part about that was it was over an old strip mine, so we had to dynamite out holes for the pipe, and probably not quite as many safety measures back then. Our safety measure was taken off running and as fast as possible and waving our hand because we were setting off the dynamite with telephone wire on a car battery. Rob Collie (05:52): Really just your average usual, like in Austin power, it's just your typical usual stuff really. Kevin Overstreet (05:58): That's it. Yeah. Nothing to see here. Rob Collie (06:00): Nothing to see here. And despite all of that, the last decade, basically. Kevin Overstreet (06:05): Yep. Rob Collie (06:05): You've been in our world, and you left all the dynamite and analytical chemistry behind. What does an average work week look like for you? What are the things that you're doing with the data platform? Kevin Overstreet (06:16): We have a massive SharePoint installation. I guess one of the great things was, was just about the time I was discovering Power Pivot, I was also discovering the power of SharePoint. It had kind of always been this black box at Lilly of this is our sharing platform of the future, and then they put out about three PowerPoint slides for how to use it, which is probably not a whole lot different, but I'm a huge SharePoint fan. At this point, we manage a lot of one off processes through SharePoint. Kevin Overstreet (06:50): Not only the data management, but the data input, sort of whatever that user facing front end is. We're doing that with Power apps. We're doing huge amounts of things with Flow. Flow has been incredibly powerful. We've taken processes that used to take us 30, 40, 50 hours a month and now have completely automated that using Microsoft Flow. Those have been incredible opportunities, things that we found. But on a daily basis, probably 95% of my day is in one of the Power Platform tools. I'm either in Power apps, Flow or Power BI. And we sort of see all three of those things going together in any solution that we deliver. Rob Collie (07:33): How does that workload, the things that you're working on, that 95%, how does that contrast with your sort of official position, your official job duties? It'd be hard for me to imagine that your official job duties are defined as that 95% that you're currently doing. Kevin Overstreet (07:52): That's always been a little weird for me no matter where I'm at. My job description, when I was an analytical chemist, I was still doing a lot of data management there and building tools in Excel. At that time, of course, it was all VBA and pivot tables and moving information around. But I guess I've always had an interest in that. When I was, technically, I was still an analytical chemist. At that point, I had already moved into a data role about 50% of the time. Kevin Overstreet (08:22): Yeah, now, for the most part, I'm known as the SharePoint guy, or the Power BI guy, or the Flow guy, whatever it is, typically, if there's something going on or somebody has a problem, eventually those things find their way to me. Because I have been using it, like you said, for about 10 years now. Thomas LaRock (08:39): Kevin, the question I have, because I'm always fascinated by data, because first of all, I think you have a fascinating background. Love the accent. Before I forget, I wanted to mention, since you're Kentucky and horse racing, all that, I was really happy to see Bob Baffert win the Kentucky Derby, because if there's ever a guy in this world that needs a lucky break and something good to happen to him, it's Bob Baffert, so I just thought you'd appreciate that. Thomas LaRock (09:07): Anyway, tell me something about the challenges you have with data over Eli Lilly, without getting really too. But I feel that you're going to share something, myself, Rob and our listeners will just sort of be like, yep, that's exactly what I go through on a daily basis as well. Tell me some of the challenges that you have. Kevin Overstreet (09:24): Sure. I mean, a lot of the challenges that we have are just compiling data. People have a lot of requirements, and for the most part, people are comfortable in Excel. I think in any company you're at, you're still going to see massive Excel reports that are out there that people are producing every day. Like most companies, we are aggregating massive amounts of data on any given day. We're compiling that into reports that are delivered either to specific people or maybe to a common location where other people are picking those up. Kevin Overstreet (09:59): And everybody talks about a golden data source, but I think that becomes even more important now as these tools get stronger and end users can begin to become more accustomed with them, they start to take those reports and files and do lots of crazy things with them. I think that's one of the things that the Power Platform gives us, is that way to sort of standardize those processes and deliver kind of a consistent product, no matter what that is, whether that's a visual or whether that's a report. Kevin Overstreet (10:32): Now, the other thing that I'm sure a lot of people struggle with, and especially in SharePoint, if you think of something as simple as document metadata. So, everybody is tagging documents with metadata and companies can have hundreds or thousands of SharePoint sites across that, and of course everybody's tagging it differently, and that can be project name, whatever it might be, depending on where you're at. Kevin Overstreet (11:00): But one of the things that we've been able to do with Power Automate is basically create one SharePoint site that we consider our master metadata and then we populate that out to every other SharePoint site sort of in our functional area there. Now, everybody's using the same metadata. So, when it comes time to compile that data, it's very easily to take Power BI connect into those SharePoint sites and be able to pull together a really consistent list of information. Because if not, over time, people record things differently, they use different terms, but then at some point somebody's going to come and say, "I want all of that data." And then that's going to result in a two or three week sprint of everybody trying to decipher all the different ways that data's been stored over, possibly years. Rob Collie (11:51): So how? The naïve way to look at it would be to say, well, of course, Kevin, you were in charge of all of these SharePoint sites, and you came up with the Schema, that was standard. Then you just rolled it out because everyone had to listen to you. I just don't think that's how it happened. How do you heard the cats like that to get everyone on the same page with such a thing? Kevin Overstreet (12:12): It's hard to get people on the same page, and I don't think by just rolling something out and saying, here it is, you're not going to get much uptake on that. People are locked into their old ways of doing things, no matter how any efficient that is, you can go to somebody and say, I'm going to save you 50% of your time. They go, that's okay. I'll just stick with what I've got, because they're uncomfortable doing something new. I think what you have to do is you have to deliver something, and this does end up in a very iterative approach sometimes, but you have to deliver something that has a personal impact to people, where they can physically sort of feel that change of, oh, this is going to make my life easier. Rob Collie (12:56): Deliver, then explain, rather than explain than deliver. Kevin Overstreet (13:02): Yeah. I tend to talk in the air a lot and people can't necessarily follow that when I'm drawing on my virtual whiteboard that's in front of me. When people come to you with an idea, they're basing that idea on what their experience is. They'll start with things of, I need a form that does this, or I need an Excel file that does X, whatever that might be. A lot of times they're just going based on what experience they've had and what they know is out there and may not even know what the possibilities are. Rob Collie (13:37): The reason I said that deliver then is it hearkens back to questions that I've been asked over the years, which is ... I get asked this less often lately, which is actually really encouraging. But for a long time, I would be asked, hey, how do I get buy-in in the broader organization or in my sister organization, the peer organization, how do I get them to buy into the power of, let's say Power BI? The advice I would give them is don't try to play, my software can beat up your software. Don't try to explain it like it's a conflict of two tools. Rob Collie (14:13): Just go and build something that was impossible. Go build something that was impossible and then give it to them and say, see, this is it. It sounds like your philosophy, even though I don't think you and I have ever actually talked about this. It sounds like your philosophy has found itself with that exact same method. Kevin Overstreet (14:31): Yeah, I think so. Over time, before this was kind of my official role, the tools that I built were for things that annoyed me. I don't want to do that anymore so I'm going to build a tool and that's what I always tell people now. If I have to do it more than twice, I'm going to automate it in some way. That's it. I can remember we built just a scheduling tool and it was because I didn't feel like walking around on the floor and writing my name on one of 50 whiteboards for a reservation when we've got ... We're all setting at our computer, at least half the day, why not build a reservation system in SharePoint? Rob Collie (15:08): That's great. There's so many things that are interesting here. One of them is, is that a lot of analyst types, I would count myself in this, a lot of analyst types, we kind of want to join the story after the data has been created. I gravitate towards, let's go look at what you're already recording about your business and see what that's telling us. Of course that's super valuable. That's, in a lot of ways, central. That's what BI is. Rob Collie (15:41): One of the things that I really like about you is that you don't perceive that artificial boundary. You're really willing and excited about engaging upstream, even in like what I would think of if I wanted to really paint it with a broad brush, I would say that you're involved in the data collection process, the data creation process, like you get into the workflows that are upstream from analysis, and I'm going to going somewhere with this overall theme, but I want to stop and see what your reaction is to that. Kevin Overstreet (16:17): I do agree with that. I think that the one thing that I've always liked is solving problems, and I think as I go back, and we were talking about all my jobs before, I think that, especially as I've gotten older, the jobs that I really joyed, even if the sort of subject matter was different, was the ability to solve a problem. Whether that's developing a new analytical method, or in this case, developing a new process or form, but sort of relating that to your, I don't want to come in till the data is there. Kevin Overstreet (16:52): I really got started in collecting the data for ... Someone was asking for me to analyze data that was in no shape for me to be able to analyze. So, it took so much manual preparation. It was like, I'm just going to go in and build something that's going to collect the data the way that I want it and the way that I need it. I think Power Pivot kind of revved that up a little bit more in 2010, when all of a sudden, now I can deal with millions of rows of data and I can deal with millions of rows of data in SharePoint. Okay, now I have no limitation to how much data I can record. Kevin Overstreet (17:27): I'm just going to record everything and I'm going to figure out what I'm going to do with it later, because I think there's value there. Rob Collie (17:33): Tom likes to talk about no one set out to be a data janitor. Thomas LaRock (17:37): That's right. No one went to school to become a data janitor. Rob Collie (17:40): That's right. Thomas LaRock (17:41): And you replied to me, and yet here we are. Rob Collie (17:44): Yeah, here we are. There's a little irony in that. At one point in his life, Kevin was cleaning out horse stalls. I think the Power Platform's got a lot to it, but I'm not sure that would really help you with the horse stall cleaning. Kevin Overstreet (17:59): It probably would with the horse racing. I've gone back and I've talked to my dad now about, oh, if we'd only had these tools back then, the things that I could do. Because I can remember my first computer was like a 386SX with a whapping one megabyte of Ram and a 40 megabyte hard drive, and I think I spent about $200 to update that to two megabytes. I was using Quattro Pro and dBASE III Plus. I was trying to do all these things with horse races. Kevin Overstreet (18:32): Even back then, I think I kind of had an interest, but in my everyday life, I don't find anywhere that you can apply some of these tools. We've had that conversation before about, anybody who has any small business at all, now would just be crazy not to jump into these tools and get a subscription because of everything that it can do for you. Thomas LaRock (18:54): What I like hearing is you talking about a success story and you're combining a couple of tools, Excel and SharePoint. Like you said, back in 2010 was the, I'll call it the first of a game changer, we'll say, right? Kevin Overstreet (19:10): Right. Thomas LaRock (19:10): Because before that, in my background being a production DBA for seven years, if anything touched SQL server, all of a sudden it was mine. SharePoint being one of those things. Back then, SharePoint was ... It was ugly. It was a mess. It wasn't something that will have the scarves. You say SharePoint, and I'm like, just get away from me, man. Just [crosstalk 00:19:34]. When I hear those stories, the thing is, is at the time, those stories didn't exist, or if they did, they didn't really resonate with me. Hearing it now, I have so much more appreciation for the collaboration that, that has fostered. Thomas LaRock (19:49): Here we are 10 years later and we see these tools, and I see them in a much more positive light. Tell me some of those success stories for you where you are. Have you seen that same thing where it really fosters more of a collaboration, there's no real stigma with it. Nobody's saying, oh God, you're using Excel or you're still writing code in Python, that type of gate keeping, like it really has helped foster and made things better. Kevin Overstreet (20:18): I think so. I mean, especially in that kind of citizen developer role. You, as the SQL guy, you had access to plenty of SQL servers and everything, where to store your data. When I first start out, SharePoint was just the only tool that was available to me. I had zero IT background. Other than some fairly basic VBA, I wasn't going to go in and write crazy SQL code or anything. I just started using the tools that I knew how to use at the time, which started to be SharePoint and then Excel, and then, as those things, then Power BI came along, and then now the Power Platform. Kevin Overstreet (20:59): I think those things just build over time. I know that Rob is constantly talking about people with the data gene. I think that's what you find. I mean, I still think SharePoint has a lot of detractors no matter where you're at. But I think part of that is, SharePoint is still kind of the wild west. If you're going to let people just create sites and do whatever they want, there's going to be some crazy things that are going to happen in there that are going to frustrate you for sure. Rob Collie (21:27): Well, now that SharePoint has moved largely to the cloud and online and it's Microsoft servers that's running it for most people, all those problems that you're talking about, Tom, they don't come to you anymore. But yeah, I completely understand. As someone who was sort of like thrust into managing a SharePoint firm 2010, because there was no alternative, oh my gosh, this is something that you would give to your enemies, was managing that. Rob Collie (21:57): From a SQL perspective, nothing could have been more unclean. There's no abuser of SQL quite like SharePoint. Kevin Overstreet (22:03): Well ... Rob Collie (22:03): Okay. Okay, people. Second to people, is that other thing. Thomas LaRock (22:13): There's still some products. There's other products. We don't have to name them. Rob Collie (22:19): Yeah. But really, SharePoint's like ... That's a product that's really kind of come into its own, I think. The stuff from 10 years ago is not remotely relevant anymore. But boy, that was not a lot of fun back in the day. Kevin Overstreet (22:32): I think people get overwhelmed by SharePoint when they begin to learn how to use it, because you search and there's all these things. I'm just a basic guy. I've never gone, I've never written one piece of custom code in SharePoint. Everything I've done has been out of the box. I've never created a content type. Everybody talks about content types and SharePoint. I've never created a content type in SharePoint. Could not tell you how to do it. Typically, whatever's there is what I use. Rob Collie (23:02): Kevin, it just now struck me, you remind me of Phil Hartman's character on Saturday Night Live, the unfrozen caveman lawyer. I don't understand this stuff. I'm just a simple guy. I've never created a content type in SharePoint. I've never written any custom SharePoint code. However, I've been a cancer researcher. I've been an analytical chemist, and on top of that, I've blown things up with dynamite. You didn't drop this earlier, but I'm going to drop it for you. You've won every horse training award that there can be won that's awarded by the state of Kentucky. Kevin Overstreet (23:36): That is true. Yeah. Rob Collie (23:38): You're just a simple man. You're really just an every man. We're all basically you, is what you're really trying to tell us. Right? Kevin Overstreet (23:44): That's it. I'm a basically A to B kind of guy. Rob Collie (23:49): Yeah, that's it. That's it. Simple. So simple. If only everyone were as simple as you Kevin. These podcasts always end up with at least one small, and not necessarily always small, but there's always some moment of Rob monologue, and I'm aware of this. Here it is. We're entering the monologue zone. One of the things that I have been witnessing and watching as a professional over the last 10 years, especially recently, is this notion that BI, business intelligence, analytics, whatever you want to call it, it's been its own thing. It's been its own discipline for a very long time, but that's only because it's been so bad. Rob Collie (24:36): When something is really bad and it hasn't delivered on its promise, or even on its need, a lot of attention gets focused on it, and it becomes a thing, almost like a goal in and of itself. BI, the I stands for intelligence. Why? Why do we need to be intelligent? We only need to be intelligent because we need to improve. We need to improve our business. I'd like to mark today, the small number of us, we try to start a movement where we rebrand the eye from intelligence to improvement. Rob Collie (25:13): Being informed is only meant to power improvement, and this is where I think that the story comes back around to you, Kevin. Because I think you don't come into this story with this like, it's BI or nothing mindset. It's just problem is problem. I talked earlier about how a lot of your activities, a lot of things that you do are about optimizing, even upstream of analysis. You're saying, hey, in order to get a better picture for our business, we need to improve the data collection process. Rob Collie (25:45): I think that's just incredibly noble. That is the right mindset. At the same time, when you're looking at a dashboard or report, whatever you want to call it, all the nouns, get confused, but really people think of them as dashboards, when you're looking at a dashboard and you see something that needs fixing, or you see an opportunity that you want to go after, there's also a downstream from the report, from the dashboard. Rob Collie (26:11): I like to talk about this as the action loop. There's upstream, and then there's noticing an opportunity or a problem, and that's what BI does. It helps you notice things, and then you take action on it, and then you see if the action worked. Did it actually have the desired impact? How much, if any, work have you've done with the Power Platform that's extending sort of downstream from power BI. We talked about upstream, the collection of data and things like that. What about downstream? What about the taking action phase? Have you done anything there? It's okay if you haven't, I'm just actually curious. Kevin Overstreet (26:43): Yeah. I mean, I think, if what you're talking about, we've started to implement more Power apps into dashboards so that people can annotate data in real time, in some cases, depending on what your application is, you're limited by your dashboard refresh rates. And other times, you can show that information in real time right in the Power app. I think, as we move forward and those tools continue to mature, and we begin to learn more how to do those things, that that's going to be the thing. Because no matter what, if you've got a number in the box, there's always context around on that. Kevin Overstreet (27:23): You're never going to be able to provide all of the context that everybody wants in that dashboard. So, giving people the ability to add comments to it, and then make that available in a popup in the dashboard, a popup tool tip where somebody can hover over something and see comments that people have left over time or questions that they may have had, or even to the point where we implement a Power apps and SharePoint sort of solution in order to be able to correct data that we're getting from other places. Kevin Overstreet (27:57): Basically, we're looking, if there's a value in our tool, we're going to take that value over the value that we may be getting for from another place. I think there's lots of opportunities to sort of affect that in both ways, but yeah, particularly upstream, and I think those opportunities are just getting bigger all the time. Rob Collie (28:15): You make that sound so easy. Thomas LaRock (28:17): He really does. Rob Collie (28:19): It's just like second nature. My grandfather, who you would've liked, had a saying, which was like, it's very difficult to fence in an earthworm. You, Kevin, are the earthworm. Kevin Overstreet (28:31): It's true. Could be. Rob Collie (28:33): Oh yeah, this is the box. No, no, this is not a container. I am going to show you that we can solve anything. Yeah. You should be the poster child for the entire Power Platform. Kevin Overstreet (28:45): A lot of times, everybody, they look at these things and go, oh my gosh, that's so complex. How did they do that? The very first form that I ever built that I integrated into Power BI was just a customized list form from SharePoint where I just went in and said customized form, and then just went in and figured out how to re place the data sources with the data from Power BI. These don't have to be complex solutions to sort of generate huge value and change. Rob Collie (29:16): Kevin, I'm going to ask you this question very delicately. Kevin Overstreet (29:20): Okay. Rob Collie (29:20): Have you ever considered a career in consulting? Kevin Overstreet (29:26): I think I'm too old now. 10 years ago, 15 years ago, I may have considered that. Now I'm getting older. I think more importantly, my dad's getting older, so at some point, somebody's got to go back and start hauling hay again, and I guess that's going to be me. Rob Collie (29:41): Ladies and gentlemen, Kevin is 37-years-old. Someone's got to haul that hay. I mean the hay doesn't haul itself. Kevin Overstreet (29:56): Yeah. I always say, I'm 53 now, but I really feel 43 because there's like 10 years in there that are a little iffy. Rob Collie (30:03): And who are you if you haven't had a 10 iffy? Kevin Overstreet (30:07): That's right. Rob Collie (30:08): You've got to have that. Thomas LaRock (30:10): What really just resonated with me just now is how Kevin, you mentioned, it doesn't have to be complex. All you did was populate that list, but that made a difference for members of your team and your organization. It wasn't complex. It was just that nobody else was going to do it. So, it's part of you being the person that, not only knows what needs to be done, but is willing to roll up your sleeves and go do it. I think a lot of that just goes back to who you've described yourself as a person. Cleaning out the stables as a kid, working the deep fryer, everything you've done. Thomas LaRock (30:48): These aren't glorious jobs, but it's work, and it's work that somebody has to take on, and you're the guy that's just, well, yeah, I'll see if I can do it. I think, for a lot of people, they kind of come at things just a little bit differently. I don't know what we need or I know what the problem is, but I don't know how to solve it. I don't know where to go. And they're just not willing to roll their sleeves and read a manual type thing. Kevin Overstreet (31:11): Yeah. I do, do a lot of reading. I've said before, I think I've got every Power BI book that was written, probably not recently, but from about four years ago, historically, I probably had every Power BI book that had been written, and three of those were autographed by the author. The chemist may be the biggest geek in the room whenever I'm around. But yeah, I think that people get caught up in those processes that they're in every day. It seems overwhelming to try to change from that. Pat, one of my colleagues at work, he always says, look, if you take 10 minutes today and learn something, it's going to save you 30 minutes, and then take that 30 minutes tomorrow and learn something else. Kevin Overstreet (31:58): Then, eventually those things are going to kind of snowball, and even when you were talking about problem earlier, you don't necessarily have to solve the whole problem, but you can always start by solving part of the problem. If you see that a solution to a problem is emailing around an Excel sheet where 50 people are making changes in it, then somehow you're supposed to take all those versions and put them together in some coherent way, there are ways to fix that. A lot of times, I think the thing that you struggle with the most is probably just behavior change. Rob Collie (32:33): It's always those people. Those people, they just don't do what they should, do they? Kevin Overstreet (32:39): They won't listen. Thomas LaRock (32:40): It's always a people problem. Rob Collie (32:41): It's always a people problem. I agree. We identify as technology professionals at our own peril. What we are, are people who are able to use technology in a human context. If you start identifying yourself as a technology professional, you kind of are already defining yourself into failure in a way. Thomas LaRock (33:01): Yep. Rob Collie (33:02): I did have one other thing I was thinking while you were talking just in Kevin, the way you were talking about these problems and how just sort of the way you approach them and everything, it's not a risk to say that a large percentage of the people who are listening to this suffer with imposter syndrome, this belief, this uncertainty that, maybe I'm just not that good. Maybe I'm not all those kinds of things. This comes from humility. It comes from questioning yourself. You really can't stop doing those things. Rob Collie (33:31): You always need to be questioning yourself. Otherwise, you never grow. You don't treat other people correctly. It is the way to be. But it does, it results in imposter syndrome. What I've found over the years, and what we found as a group, is that the only antidote to imposter syndrome is understanding, I'll use Kevin as example, understanding that people like Kevin, or yourself, whoever's listening, that most people aren't like you. Rob Collie (34:00): The things that you're saying, Kevin, you do, you make them sound so simple. Of course, it's just this kind of problem and we're going to approach it this way and we're going to solve it. To you, that doesn't sound that special. It's obvious. It's almost like table stakes to you. I'm not going to say there's only one you, even in your workplace, but there aren't that many. You stand out. People come to you from all the way across the organization, people who don't even work in the same building. Well, we don't work in buildings anymore, do we? Rob Collie (34:28): But back when you worked in buildings, they would come from other buildings to seek out your help and your advice on whatever their project was. That's because you do stand out. Our team, our whole company, we're populated with people who do experience imposter syndrome despite being very, very good. You just got to understand that there is something, even though you're questioning yourself all the time, you are kind of special because no one else is really doing those same things. Kevin Overstreet (34:55): Well, I appreciate that. I mean, we've had this conversation before that, I don't feel like I'm been doing things that are that complex. I've told you, when I get on your site or I get on other sites and I look at the things that people were doing and the DAX code that they're writing, even like Pat, my colleague and friend that we talked about earlier, he has a blog post on your site where he calculated like 10 billion nations for like Texas Holdem or something. I'm not capable of doing those things. Kevin Overstreet (35:28): I kind of always have an interest in stuff, so when something new comes out, I at least want to investigate it. Although I may not be able to solve something immediately and I'm not going to write these incredibly complex solutions, I think that over time, I have found approaches that work to problems and you kind of try to categorize those, and then use those proven approaches that you know are going to work. I mean, to this day, and especially now with Power Automate, I still do a lot of stuff with Excel and Power Query, where I may need to pull information from one location and transfer that to another while I just make a Power Query connection to the data location. Kevin Overstreet (36:13): I can then drop that Excel file into a SharePoint and have Power Automate, go and create those items as many places as I want automatically. It's just sticking with those things that I know have worked over time. Rob Collie (36:28): Do you have access to Dataflows? Have you had any exposure to it? Kevin Overstreet (36:33): We do. Yeah. Rob Collie (36:34): It seems like this is your next place where you just catch fire. Kevin Overstreet (36:39): Yeah. So, we've started exploring Dataflows a little bit. The other thing that I've started to get interested in is basically creating a Power BI data model that you can then publish into the Power BI service, and then that can be used as a data source for your power app. Now we're taking information that maybe is in all of these different locations, and we only need a little bit of that, but we need a little bit from every location. Now we can just use Power BI to grab all of that, throw that together and basically create a data source for a Power app or for Flow to operate against. Thomas LaRock (37:21): What you've just described to me, Kevin, is essentially nothing short of a nightmare. Because now, now you've got data everywhere, right? That's going to lead to other problems. Now you're at risk of somebody takes that power BI, that project, and they bring it home with them, or they put it on the USB and they leave it on the bus. How do you handle that situation? Because I mean, you've done a great job of curating your data, collecting it, moving it around, getting it to the right people. But how does Lilly make sure it doesn't walk out the door. And don't give away any trade secrets, but obviously it must be a topic of discussion. Kevin Overstreet (38:00): I can't speak for Lilly. I can only speak for just sort of in general, but I think that absolutely security has to be a top concern, whatever that is. I mean, I know now that a lot of companies, you can't even copy information to external drives. That happens. I've been through this discussion multiple times, because a lot of the times, I cannot necessarily get the access I need. I may have access to a reporting portal where I can get all the information I need, but I can't get direct access to the data. So, I developed some process to get around that, to get to that data that I need. Thomas LaRock (38:41): La, la, la, I'm not hearing that. I'm just going to put fingers in my ears. Kevin Overstreet (38:46): Here's the thing I would say. I always say the security that we need to worry about is not necessarily at the system levels, or even at the Power BI level where somebody else is controlling security. I mean, no matter what company you are at, there are millions of Excel sheets floating around through email and SharePoint and networks and wherever else it may be. I mean, I always think that, that's probably a bigger security concern than putting some in Power BI. Rob Collie (39:19): Oh yeah. Tom's reaction to what you were just saying. I know Tom was being tongue in cheek, but at the same time, it is very much the stuff of IT nightmares, the earthworm that can't be fenced in. You're out there every day generating systems. You're generating, whether you think of it as code, sometimes you do, sometimes you don't, it's all code. You're generating solutions every day. I think the real problem here is that, as soon as something becomes a solution, as soon as it crosses a certain psychological threshold, now it sounds like IT's problem. Rob Collie (39:59): Whereas the spreadsheets don't most of the time. They don't sound like it's problem. Spreadsheets seem like they're the businesses problem, until those spreadsheets integrate with some sort of backend system in some sort of Rube Goldberg way that IT didn't anticipate, those spreadsheets are the business problem. But you start building things like automation workflows that go from SharePoint to this, to that, and then you decide to go back to bailing, hey, you go back to the farm. What's left behind is something that someone on the business side created, you, and now you're gone, and IT, they're not necessarily up on those languages. Rob Collie (40:40): They don't have what was in your brain. Now it's their responsibility. That's where the perceived trouble starts, or that's where the fear creeps in. Even if you're still there, you haven't gone back to bailing hay, you're not cleaning stalls. You're still just an analytical chemist masquerading as a data platform specialist. That's what you are today. What you've done though, has reduced the total surface area of problems for the broader organization. It's just done it in a way that now it starts to feel like it's IT's problem. That's a really interesting thing for us to come to terms with. Kevin Overstreet (41:17): I that's always going to be the case. Like I said, I'm sort of in a weird position where anybody who's on the business side sees me as IT, and anybody who's on the IT side sees me as a renegade or a cowboy or something, depending on the solution. Eventually, you've got to build those relationships with the people in IT who can help you get that done, I think, if you want to move forward, and especially, let's face it, eventually at some point, if you're doing enough, you're going to need IT's help to get some of that stuff done. Kevin Overstreet (41:55): Speaking from, on my standpoint, I'm not going to be able to go out and build some SQL server solution, even though that may be what I need. Now, I may be able to figure out how to populate that once you've got it for me, but you have to build that relationship with IT. I think part of that is, is just over time, I think, as I've began to develop more complex solutions, they've become more comfortable with me because I've been kind of doing it for so long now. Rob Collie (42:24): There's an old blog post, it's several years old now on our site, something about the Twilight Zone, where people who work for our company talk about how they'd found themselves in a similar spot. They didn't really belong to either world. The business viewed you as IT, IT viewed you as renegades. What I view you as is the future. This is where it happens. What you're talking about, where you're gaining acceptance over time from IT. You're seeking and increasingly winning their support. That's where things are headed. Rob Collie (43:04): You, off on your own, building these solutions without any IT visibility or any it support, that's bad. But the answer to that isn't to have you not doing it. Having you not doing it is worse. The right answer, in my experience, is that it needs to recognize you, people like you as their greatest ally, the ally they've never had. It's like having a bigger organization. There's more IT people out there. They're not going to get headcount to go and like triple the size of their IT org. But if they embrace people such as yourself, you're not exactly dime a dozen, but there's enough of you out there in the business. Rob Collie (43:52): If you were all embraced and sort of deputized and treated as ambassadors between the business and IT, IT will have a much greater capability, not of just to play defense, like they typically are required to, they can play offense with your help. They can be part of advancing the business goals, and not just keeping the lights on. It is really encouraging to hear that, I mean, you work for a really big company, and really big companies are the places where this sort of change happens the slowest. So, it's really encouraging to hear that it might not be like overnight. I think that's a really positive thing. Kevin Overstreet (44:27): I think so. I mean, I think that one of the things that helped me be successful is the fact that I was familiar with the processes and data that I was trying to work on. You were talking earlier about, had I ever considered in consulting work. I have to get in and sort of understand the data and the processes and what they're doing a little bit. I think there's a difference between building something from just a simple set of requirements that maybe somebody in IT has put together versus having an IT sort of technology oriented business person in there to help translate that into, okay, that's what this really means though. Kevin Overstreet (45:14): Because I think we've all been through those things where people go off and they build a system or they build a dashboard, and they come back and technically we've met all the requirements, but that's just not what we had in mind, or whatever it is. Rob Collie (45:27): Well, I mean, even if the business were able to, so this is the problem with the requirements gathering process. This is the problem with waterfall oriented business intelligence. It's super fundamental and it needs to go away. If the business were able to, in a reasonable timeframe, collect and convey their requirements, in a lossless form that was then transmitted to IT and understood without noise, without misinterpretation, without loss, and then implemented to that specification, first of all, that'd be the first time in human history that, that had ever happened once. Rob Collie (46:14): It just has never, ever gone down like that. It always takes forever. It's always littered with omissions. It's always littered with misunderstandings and misrepresentations. But even if you got there, as soon as the business saw exactly what they wanted, they would go, oh, you know what? We need something different. We didn't know better. This is why we've come around at P3 to the saying that human beings do not know what they need until they have seen what they've asked for. Rob Collie (46:49): The old tools were just so slow. It just didn't make sense. As a business stakeholder, you would never sit in the same room as someone who was developing like an old school ETL process to power a data warehouse that someday might have an analytical model built over it, that someday would put its first dot on a chart. You would never sit there and watch it. But today's tools move so fast that you can actually sit with the person who's building it and you can develop it, and you can just get started. Rob Collie (47:23): You'll find yourself, even after like the first couple of days, in a place that you ... It sounds like an exaggeration. It sounds too good to be true, but you can find yourself, after a couple of days, in a place that it would've taken you six or seven months to get to. By the time six or seven months have elapsed, the original need is already evolving, people are exhausted. They want to give up. They don't want to keep on it. It's not even a matter of like a couple of days versus six or seven months. It's a difference of we got somewhere versus we didn't at all ever. Rob Collie (47:57): That's a huge change. And boy, is the world slow to wake up to this change. You can deliver the technology all at once that makes this possible. It's those people again, isn't it? People take a long time to trust that the rules have changed. Kevin Overstreet (48:14): Yeah. I think that a lot of times people are locked into all of these old tools that they're comfortable with, that they just don't even know what's even possible, and that's going in and trying to explain what's possible, doesn't really work, so I think you have to give them something to react against. Like you said, give them what they've asked for. Rob Collie (48:37): We'd be remiss if we didn't close with the story of how we first cross paths. There's an old, and it's actually, it's just a truth, which is that the things that are hardest one are the things that we end up valuing the most. I think your journey down this path started with an accidental psychological operation committed by the universe against you. Kevin Overstreet (49:02): It did. Not long after I discovered Power Pivot, it was probably three or four months, so when I discovered Power Pivot, I discovered the blog. That was when I saw the training class, so I was like, going to go to the training class. I went through all the effort to convince everybody that this training was going to be hugely beneficial to me. It was in February, and I believe it was 2012, and so I had had to go from Indianapolis to Cleveland at the time. I left Indianapolis, and it was snowing, and by the time I was five miles outside of Indianapolis, I was right in the middle of what we now know as the Ground Hog day blizzard with 40 inches of snow across 74 in Cincinnati. Kevin Overstreet (49:55): I was driving the most appropriate vehicle, which was a 2010 Honda Civic Si hatchback. You can look that up. I basically was driving like a snowmobile. That was it. I fought my way through the 40 inches of snow and was now headed north towards Cleveland, when I started to notice that my lights appeared to be dimming. Then I realized that I'd had trouble with my alternator, but that had kind of cleared itself up. Kevin Overstreet (50:29): But at this point, I thought I was having alternator problems again, so the first thing I did was turned off the lights, turned off the heat. I got on the phone, called my wife and said, if we haven't joined AAA, do that now. So, I drove about 60 miles in complete darkness through what was left of the ice and snow to make it to a quality inn outside of Cleveland. At the time, I was coaching softball and I had like a training bat. Kevin Overstreet (50:59): It was about a half an inch around aluminum ride that was three inches long. While the car was running, opened the hood, down in the engine and start tapping the alternator to try to get it to kick back in. And all of a sudden, I hear the car rev up, alternator starts charging again. I jump in the car and turn on the heat. Because it'd been zero the entire way there. Finally made it to the hotel that night, went through the two days of Power Pivot training. At the time, I talked to Rob and I said, I just don't know. I don't know where I'm at. Kevin Overstreet (51:37): I asked him one question, and he said, "You're way down the rabbit hole." That was kind of the start of my journey on Power Pivot and everything. Yeah, what should have been about a three or four hour trip ended up taking about nine hours through 40 inches of snow and complete darkness with no alternator in the basically a little tiny sled. Rob Collie (52:04): Yeah. I mean, I don't even know where the alternator is in my car. I suppose I used to know. I mean, it's probably the thing that the wires come out of, right? But I wouldn't think to tap it. This is some real MacGyver stuff here. Thomas LaRock (52:18): Roll up your sleeves and get the job done. Rob Collie (52:20): Yeah. See a problem, fix a problem. What's the big deal? It's the Overstreet way. Kevin Overstreet (52:26): It is. I mean, it's probably not the safest way. There was multiple fan belts and fans and all the things spinning around that probably could have flung the ... Became a flying death rod. Rob Collie (52:41): I'm imagining you timing the fan blade. I got to get the bat in between the fan blades as it's spinning at 3,600 RPM. You're like, one Mississippi, two Mississippi. Kevin Overstreet (52:54): Yeah. I was reminded of the times when I was a kid and my dad would be, want you to put your hands somewhere and go, "Here, hold this." One time we had the axle broke off of a trailer and he wanted me to get underneath it and hold it up while he chained it up with a piece of chain that we had gotten by calling a trucker on the CB and he had stopped and given us a scrap piece of chain that he had. So, we were going to haul the horse into the track on three wheels. Well, we ended ... We did haul the horse in on three wheels with the axle chained up with a scrap piece of chain from a trucker, and the horse in that trailer would eventually become a world champion. Rob Collie (53:38): I mean, it's really just a story, a story as old as time, isn't it? It's a classic archetype. Let it be said that data people are not all cut from a single mold and it's never boring, is it? Kevin Overstreet (53:53): No. Rob Collie (53:54): And Kevin, thank you so much for being on the show. I know I've enjoyed it immensely. I hope everyone else does as well. We'll be sure to catch up soon. Kevin Overstreet (54:02): Well, thank you guys for having me. I've enjoyed it a lot. Rob Collie (54:05): We might have to have you on again. I just don't think that this is the last of Kevin Overstreet on Raw Data by P3. Kevin Overstreet (54:11): We could talk about all the different times I should have used data to prevent near death experiences on my farm or something maybe. Rob Collie (54:19): Yeah. The 11 times Kevin almost died if only he'd had data, Kevin Overstreet (54:25): That's it. Rob Collie (54:25): All right. Well, I'm glad you survived all of those occasions to make it to this point. Kevin Overstreet (54:31): I am too. Rob Collie (54:32): All right. Well, thanks again. Kevin Overstreet (54:33): All right. Thanks. Cheers. Announcer (54:35): Thanks for listening to the Raw Data by P3 Podcast. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Interested in becoming a guest on the show, email lukep, L-U-K-E-P, @powerpivotpro.com. Have a data day!

Hello fellow Americans! It's the day after the election, and who better to have on than sports and analytics journalist Michael Salfino from FiveThirtyEight, The Athletic, and The Independent. Data and politics and a little sports mixed in for good measure!

Every guest on Raw Data By P3 has been top-notch, but we haven't yet had a Doctor on...until now! Dr. Wayne Winston is Professor Emeritus of Decision Sciences at the Kelley School of Business at Indiana University, he's won numerous teaching awards, is a prolific writer, has consulted the NBA's New York Knicks and the Dallas Mavericks, and he's a 2 time Jeopardy winner! He's a highly entertaining and intelligent guest and we're honored to present this episode of Raw Data By P3 with Dr. Wayne Winston Click Here to check out Dr. Winston's book-Analytics Stories: Using Data to Make Good Things Happen Episode Timeline: 2:00 - Wayne's history and impressive pedigree 3:16 - Wayne makes a bold statement about a famous former student 4:41 - JRR Tolkien comes up in a random and interesting way 8:26 - The Jeff Sagarin connection 9:49 - Some great forecasting stories 13:33 - Random events and coincidence that lead you to where you are now 17:40 - The word is finally mentioned...Excel! And the subsequent Excel discussion 25:07 - Tenure process in a University, and how academics are rated 34:16 - Wayne's trips to Microsoft, and how he affected the Excel team 39:22 - Wayne's amazing new book-Analytics Stories: Using Data to Make Good Things Happen 45:24 - Does High School math need a revamping? 50:54 - Bitcoin-Is it legitimate? 55:10 - The Gender Gap in STEM Fields 1:06:04 - Excel VS Python 1:18:07 - Analytics In Sports 1:28:13 - The Election and Politics, and a prediction! Episode Transcript: Rob Collie (00:00:00): Hello friends. Boy, do we have a real treat for you today because we are welcoming the Wayne Winston to the show. Now Wayne has shaped and changed many, many, many lives. He's essentially the Excel Yoda figure at the Indiana University Kelley School of Business. His students even include like back in the day, Mark Cuban. The tagline for our podcast here is "Data with the human element", and I don't think anyone could really personify that more than Wayne. An amazing personality, a great guy, and there's no doubt that he's up to his eyeballs in data and data techniques all day, every day. He crunches data on really an amazing range of topics from sports to politics, to social justice. He's always bringing it back to the people involved and how it affects them. And the man's got a freaking Wikipedia page for crying out loud. So no more to delay, let's get after it Announcer (00:00:56): Ladies and gentlemen, may I have your attention please? Announcer (00:01:00): This is the Raw Data by P3 podcast with your host, Rob Collie, and your co-host, Thomas LaRock. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Raw Data by P3 is data with the human element. Rob Collie (00:01:18): Welcome to the show, Wayne Winston. I am over the moon, seriously, over the moon about having you on the show. Dr. Wayne Winston (00:01:27): Thanks Rob. Well, you guys have done wonders for the world with Power BI. Rob Collie (00:01:31): Well, I appreciate that and knowing what I was going to be up against today... what we were going to be up against today, I drank even more coffee than usual. I knew I needed more to do battle with the great Wayne. Dr. Wayne Winston (00:01:44): Oh, thanks. Rob Collie (00:01:45): So Wayne, let's just start off. I met you a long time ago when I was still at Microsoft. It's probably like- Dr. Wayne Winston (00:01:51): Early 2000s. Rob Collie (00:01:52): Yeah, probably like 2004, 2005. You wear a number of hats in your professional life. Why don't you just take a moment and give us a quick summary? Dr. Wayne Winston (00:02:01): Okay. So I went to MIT undergrad. I don't want to say what year. Studied math. Then I went to Yale, studied Operations Research Management Science, never took a business course in my life. And then I got the only job I've really ever had, teaching quant methods in the Indiana Kelley School of Business, which is ranked in the top 10 in the country now and it's a very good school. Had a great department. Wonderful people who sort of mentored me through life because I was pretty immature when I came here, and then I met my wife here. It's been great. And so basically I've gotten consulting jobs. Like I got asked to teach Microsoft Enhanced, which Rob is talking about through [Jennifer Skoog 00:02:39], and came out there probably about... for 10 years, probably once a month to teach Microsoft Enhanced. Went out there in 2018, but then I got luckily asked to write a book on Excel by Microsoft press Alex Blanton. I don't know if Rob knows him. Dr. Wayne Winston (00:02:55): Then I was asked to write a book for Microsoft Press on Excel, which has sold very well and I look on Amazon now and Rob's book is number four on Amazon in Excel books and I'm number six, but some days I beat him. Okay. And I have the Kindle version and I have my old Excel book, but that's really helped me a lot. I've gone to a lot of companies besides Microsoft to teach Excel, and then I was fortunate enough to have Mark Cuban in my class in 1981, who everybody knows from Shark Tank, and I'll predict will run for president in 2024. Dr. Wayne Winston (00:03:26): He's real interested in rank choice voting as an alternative to the two-party system and I think he had actually a podcast of his own on it yesterday. He almost ran this year. I think he just didn't have the time to get on the ballot. And so basically Mark was in my class. He bought the Mavericks. My family went down to see a Pacer Mavericks game, because my son is a big Indiana Pacers fan. He recognized me in the arena and he said, "Wayne, do you have any way to make the Mavericks better?". So my best friend from MIT, Jeff Sagarin, who's the USA Today computer rater, we collaborated on rating teams in lineups. And so we got into the sports analytics area and then I wrote a book, "Mathletics" on math and sports, and I won two games on Jeopardy!. Rob Collie (00:04:09): We didn't know that. Luke looked that up. He was researching you yesterday. Dr. Wayne Winston (00:04:13): I wanted to get on Jeopardy! from the time I was in seventh grade, and so if I had lost, I may not have come home. And the guy who was on last week said he used data mining to study for Jeopardy!. Now he won one game and then he got slaughtered, but I mean he said he developed word clouds like on King Solomon. If certain words appear in a Jeopardy! clue, he would know the answer is King Solomon. And so I thought that was very interesting, using text mining. Rob Collie (00:04:41): So often things get judged prematurely by their first outing. Dr. Wayne Winston (00:04:46): That's true. Rob Collie (00:04:46): A good idea doesn't always necessarily succeed its first time out. I'm always reminded... And this is really nerdy, but that's what we do. In The Silmarillion, the prequels to the Lord of the Rings, written by Tolkien, the first dragon gets introduced after it was born and it goes out into combat, but its skin isn't thick enough yet and it takes a bunch of arrows and it retreats into the mountain and it's forgotten for a long time. Like for centuries, but centuries later it comes back, and now it's ready, and now it's a world beater. There's so many things like that, I think, in life. Dr. Wayne Winston (00:05:20): That's a great analogy and I bet Tolkien planned that out. Rob Collie (00:05:25): Tolkien was something else. I mean, you understand the- Dr. Wayne Winston (00:05:28): Yeah, I haven't read those books. I need to. Rob Collie (00:05:29): Tolkien... We're already way off into the weeds, but I love it. Tolkien wrote the Lord of the Rings as a criticism of the industrial revolution. Dr. Wayne Winston (00:05:39): Oh, I didn't know that. Rob Collie (00:05:41): There's a whole subtext to the Lord of the Rings that I wasn't aware of as a kid where... Like he basically lived in the Shire growing up, pre-industrial revolution. He lived in an idyllic farming and wooded community, and then the industrial revolution came through and cut down all of the trees and burned them for fuel for the factories and things like that. And so this is what Sauron represents. The ring represents technology and the corrupting power of technology. When you watch the movies or read the books with that in mind, it just makes it... To me, it's just so much more emotional. Dr. Wayne Winston (00:06:18): Oh, that's interesting. You should get a PhD in literature. Rob Collie (00:06:20): Yeah, well. There are people who get Elvish tattoos. Elvish is a real language. Dr. Wayne Winston (00:06:26): Right. Rob Collie (00:06:26): Like they're fully developed bilinguist. Tolkien was sort of next to none in world building, but- Dr. Wayne Winston (00:06:33): Well, I'll have to watch the movies at least. I haven't even done that. Thomas LaRock (00:06:35): Yeah. So Wayne, first of all, I'm a huge fan. Dr. Wayne Winston (00:06:39): Oh, thanks. Thomas LaRock (00:06:39): And one of the highlights in my life is at the first business analytics conference in San Jose years ago, I got to meet you because I was fawning over you to Rob. I'm like, "Rob, Wayne Winston's here", and Rob looks at me and goes, "Oh, you want to meet him?". I'm like, "You know Wade?". Dr. Wayne Winston (00:06:56): Well, I appreciate like that. Thomas LaRock (00:06:58): Yeah. So we had a quick little chat, the three of us, and I'm a huge fan, but I just wanted to call out something. You said teaching at IU is the only job you've had, but I knew you because of the work you were doing for the Bucks and the Knicks. So you've had more than one job. Dr. Wayne Winston (00:07:13): Oh yeah, we did work for the Knicks too. Not the Bucks, but I guess I should be really proud we worked for the Knicks after the Mavericks. I forgot to mention that, because the only... Are any of you guys from New York? I don't know. But the Knicks won 54 games the year we worked for them. That's the most games they've won in 30 years because they followed our lineup advice. But I mean, I guess I thought more about the Maverick stuff, but yeah, we did a tiny bit of work for the Sonics, who were run by Wally Walker, who basically was a Goldman Sachs alum. And then since you guys have Seattle connections probably... Some of you. Rob does. When they sold the team and moved it to OKC, I was actually in Wally Walker's office when Howard Schultz came in to tell Wally they were selling the team to OKC. Rob Collie (00:07:58): Wow. Dr. Wayne Winston (00:07:58): And then knew they were going to go there and OKC supported, but the NBA owes Seattle a team. Thomas LaRock (00:08:04): Yeah. Rob Collie (00:08:04): Yeah. Dr. Wayne Winston (00:08:05): And Ballmer now owns the Clippers and he fires everybody left and right, but that's different. Rob Collie (00:08:10): Yeah. That is different. Dr. Wayne Winston (00:08:11): Yeah. Rob Collie (00:08:12): Yeah, well I plan to bring Ballmer up, but- Dr. Wayne Winston (00:08:14): Oh yeah, I'd be interested. Rob Collie (00:08:16): We'll save that for a little bit. One of the things we say from time to time on here is, "Let's make sure we slow cook this meal". A lot of things came out there in a short period of time. I want to make sure we... So first of all, I had no idea that you and Sagarin were close. Dr. Wayne Winston (00:08:30): Oh yeah. We met in the dorm at MIT, watching pro football. We watched the Jets win the Super Bowl and bonded over that. And he moved to Bloomington, to Indiana, in 1978 because he visited me and he saw there were lots of beautiful tall women and that convinced him he should move to Bloomington. Rob Collie (00:08:48): The analytics indicated he should move. Dr. Wayne Winston (00:08:49): I don't think it was analytics. Thomas LaRock (00:08:49): I don't think it was analytics. Dr. Wayne Winston (00:08:55): But he's never got married and I found a wonderful wife here, so I was lucky. Rob Collie (00:08:59): Oh, there you go. There you go. Because yeah I mean, I remember reading Sagarin in the newspaper when- Thomas LaRock (00:09:06): Same. USA Today. Dr. Wayne Winston (00:09:06): Yeah. He's still in USA Today. Rob Collie (00:09:08): Yeah. When newspaper was really a thing, back when that was like the sports content that I would digest. It wasn't on internet. There was already a Sagarin. That's so cool. Dr. Wayne Winston (00:09:17): Right. I mean, he's been doing it since the mid to late seventies, I would say. Thomas LaRock (00:09:21): But he started doing the college basketball rankings- Dr. Wayne Winston (00:09:25): Right. Thomas LaRock (00:09:25): Better than... That was one of the first things I remember hearing from Sagarin. I was just so enthralled going through the formulas of how he was determining that. Dr. Wayne Winston (00:09:34): He doesn't even let out what the formulas are, but there's a site, thepredictiontracker.com, lets you evaluate all the forecasts by all the computers- Thomas LaRock (00:09:43): Forecast, right. Dr. Wayne Winston (00:09:43): And he's invariably... Because that's one thing that came up a lot at Microsoft. Most companies don't know how to evaluate their forecast and this is a very... I'm sure Power BI helps with that. So I remember, if I can say one great story from Microsoft about that. I guess I shouldn't mention the person's name, but he was a controller at Microsoft and basically, so I was teaching about bias and forecasting. That you forecast too high or too low. So I would ask the finance people who Rob would know, "Do you think marketing forecast too high for sales?", and they would say yes. So I'd say, "How do you correct for that?", and the controller said, "We divide by 10". Which is hilarious, but I use that in class all the time when we talk about forecasting because I mean, forecasting is... Accuracy is important, but like Las Vegas, I did a study for my book that basically, when Las Vegas says a team will win by 10, they don't win by 10, but on average they win by 10. Rob Collie (00:10:40): Mm-hmm (affirmative). Dr. Wayne Winston (00:10:41): So that's unbiased, and so they couldn't stay in business if they weren't unbiased. Rob Collie (00:10:45): That's right. Dr. Wayne Winston (00:10:46): And so basically... But that's the best example of bias in forecasting when the controller... Off the air, I can tell Rob the person's name. Rob Collie (00:10:53): Okay. Dr. Wayne Winston (00:10:54): But basically they would divide by... he said, "Just divide by 10", and the room cracked up because finance's job is to be the bean counters and make sure that basically what you're putting out on the numbers are reasonable forecast. Unbiased and accurate. As accurate as you can be. That's a very important thing that I teach. Forecasting is a very important skill. Rob Collie (00:11:16): I've got a forecasting bias story for you. Dr. Wayne Winston (00:11:18): Okay, good. Rob Collie (00:11:19): Forecast is really important in our business because we move a lot faster. We burn through projects a lot faster than the traditional BI firm. We just move at a much faster tempo, which means we have a lot more moving pieces to our business than an average BI firm. Maybe even 10 times as many moving parts. And so we're not parking people for six months at a time on a single project because we finish projects fast. This actually makes it difficult to keep utilization high. It makes it difficult to know how much money is going to be going in or out two months in the future. And one of the inputs to this is that each consultant forecasts [crosstalk 00:11:57] how much of their work they're going to be doing over the next n-weeks. Rob Collie (00:12:01): Well, the subtitle of this podcast is "Data with the human element", and this is a perfect example of the blending of the two. Our forecast for a long time now actually, has been calculating the inherent bias in each consultant's forecast historically, and correcting for it. Dr. Wayne Winston (00:12:21): Oh exactly. Rob Collie (00:12:22): On a per consultant basis, right? Because some are optimistic. Some are pessimistic. Some are more dead on. And rather than fight that... Of course we feedback and we get better at forecasting and all of that, right? But at the same time, it's like belt and suspenders. Rather than deny the fact that humans are imperfect at this, we just correct for it on a per consultant basis. Dr. Wayne Winston (00:12:42): No, that's great. And you could build Montecarlo into that because the forecast... And if you want to know the chance your revenue in the next two months will be between X and Y, you could build Montecarlo based on the accuracy of these consultants. Rob Collie (00:12:56): Yeah, but that involves knowing the probability that they're right, right? Dr. Wayne Winston (00:13:00): Well, I bet if you can get the forecast error model that is normal, you could do it. Rob Collie (00:13:04): We need to talk about that. Dr. Wayne Winston (00:13:04): Yeah. We'll talk about that. Yeah. I'll give you my phone number when we get off and you can call me anytime. Rob Collie (00:13:09): Aw yeah. We're going to make beautiful music together. Dr. Wayne Winston (00:13:12): Oh, I'd love that. Especially since you're only an hour and a half away. Rob Collie (00:13:16): I know. I've only lived here now for five years and we haven't gotten together. Dr. Wayne Winston (00:13:20): Well, I thought you were in Cincinnati. I guess I lost track. Rob Collie (00:13:22): Oh yeah. I was in Cincinnati. That's why we haven't gotten together. No, I've been in Indiana. Dr. Wayne Winston (00:13:26): Well, I think you improved by coming to Indiana... Carmel. Rob Collie (00:13:30): There you go. So you mentioned a couple of things earlier as well, like math at MIT. That's a pretty tall hill, and then Operations Research at Yale, yeah? Dr. Wayne Winston (00:13:39): Right. Rob Collie (00:13:40): And then right after that, you said something to the effect of how grateful you were to have come to IU, because it was a place where you could grow up essentially. Dr. Wayne Winston (00:13:48): Yeah, it was just great. Rob Collie (00:13:49): I bet you didn't know this... Of course you wouldn't, but as I was finishing my undergraduate degree at Vanderbilt, I became obsessed with the idea of going to graduate school for operations research. That's what I was going to do. I don't think I knew what it was really. I still don't really know what operations research is or was, but it just seemed like it was right in my wheelhouse of things that I found stimulating, is what it really did. I just sort of got like an emotional feeling from it. I ended up skipping graduate school because I got to the front of the line to take the GRE as a walk-in standby. I got to the front of the line and my checkbook... I was out of checks and I took that as a sign. I was like, "Oh, and Florida's playing Auburn on TV right now. I'm going to just... This wasn't meant to be". So I just decided to go get a job at Microsoft instead. Dr. Wayne Winston (00:14:38): Well I think that probably turned out okay. Rob Collie (00:14:41): It did, but it involved a lot of what you said as well, which is like, I had no idea how much immaturity needed to be beaten out of me. Dr. Wayne Winston (00:14:50): Well, I was just a real introvert and I mean, my department was just a great bunch of people. Rob Collie (00:14:55): You don't strike me as terribly introverted, but I understand what that means. That word has a lot of- Dr. Wayne Winston (00:14:59): No, I... I could go into deep details about that, but I mean, I was... MIT, I was really introverted. Yale, I was really introverted, but then the head of my department was the greatest person I've ever met. [Vic Cabot 00:15:11], who you probably don't know, but do you know the Princess Diaries? Rob Collie (00:15:14): Mm-hmm (affirmative). Dr. Wayne Winston (00:15:15): His daughter wrote them, but he died tragically of cancer in 1994. But he was so funny. He came to the office, he was a little bit drunk sometimes. He would tell the secretary's dirty jokes. He'd come in painter's pants or shorts and he would teach his class in them. So we had this great... I really described our department as "summer camper adults". And then the first year Indiana won the national championship, it was undefeated in basketball, and nobody's been undefeated since. Dr. Wayne Winston (00:15:45): So I mean, it was just such a great place, and being unmarried and being a professor was really good to be at a school like Indiana. I lucked out. I guess my first choice was to get a job at Cornell, but they didn't pick me. I had a bad back at the time I got recruited, but I mean, I was so... In all your lives, there's karma. There's just random events that, like meeting your spouse or whoever is important to you, it's totally luck, or whatever you want to call it. Design. I think about it, if I hadn't gone to MIT, I wouldn't have met Jeff. If I hadn't gone to Indie... And I wanted to go to Princeton, but they didn't let me in because I'm from New Jersey. And so basically I wouldn't have met Jeff and wouldn't have got into sports. Dr. Wayne Winston (00:16:33): Hadn't gotten into Indiana, I wouldn't have met my wife. And honestly, no other department would I have had such a good time as with the people that were here and do well in my profession. A lot of things are not in our control. Rob Collie (00:16:47): That's right. Dr. Wayne Winston (00:16:47): Some things are. Guy named Paul Oster writes books about novels about the role of coincidence in life. In every novel it's about coincidence. I mean, the whole plot is based on, "If I was 10 minutes earlier to this place, my life would've been different". All sorts of crazy things like that, but back to the subject, I guess. But go ahead. Rob Collie (00:17:07): No, this is what we do on this show. So we [crosstalk 00:17:10]. Dr. Wayne Winston (00:17:10): No, this is great. Rob Collie (00:17:11): You're fitting right in. Dr. Wayne Winston (00:17:12): You're like the opposite... I do these webinars for Becker CPA review and if I mention anything, like I mentioned we should have a third party or rank choice voting, I get comments saying, "He shouldn't talk about that". Rob Collie (00:17:27): Mm-hmm (affirmative). Too human, too authentic. We need to drain that. We need to drain that out. Dr. Wayne Winston (00:17:31): No, I mean that's what the world needs more than ever. Rob Collie (00:17:34): And we're going to get to that, man. Thomas LaRock (00:17:35): I got a bunch of questions. Rob Collie (00:17:36): Let's finish laying the foundation here. So we haven't even really said the word yet. Excel. Dr. Wayne Winston (00:17:42): [crosstalk 00:17:42]. Right. Rob Collie (00:17:42): When people ask me who you are, one of the first things I say is that... I don't know if you agree with this, but I say like, "If there's a Yoda figure for the spreadsheet arts at IU, it is Wayne". Dr. Wayne Winston (00:17:54): Well at IU, that would be true. Rob Collie (00:17:56): Yeah. You get sent to Dagobah, you're going to be balancing things on your head and going into caves unarmed and all... Carrying Wayne around in a backpack, running through the swamp. That's what you're going to be doing. For the average human being... Not for the average human being listening to this podcast, but for the average human being, they would hear things like MIT math, graduate degree, and something arcane at Yale, and then they'd hear Excel and go, "what?". Dr. Wayne Winston (00:18:26): Oh, so okay. Yeah. That's really a good question. It's not really in the form of a question like Jeopardy!, but it's really a good question. So basically my background was totally math and operations research. For example, the simplex algorithm, which is the way you solve linear program problems. I'd say my first 15 years of teaching, we would have a manually pivot-in and pivot-out variables. Don't worry if your listeners don't know what that means, but then Lotus came out, I guess. I started using that in class the late eighties, but then in 1992, one of my students said, "You should use Excel in class". And so mainly what I would teach is simulation and optimization, and so then I started learning in Excel. I said, "Gosh". So I teach MBA, so I don't teach math people really. So I said, "I can make this so easy and fun for them using Excel". Dr. Wayne Winston (00:19:14): So then I would learn the optimization and simulation in Excel in 1992, and then as I got... all my teaching involved Excel, I saw I could do all the statistics in Excel, like the tables and the stat book. I always tell people the Z-tables, the normal random variable tables and the stat book, they're different in every book. It's just norm dist and norm inverse in Excel. The key to Excel is like... My wife watches the Food Channel. So Food Channel, you got to know how to mix up ingredients. So the functions are the ingredients in Excel and basically a student... Would always tell students, "Send me questions after you graduate". So one student sent me a question, I needed to learn offset. Another student sent me a question, I had to learn indirect, which I consider the two hardest functions in Excel. I don't know what you think are the most difficult functions in Excel for people to learn. Dr. Wayne Winston (00:20:03): I'd say offset and indirect, but I don't know if you have another one. But then I would just pick up all these functions and then Microsoft would add things like "sum fs", "count fs", and things like that. Like there's people... Okay, [Walkenback 00:20:18]. He's way better at Excel than I am, okay? And Bill Jelen is better at Excel than I am a little, I'd say. But I think my strength is, if you take my knowledge of math, my knowledge of business from reading books, and my knowledge of Excel, I'm sort of near the top in all three. Rob Collie (00:20:35): Yeah. Dr. Wayne Winston (00:20:36): Like using optimization in Excel. Since I came from that background in simulation, that's so easy for me because, I mean... But for other people who weren't trained with the graduate work that I had and the undergrad work that I had, they're not going to get the most out of Excel. So like optimization, I'll bet most of your listeners that they've used the Excel solver. Don't use the evolutionary solver or the GRG multi start. But honestly, you can solve 10 times more problems with GRG multi start and evolutionary than with the linear simplex solver, which is basically what everybody teaches in school. Even the top MBA programs hardly teach evolutionary, but evolutionary solver is... And I've met [Dan Folstro 00:21:22], who wrote the solver, on many occasions, and he's... I don't know if he's been on your podcast. Did you say he was? Rob Collie (00:21:27): Oh, he's queued up. I was going to say, he's coming up on the list very shortly. Dr. Wayne Winston (00:21:31): So what I would say about Dan is he's the best combination in the world of math, business, Excel, and computer programming. He knows the packages that are out there, like add-ins for Excel and things like that. And he went to MIT the same time I was there. I didn't know him, but he did. And he ran for treasurer on the libertarian [inaudible 00:21:52] in Nevada. Did you know that? Rob Collie (00:21:54): I did not. Dr. Wayne Winston (00:21:54): Well, you have to ask him. Rob Collie (00:21:55): It doesn't surprise me. Dr. Wayne Winston (00:21:56): And they moved from California to Nevada because the taxes were too high. Rob Collie (00:21:59): Wow. Dr. Wayne Winston (00:22:01): But Dan, I've got so much... because we would be in some meetings with the Excel people. Sam Savage, who's done some work on operations research, but he, Dan and I would spend a day or two there giving them our opinions on Excel and what they could do in the future. And Dan, he's like world-class, topnotch programmer, topnotch math. Really, he never took a business course. He' the right thing, but he taught himself through his clients, I think, "What does business need?", as you guys have to. You guys probably didn't take business courses, but you know business now because you work with businesses. Rob Collie (00:22:39): Our consulting team, they all came up working in business. Dr. Wayne Winston (00:22:43): Oh, I didn't know that. Rob Collie (00:22:43): Several of them have MBAs, but that's not a requirement at all. But our team comes from the business side of the house, not really from the IT side of the house. Dr. Wayne Winston (00:22:52): Now that actually surprises me. It makes sense. Rob Collie (00:22:53): I think, once you think about it, yeah, it starts to make sense, because like it's about business. It's not about the tech. We're really good at the tech. Dr. Wayne Winston (00:23:00): Oh yeah, for sure. Rob Collie (00:23:01): I promise the listeners, we do not brief our guests on themes we want them to hit. Dr. Wayne Winston (00:23:06): It's like Jeopardy!. They don't tell you the questions. Rob Collie (00:23:09): That's right. So a couple minutes ago you said something and I wanted to underline. Dr. Wayne Winston (00:23:12): Yeah, go ahead. Rob Collie (00:23:13): I think it's come up even on the podcast before. Something that I believe very, very, very strongly, which is, life isn't the javelin throw life. Isn't the pole vault. Life and career is the decathlon. Dr. Wayne Winston (00:23:26): Exactly. I could tell you were going for that. Yeah. Rob Collie (00:23:29): Yeah. And so you don't have to be the best shot putter or whatever. I don't even know what's in the decathlon. Dr. Wayne Winston (00:23:36): That's the best way of putting it 10 events. I don't know what they are. Rob Collie (00:23:39): Yeah. Me neither. I mean it's a really boring thing to watch, but it's a great metaphor. Dr. Wayne Winston (00:23:43): No, it really truly is. I'll use that. Rob Collie (00:23:46): So if you're 90th percentile at a handful of things, you are suddenly 99th percentile at the combined thing. Dr. Wayne Winston (00:23:54): That's exactly true. Rob Collie (00:23:56): And I've learned that about my own life, my own career. I spent a long time struggling with the fact that I wasn't the best at anything, and then one day I suddenly just like found my- Rob Collie (00:24:03): The fact that I wasn't the best at anything. And then one day I suddenly just like found my niche, which was all of these things together and oh, okay. That's that's it all just kind of made sense. Finally, it's like that, that coincidence thing you were talking about, which is another huge theme for us, like, the children's series of books called Lemony Snicket's A series of Unfortunate events. But, there's a play on that, which is Lemony Snicket's series of fortunate events. And it's the path that brought you here, it's almost unbelievable how much coincidence was involved. Dr. Wayne Winston (00:24:33): There's a new book on the evolution of life called a series of fortunate events. Rob Collie (00:24:37): Oh really? Dr. Wayne Winston (00:24:38): By Sean Carol, a biologist. Do you know like the most important thing that happened, that explains why humans are here, the dinosaurs getting extinct from the Meteorite because if they were here, they would've eaten us. Rob Collie (00:24:51): That's probably true. Dr. Wayne Winston (00:24:52): I never thought of that, but that's in that book. Rob Collie (00:24:55): The Jurassic part movies show us this very clearly, don't they? Dr. Wayne Winston (00:25:00): Yeah. The guy of the toilet. Rob Collie (00:25:03): Whatever, that's how the last human would've died. Right? Dr. Wayne Winston (00:25:07): So one comment on academia on what Rob said, which I think was totally brilliant. To get tenure in universities now you need an area that you can publish in and you almost have to be a specialist in something that nobody cares about basically. So like a generalist would have trouble getting tenure. I mean really, because there are some exceptions to the rule, some brilliant people who can publish in master many fields, but I can tell you, in most universities, people have one narrow area, they get their five or 10 articles on something maybe a couple hundred people care about, but that'll get them tenure. But, it's really hard. I got enough articles for tenure, I did fine with that. I'm not a role class publish one guy I went to school with at Yale. Dr. Wayne Winston (00:25:57): Paul Zipkin was the most cited business author because he had some really good inventory papers. But Inventory was his thing and basically also music. He liked to play music, but basically most people and another person I went to school with a Yale, her thing was queuing theory, waiting lines. I always have liked to read a lot. I'll devour books on now, I'm teaching this class on analytics. And so we're going to talk about the math of COVID after Thanksgiving, when we have to do Zoom. And then the last week we're going to talk about the mathematics or the empirical evidence on racial injustice. I read three or four books on that. Then I go to the original. Dr. Wayne Winston (00:26:37): If you work at a university, you can get access to any journal, I can find that source. I can [inaudible 00:26:43] know enough math. I can read the source articles for these books on racial injustice and go back and say, is this study a good study? Does it make sense? And so I gave them homework problems on COVID where they had to say, is this treatment better than the other treatment based on the numbers in the original study. So, it's knowing how to use the tools in your arsenal. If you have a good basic brain and you have a good basic set of knowledge in several areas, then it's up to you to go out and just learn the other stuff, which I think you three have done, because you have to, because you're working for companies that do everything. Rob Collie (00:27:19): That's right. So something Tom and I have talked about a lot is Imposter syndrome. There's something about the way you describe coming up that made me wonder, did you wrestle with Imposter syndrome sort of in the earliest phases of your career? Do you, are you familiar with, I was going to ask you. Okay. So imposter syndrome is something that tends to afflict, intelligent people with integrity, which is they think of themselves as a fraud. When they start to achieve like those first glimmers of success, they don't feel like it's deserved. They feel like maybe they aren't good enough. Does this sound familiar to you? Maybe, you're like the most interesting man in the world. And you're like, I don't understand what an uncomfortable moment would feel like Rob, I'm sorry. I can't help you. Dr. Wayne Winston (00:28:02): No, I mean, I think I never had that problem because I always had like a low opinion of myself. Rob Collie (00:28:07): You always had imposter syndrome is what you [crosstalk 00:28:11] Dr. Wayne Winston (00:28:11): I think it comes to my life from something different that I probably shouldn't talk about. I never had a high opinion of myself when I look back at what I accomplished in the business school compared to the other people there. I think my body of work probably had more influence. I would say now to a wider group than the people. Almost, not everybody. There's a couple than almost everybody at the business school. You don't get recognized for writing books that are impactful in the business school. They don't care. They really don't. They care a little more. Now what we have in academia is in every field, we have A journals and A minus journals and nothing below that matters. So if you're an economist, you want to get an econometric or journal political economy. If you're an Operations researcher, get an operations researcher management, science, and there's a point system. Dr. Wayne Winston (00:28:58): And literally, they can deny it all they want. But basically if you've got four articles in econometric and you're an economist, you're great. Even of those articles, if they got in, they must have some impact, but there's some articles in those journals that have world shaking impact like pricing options. In 1970, the black-scholes option pricing model was in journal political economy and they won the noble prize. It's a black died and you can't win when you die. So he should haunt Stockholm because he should have won because he was the smartest of the people. In Indiana, we had a guy who was a genius, Jack mu. He was really an introvert. He passed away unfortunately a couple years ago, but there's a thing in economics called rational expectations. And it had a huge impact on economics. You have to factor in how the people will sort of know what's going on. Dr. Wayne Winston (00:29:54): So he invented that and got into Econometrica, but he never went to conferences. He wasn't outgoing and he was very humble. But then 10 years later, Robert Lucas at the University of Chicago wins the noble prize for rational expectations. But Jack invented it. He should have shared in the prize. There are articles on this. And so, his feild was Operations management, which has nothing to do with economics really. And so it's strange how people get rated in academia. In business, if you guys weren't good, you wouldn't have food on the table. I mean, that's the way, you have to deliver. But I mean, there are articles. Dr. Wayne Winston (00:30:35): We have blind referee, but there are a lot of articles that, they're mathematical gymnastics, but do they do anything for the world? Like economists are terrible forecasters. If you look at the forecast of economists one year out, just predicting next year's GMP changes by last year's GMP is better than the best economic model. They have all these papers on how out of forecasts in economics, but the forecasts are crummy. They get their published, but are they good papers since none of this stuff works. Rob Collie (00:31:07): So many things to react to really quickly. First of all, you hear that folks? No one will admit it, but there's a point system behind the scenes for publishing. Dr. Wayne Winston (00:31:15): I Think get them drunk. They'll admit it. Rob Collie (00:31:17): Okay. But that tells me, somewhere there's a PivotTable. That's deciding whether people get tenured or not. Dr. Wayne Winston (00:31:23): It's the Citation list, which I've never been able to find. That rational expectations paper probably had thousands of sites. So when you're up for tenure, they'll look up the site. If a tree falls in a forest or noise, if your paper got in the best journal, like nobody ever cited it, then basically they'll think less of it. So there's these citation indexes that are important. Like my books get a lot more sites than my articles. I thought I've never figured exactly how to go to that citation index question. Thomas LaRock (00:31:52): It just sounds like it's, backlinks, it's just webpages and back links. It's like Wikipedia. Dr. Wayne Winston (00:31:57): It's like a search [inaudible 00:32:00]. Rob Collie (00:31:59): Like SEO. Dr. Wayne Winston (00:32:00): It probably is like that. I've never looked up the math behind it, but I think that's got to be what's behind the citation. It's got to be. And that's a solver model really. Thomas LaRock (00:32:10): I'm not sure that's the right way to do it. Dr. Wayne Winston (00:32:12): What would you change? Thomas LaRock (00:32:14): Well, just because something is popular, doesn't actually mean that it's good or accurate. Dr. Wayne Winston (00:32:19): No, that's true. I guess if the reviewing process is pretty strict, they'll find flaw. But that's the thing if you got in the best journal and somebody finds a flaw later, it probably won't hurt you in your tenure. There are some articles on racial justice like that, where they were talking about, okay, stop and frisk in New York. So the first couple articles would say that wasn't racially biased. And then somebody came out with a new article this year saying the previous article was wrong. But I bet the guy who wrote the previous article, who actually happens to be a black economist at Harvard, basically he'll still get credit for that article. You make a very good point, because you might have been cited because your article was wrong. I didn't think of that since I was never on a promotion and tenured committee. Dr. Wayne Winston (00:33:05): I avoided every administrative job as long as in my career they asked me to be MBA chair. And I said, look at my office, you don't want me running the program because I am a slob. That got me out that job. Because, I knew I was really good at teaching, although I wasn't at first, but Excel made me good at teaching because I have terrible handwriting. As soon as I saw what Excel could do, I would have a mind meld with that like Spock and I could just see, I developed courses on spread marketing, spreadsheet finance, spreadsheet supply chain and spreadsheet sports. I got so sick of seeing these books that didn't do the stuff in Excel and made them use the tables in the back of the book. I couldn't stand that. It was so stupid Thomas LaRock (00:33:49): Look, at some point there Wayne said I was right. So can we get that to a sound bite? Because, I'm putting that on my blog. Dr. Wayne Winston (00:33:56): Thanks. You're absolutely right. Because, those sites could be that your paper was junk. Rob Collie (00:34:01): He just did it twice. Dr. Wayne Winston (00:34:04): I really think you're right on that. That's really good. Rob Collie (00:34:07): Three times he said you're right. Dr. Wayne Winston (00:34:10): No, I had never thought of that, which I'm ashamed of. Rob Collie (00:34:12): There's still a couple pieces of background I want to fill in. We also talked about and where you and I first met was when you were out at Microsoft on one of your many ongoing trips, your visits to teach Excel to Microsoft. Dr. Wayne Winston (00:34:30): Old to new castle- Rob Collie (00:34:31): Which is awesome. You said something else as well, which is like, you know how you learned Excel over time? Like the students would send you another challenge and that would force you to learn indirect, or offset or something like that. You definitely need some education, but the way you just described it, that incremental climb is even more important. That is really the path to learning it. Dr. Wayne Winston (00:34:52): Exactly. To learn anything- Rob Collie (00:34:53): That incremental journey is crucial to so many things. It's shocking to me, like how much of a culture of techniques can get developed at a particular organization. I would spend a lot of time on the Excel team, when I was working on Excel. We would need to do customer visits to see how people were using it and try to research what we could do to improve it. And all of that. Rob Collie (00:35:15): Well the most convenient customers were our own finance department right at Microsoft. So, we would visit our own finance department to learn how to improve Excel while you were on the other end, teaching them Excel. And so there's sort of like this loop that was forming there. And then you come talk to us separately as a team, in the evenings, after you were done to teaching our finance team, just give an example of how crazy this gets. There was a way that Microsoft finance would use the get pivot data function. Dr. Wayne Winston (00:35:45): Oh yeah, which is a great function. Rob Collie (00:35:48): A great function. You don't really need it anymore because [crosstalk 00:35:50] of what you did. At the time, anyway, it was awesome. But they had this trick that they almost like hacked GETPIVOTDATA. They would change some of the hard coded references that get Pivot data would give you on click to sell references to relative. Dr. Wayne Winston (00:36:06): That's what I do. Yeah. I do that when talking. Rob Collie (00:36:07): Okay, so you taught them this. All right. Dr. Wayne Winston (00:36:08): I'm not going to say that. I taught them that. Rob Collie (00:36:13): I bet you did. [crosstalk 00:36:14] Dr. Wayne Winston (00:36:14): Wow. That makes me really happy. Rob Collie (00:36:16): Okay. So this influenced a lot of our thinking in terms of what we did with those features going forward, because we're like, oh my God, we cannot break that. Because, people are using that. We had no idea. And I learned later that it was only Microsoft finance doing this. When I talked to bill Jelen about this technique is if it was just common knowledge, right? Bill's like what? Thomas LaRock (00:36:38): He put it in his book. I know. Rob Collie (00:36:39): And then he went and looked at, it's like, oh my God, that's amazing. The thing is, bill teaches. I think he's retired from this now. But he teaches seminar. He tons of seminars all over the country, accountants. And he had never once run into this. Not once. Dr. Wayne Winston (00:36:52): That surprised me, that he didn't, put in his newer books and we borrow from each other's books. So I don't know. Rob Collie (00:36:58): Of course. Dr. Wayne Winston (00:36:59): I get things from his books and like indirect with range names. I put that in my book and people took that. You make up your own examples. That's a really great example because, if you're not using Power Pivot, I'd say almost nobody who's in business knows that get pivot data, you can take the data out of a PivotTable, make it look at the way you want for reports and charts. And it's easy. You're right, you put one formula and a cell and then you put in row and dollar sign the column and just copy it across. Rob Collie (00:37:35): That's right. Bill's jaw dropped. Dr. Wayne Winston (00:37:38): I'm glad to know that. I met him at a conference. He's a really great guy. Rob Collie (00:37:43): Well, we're going to have him on too. There needs to be some crossover episodes where we get some of these people together. Dr. Wayne Winston (00:37:48): Yeah. I'm glad to Be on any time. It's my schedule. Rob Collie (00:37:51): In some sense, we're almost assembling our cast of characters in the early guy, like Lord of the rings. Yeah. Yeah. It's like, and now we're going to introduce yeah. It's like when they discover a particular pod of whales, like killer whales, in a particular corner of the world, that's learned some new behavior and they're teaching their kids how to run up through the shallows and grab things off of land, you know? And no one else in the world does this. There's some habits that develop in particular organizations around Excel. I just thought it doesn't surprise me, that was something that you were teaching, but I wanted to connect those dots for you. Because, I've been haunted by that for a long time. Dr. Wayne Winston (00:38:27): I'm not sure if it was me who did that. Actually, I came up with the idea of using that. Of course, I would teach Cisco finance because Jennifer and Microsoft would recommend me to Cisco. And basically somebody said, I get this number out of a PivotTable, but then I add a new country, like I rack to the PivotTable and then I don't get the right number. Then I learn, GETPIVOTDATA. And basically I said, well you need to copy, GETPIVOTDATA around. Across and down. Thomas LaRock (00:38:56): Well, we solved that problem in 2003 when we removed a rock from the PivotTable. Dr. Wayne Winston (00:38:59): Okay. No, that's probably when the [inaudible 00:39:06]was. Rob Collie (00:39:06): Yeah, it was, it was, it was 2003. Yeah. Okay. Dr. Wayne Winston (00:39:08): I didn't even know that. I remember having debates about the middle east with Charlie Ellis who you probably knew. Rob Collie (00:39:14): Oh yeah. I've been on the phone with Charlie within the last month actually. Dr. Wayne Winston (00:39:18): Tell him, I say hello. He went on his own too. Rob Collie (00:39:22): He did. Yeah, so you've written a number of books, Wayne and you've got a new one out right now and I did the good podcast host thing and I gave it a good skim this morning on Kindle. Dr. Wayne Winston (00:39:36): I appreciate that because I gave you no notice. Rob Collie (00:39:38): It's a very substantial book. Oh my gosh. Dr. Wayne Winston (00:39:41): It was hard. Rob Collie (00:39:42): 670 pages on Kindle. I don't know how many pages I ordered the print copy. Dr. Wayne Winston (00:39:46): 500. Regular. Rob Collie (00:39:48): I love the tone of it. Dr. Wayne Winston (00:39:50): Thanks. I tried. Rob Collie (00:39:52): There are so many books on topics like these that are inscrutably, academic and arcane. I can't engage with those. And then on the other end of the spectrum, there's the Freakonomics type books that don't really get into too much of the technique. They almost read like a novel. They're very entertaining and intellectual. I kind of get this sense that this new book of yours is like between those two. Dr. Wayne Winston (00:40:13): You hit a nail on the head of whatever the phrase is. That's exactly right. They're the books that are like Chinese food. You're hungry an hour later because it doesn't tell you how to do anything. And then they're the books for the data scientists, which you need your PhD. I'm using this book this afternoon and in the class I teach this afternoon. I have freshmen through seniors at IU, it's Honor students so they're smart. But they major in government, they major in biology, they major in psychology. My one freshman, who's doing great, she majors in dance but she got a nineties five on the test. So, she's doing great. Rob Collie (00:40:50): She's going to run an analytics and BI company some day. Dr. Wayne Winston (00:40:53): Yeah, you have to have multiple perspectives and we're going to simulate the election in class today and we're going to do and talk about ranked-choice voting and stuff like that. But that's exactly I tried to come up with interesting topics. Like how should they have known Bernie Madoff was a fraud. There are three ways they should have known that. How did they figure out what caused cholera in England in 1854. That was probably the greatest doctor of all time, was the guy who did that. What he did was amazing. And the randomized trials we hear about with COVID I finished this book before COVID so it's not in there. I've finished it in March. I've written a chapter on COVID, that would be an addendum. Rob Collie (00:41:34): Let's give it its due. What's the title of the new book? Dr. Wayne Winston (00:41:36): Analytics Stories. So, the cover is basically turning lead into gold or frog into a prince using data, which would fit you guys. My daughter works for a social media company and she's in advertising. And so the first cover was really a boring cover that had all these photographs of things in the book. She said, you need a fairy tale concept. So here we go. Rob Collie (00:41:57): I wrote a book called PowerPivot Alchemy sort of under the same guys. That's a good title. Well, it didn't sell very well. I'm sorry to say- Dr. Wayne Winston (00:42:05): But your other book has sold. I don't even know how many copies and it deserves to because I think you and I have, you guys have the same philosophy as I do. So you probably are too young to member show Hogan's heroes. Rob Collie (00:42:17): We got it in syndication. Tom's old enough. Dr. Wayne Winston (00:42:19): I know nothing the prison guard used to say. So my philosophy in teaching is first of all, teach by example. And so that's the incremental approach. And then as soon that the people in my class are smart, but they don't know anything about the subject. And I think you did that in the PowerPivot book. You didn't have to know anything to go into that book and follow through your good examples. You're better than I am at. Like you have more patients with doing good screen captures than I do. Because that's a key to the book because that's what takes so much time in a book to do the figures. Rob Collie (00:42:54): Yeah. It's a lot of work. Thomas LaRock (00:42:54): I love the book. I love just how it read. Like you said, you're a professor that wrote this book and telling these stories and the way it gets broken down, like you say, you teach by example. I love [crosstalk 00:43:03] thanks. You start right off. It's like, Hey, what happened? What will happen, why did it happen? And the best part to me was how do we make something good happen? Right. And that really kicked me in the fields. Dr. Wayne Winston (00:43:14): That's Optimization. Thomas LaRock (00:43:16): I want to ask about the chapter on American workers. Dr. Wayne Winston (00:43:19): Yeah, sure. Thomas LaRock (00:43:20): Because there are two things that stood out to me, the first and I don't want to be critical at all, but I want to point out the fact that you said that Reagan beat Carter in a 2000 election and I know you meant 1980. Dr. Wayne Winston (00:43:31): No, that was a typo. Thomas LaRock (00:43:32): I just wanted to say, if you need help with technical reviews, I'm available. I charge reasonable rights. I'd love to help you. Dr. Wayne Winston (00:43:38): Okay. If I write another book, but I'm burned out at this point. No, that's the worst typo in the book I hope. Thomas LaRock (00:43:44): The chapter titles is the lot of the American worker improving. Dr. Wayne Winston (00:43:49): Right. Thomas LaRock (00:43:50): And you gave a lot of evidence, but I'm not sure you actually answered the question. Is it improving or is it just something that we're supposed to figure out if it's improving for us as individuals? Dr. Wayne Winston (00:43:59): Well, by looking at the median, I said it, the median is my measure of income. And it's not improving, but I do think my weakness as a writer is writing conclusions. Thomas LaRock (00:44:11): Okay. Dr. Wayne Winston (00:44:11): And I think a lot of these chapters probably needed conclusions. I was hoping in most chapters to let people make up their own mind. What I didn't do with that chapter and I regret it deeply now is look up the data by race, you know? Thomas LaRock (00:44:25): Oh, okay. Dr. Wayne Winston (00:44:26): I've seen that since the emphasis on racial justice recently, I've seen data on that. If we looked at the white non-college grad and we looked at the Black American and the Hispanic American and the Asian American, the Asians have done great. One thing I don't think I put in the book, they're intergenerational mobility by race. The Asians, basically by far the most intergenerational mobility, like what percentile does the child go in the income relative to the parent and for the Black, there's the worst intergenerational mobility. Dr. Wayne Winston (00:44:59): But I just found that like a month ago I used it in a Homer problem on that chapter. Each of those chapters could have been a book, I guess it is my point. There are mostly stuff that hasn't been out there and they're pretty good. But there are books on the American worker. They're like the freaking up, you read 200 pages to get five pages. And so what I wanted to do is this, if you read 10 pages of this, you're probably getting 15 pages of stuff. My dream is this book would be taught in a high school like [inaudible 00:45:27] high school, because there's been a big debate about how high school match should go. And the Freakonomics author has been in the middle of this saying, we need to teach like analytics in high school. So people can be better voters and better decision makers. Dr. Wayne Winston (00:45:42): You have AP statistics. A great thing about Excel, which we hadn't talked about is basically, it's got like zero cost to enter into Excel, to teach somebody something. You just start typing number. These people in my class, they didn't know much Excel coming in, most of them except the business school students. I needed to teach them how to draw a scatter plot. So, we did trendline. I mean it took 30 seconds. We needed to teach them tons of PivotTables. Because like half the chapters, we had to create a PivotTable to show something. I just show them how to do PivotTables. They're smart. And I had to teach them how to do VLOOKUP. I know you don't need it with Power Pivot, but I had to show them how, if you have two lists scrambled, you can take the stuff from the second list and put it in the first list. Dr. Wayne Winston (00:46:31): The thing is, there's no learning. If you teach Python, which is definitely the coming thing. There's a big learning curve. You can go into Excel, teach people who know nothing how to do a lot of stuff. And that to me is really you teach them first day of class we did count FS, sumifs and basically they never knew it, but they had no trouble internalizing what it does. What would you say are the most important functions in Excel besides maybe VLOOKUP? I'd say those functions count FS, sumifs, average FS. I think those functions, they are probably more important than VLOOKUP, I'm not sure. If you had to rank Excel functions in importance, that's where I go. You could have a talk with Microsoft about that. Rob Collie (00:47:17): But you could get all kinds of combative about this. You know, I just pick the IF function and stand on it. Dr. Wayne Winston (00:47:22): IF is there too. I should have put IF there. Rob Collie (00:47:25): Maybe that's a whole podcast where we fight about ranking the functions. Dr. Wayne Winston (00:47:30): You should have Bill Jelen on. Rob Collie (00:47:31): It's like the desert island functions. Like you can only take 10 functions with you, that's only 10 you get. Dr. Wayne Winston (00:47:35): That's right. There's a song top 10 breakups. Rob Collie (00:47:39): That's right. Dr. Wayne Winston (00:47:40): Try fidelity. Rob Collie (00:47:41): Let's talk about high school math. You mentioned that. This is something that I've been reflecting on a lot. Dr. Wayne Winston (00:47:46): And your son's in senior, in high school. Rob Collie (00:47:47): Yeah, that's right. Yeah. I was like an apex predator of the high school Math scene. I was on the Math team. I took all of those classes. We went to the final four in the Florida state calculus tournament. Dr. Wayne Winston (00:48:02): You're from Florida. Where are you other guys from? Rob Collie (00:48:03): In the Florida state calculus tournament. Dr. Wayne Winston (00:48:03): Oh, You're from Florida, where are your other guys from? Rob Collie (00:48:04): Our crew here, we're from all over. Luke and I met in Florida, but let's zoom in on high school math in particular. Dr. Wayne Winston (00:48:10): Yeah, sure. Rob Collie (00:48:10): There were so many kids in those classes with me who had been forced into them by their parents, or were supposed to take them, you take the next honors class, whatever. There was always this refrain of when are we ever going to have to know this? When are we ever going to use this in real life? And at the time I would just viciously crucify them when they said that. I was such a not nice person. I didn't know it. I would say to them, well, you're only saying that because you're not any good at it, but they were right. I'm here to tell you that they were right, because I basically do math for a living. Dr. Wayne Winston (00:48:44): Right. Rob Collie (00:48:45): It's a reasonable thing to say. And I don't use any of that. I don't use Algebra II. I don't use Trigonometry. I certainly don't use Calculus. Dr. Wayne Winston (00:48:54): Great points. Rob Collie (00:48:55): There's something about it, looking back. Are we still just trying to build the engineer's of World War II? Dr. Wayne Winston (00:49:03): Well, okay. That's a really great question because in the business school at Kelly, and I think most business schools, we require calculus and finite math; which your kid probably took finite math in high school. I would think maybe. Rob Collie (00:49:15): But I like finite math. I would still probably teach something to do with that. Dr. Wayne Winston (00:49:19): Yeah, I think that's good. But they don't use any calculus in the business school, except if you're an investment major in finance. I'll give you a question and see if you can answer it. Why is calculus required in the business school if we don't use it? I can answer this for you, but take a guess. Rob Collie (00:49:36): Is it just a weed out? Dr. Wayne Winston (00:49:37): No. It's because the math department would be financially bankrupt if we didn't require it. Rob Collie (00:49:44): Even better. Thomas LaRock (00:49:45): Oh wow. Dr. Wayne Winston (00:49:45): That's the truth because IU runs on budgeting based on credit hours. So if the math department didn't get their credit hours from calculus, they'd be out of business. So the president would never let us take it away as a prerequisite. Though we're looking for a new president. Rob Collie (00:50:00): This explains why I had to take differential equations as part it of computer science. I wasn't going to be an electrical engineer. Dr. Wayne Winston (00:50:06): No differential equations, I think, helps you think about the world. But I think, as you put it, I'll put it more delicately than engineers of World War II. It's a stem curriculum. So basically the high school math curriculum is to prepare you to go to Purdue and be an engineer. Okay. There's nothing wrong with that. What percentage of the kids, even at a great school, like Carmel, which is about the best high school in the state, are going to be engineers? Not that high. You can find Levitt's podcast somewhere where he says this stem curriculum, it shouldn't be for everybody. You're right. You're incredibly successful doing math for a living. Calculus has no importance to you and trigonometry has less, unless you're building your own house. Rob Collie (00:50:55): Let's fight about Bitcoin. Dr. Wayne Winston (00:50:57): Okay. Thomas LaRock (00:50:57): Okay. Rob Collie (00:50:57): I don't own any. Dr. Wayne Winston (00:50:58): I don't either. I wish I had in the beginning. Rob Collie (00:51:01): I think that Bitcoin and cryptocurrency in general, it has a much greater shot at being legitimate than a lot of people give it credit for. And for one very, very, very important reason, which is you can't fence it in. Dr. Wayne Winston (00:51:15): That was the whole point of it. I think. Rob Collie (00:51:16): Yeah, it is immune. Not completely, but it's close to immune to capital controls. It is so easy to flee a country with your wealth in Bitcoin form. Dr. Wayne Winston (00:51:27): That's a good point. And I think the Democrats if they win will put a wealth tax. That's an interesting point. Yeah, but I don't know enough about Bitcoin to comment. Rob Collie (00:51:37): You've heard about the people who sink their money into gold and then claim that it was lost in a boating accident, right? It's taking wealth just completely off the table; taking it completely off the radar. However, try to go across a border with $5 million in gold coins, right. And see what's going to happen to you. Dr. Wayne Winston (00:51:55): That's a very good point. I think you're right on that. Thomas LaRock (00:51:58): But Rob, in the future, you're going to try to go across the border and they're going to know how much Bitcoin you have. Rob Collie (00:52:03): How would they know? Dr. Wayne Winston (00:52:05): I don't think they can. I really don't. Thomas LaRock (00:52:06): But they can always change the laws. If they find that's an issue; people are fleeing. You know, that they'll change the rules. Dr. Wayne Winston (00:52:12): That's a good point. Rob Collie (00:52:13): At the same time. I just don't know how you enforce any such law. For so long Swiss bank accounts were a safe haven until the U.S. finally leaned on them and broke them of that. But for a very, very, very long time, Swiss accounts were used for all kinds of things that were illegal, but unenforceable. I think what we found in the cryptocurrencies is something that is orders of magnitude harder to enforce. Dr. Wayne Winston (00:52:45): I'm going to tell you both that there's a podcast you should listen to called The Missing Crypto Queen. Thomas LaRock (00:52:49): Okay. Dr. Wayne Winston (00:52:51): You should subscribe to that. It's out of the BBC, it's an eight or nine episode series at this point and you can binge it fairly easily. And it's a nice tale giving you an idea of cryptocurrencies in general. I don't want to pick on just Bitcoin, but there's a lot of coins out there and they're not all legit. Rob Collie (00:53:10): No, that would be good. Thomas LaRock (00:53:10): Right, I'm sure there are. Dr. Wayne Winston (00:53:11): It's a brilliantly woven story. It plays like an old radio serial. Basically this guy or a team of investigators goes after this missing crypto queen. Rob Collie (00:53:20): So is it set to, I don't know how to pronounce it. Is it Abba? ABBA? Is it set to ABBA's Dancing Queen? Dr. Wayne Winston (00:53:26): It is not. There's nothing musical about it. No, but it's a brilliant production. The quality is amazing. Rob Collie (00:53:34): I'll have the lyrics to crypto queen later today. Dr. Wayne Winston (00:53:39): I think you would both appreciate it. Thomas LaRock (00:53:40): No, I'll listen to it. Rob Collie (00:53:41): I hear what you're saying, but there's also like, how does that wealth actually... So when Wayne says hey, maybe it's worth 20,000. At some point, it's like any idea how it might have actually gotten to that 20,000? They really peel the layers of the onion about the entire industry. Dr. Wayne Winston (00:53:58): Oh, that will be great. I will listen to that. Princeton university press has a good book on the math of blockchain and Bitcoin. I have got the book, but I have not read it. I just haven't had time to read it because it's not easy. Rob Collie (00:54:09): And they all have, I think, vastly different processing power requirements in order to conduct a transaction. Dr. Wayne Winston (00:54:15): Yeah, they burn a lot of energy. Rob Collie (00:54:16): That is just a crazy, crazy, crazy downside. You conduct a $9 transaction and you burn more power than your house does. It's insane. We don't need that, that's something we don't need. Dr. Wayne Winston (00:54:30): That's interesting. I'll have to look into that. Rob Collie (00:54:32): Bouncing around because that's what we like to do. Dr. Wayne Winston (00:54:37): Yo, that's great. I do that all the time in class. I talk about TV after I talk about offset function. Rob Collie (00:54:41): And I noticed that in your book, you're quoting Billy Joel lyrics and things like that. Dr. Wayne Winston (00:54:46): Right, which I thought was perfect there and offset by the way is Cardi B's ex-husband. Just to mention that. Thomas LaRock (00:54:55): I love the references. The pop culture references and things that you sprinkle. It's just; it was great. Dr. Wayne Winston (00:55:01): My students love that. I'd be talking about the off function and then I'd say, did anybody watch Gossip Girl Monday night? And they just crack up because nobody else does that. Rob Collie (00:55:11): So just back briefly to the high school math thing again. Dr. Wayne Winston (00:55:13): Yeah, sure. Rob Collie (00:55:14): There's another thing that's been haunting me about all of this; haunting me in a good way. So maybe haunting is the wrong word. It's really a feel good thing for me. Which is that I don't see much correlation at all between the people who are good at Power BI and their interest in math in high school. Dr. Wayne Winston (00:55:32): I'd agree with that. Rob Collie (00:55:33): It's not a negative correlation either. It's just no correlation. I said we're building World War II's engineers and you said, it's just a stem curriculum. But to me stem doesn't carry, maybe it should, a negative connotation. It doesn't really have any explanatory power as to what we're doing wrong with it. We tend to talk about stem as a good thing. Rob Collie (00:55:58): It's sacrosanct in a way. Then we also talk about, for instance, and this is a topic that I'm only halfway qualified to speak about, but that's good enough. Tom used to be involved and still is involved with a lot of conferences. Maybe we'll get back to that conference game after COVID runs its course. But at every conference that I've ever been to, especially the ones that I've been to with Tom, there's always been panel conversations about how to encourage more women into stem fields. And that's a recognition of the fact that it is a lopsided male, female ratio in a lot of stem fields. However, in our business, like when we're teaching classes, that's not the only thing that we do. Most of our work is actually consulting work, but we do teach a lot of classes. Dr. Wayne Winston (00:56:42): Right? Rob Collie (00:56:43): The people in our classes, it's like 55% female. Dr. Wayne Winston (00:56:48): Oh good. Rob Collie (00:56:48): The students, right? So there's a quantity advantage. And the quality is also what you would expect. It's 55% of the time, the best student in the class is also female. This is a real disconnect from what you would consider other stem fields. And at the same time, isn't what we do kind of a stem field? So whatever the forces are that lead to the average stem field being predominantly male, those forces are absent in my line of work. I don't know what the actual population composition is, but it's actually, if anything, it's slightly more tilted. It's just slightly tilted towards female in our line of work. And I just have found this fascinating for so long. Dr. Wayne Winston (00:57:29): I got a theory on that when you're done. Rob Collie (00:57:30): Okay. Let's hear it. Dr. Wayne Winston (00:57:31): Okay. So my theory is, women are more patient than men. If you're going to be good at Excel or good at Power BI, you got to learn how to Google or Bing something when you don't know how to do it. Almost nobody has the patience to do that. We do because that's what we are, but most people they're just not going to do that. I bet women when they're stuck on something are much more likely to figure out how to find the answer to it than men are. Maybe men are too proud or too lazy or whatever. But I think that may be why it's not going to help you in stem to Google stuff really. Rob Collie (00:58:15): No. Dr. Wayne Winston (00:58:15): Somebody, she now works for Microsoft and she had a problem, and I think I gave her slightly the wrong answer last week, because I didn't quite understand the problem. But then she found on Google or Bing; probably Bing because she works for Microsoft. She found it somewhere. She knew how to search and find somebody online who had solved the similar problem and she could figure it out from there. So this incremental learning, a lot of it is you have to be willing to look something up. For instance, Bill Gelin didn't even know that power get pivot data, but he knows tons of things I don't know. I can almost always find online something I don't know but then I know it. I think women are just more likely to cycle through that pathway to knowledge than men. But that's my only answer to you. Rob Collie (00:59:06): Well as a funny story, this isn't going to be confirmation of any sort, but in college I took two psychology courses because the computer science department required me to do two classes in something. So I picked psychology and of course the real purpose of Psychology 101 classes is to provide the psychology department with subjects for all of their various experiments. Dr. Wayne Winston (00:59:28): Oh, right, right, right. Rob Collie (00:59:29): So in order to get extra credit, you've got to sign up and be a guinea pig in various things. I got into this one where they provide a battery of interview questions like Scantron, multiple choice tests. What it really came down to was the theory they were testing was "were androgynous personalities more likely to be emotionally reactive than unipolar male or female traditional personalities"? I ended up watching these movie clips. Everything from boring stuff, horror films, sad things that happened in movies, all with these electrodes wired all over my body, on my hands to see if I was sweating in various moments. At the end of it, they explained to me that your interview questions, Rob, puts you very firmly in the androgynous category. And so that's why we promoted you to round two of watching the movies and everything. So there you go. It's the female part of me that draws me to excel. Dr. Wayne Winston (01:00:33): That's why you're good at this. No. In the business school, we don't have as many women as we have men. The women will major more in marketing and management. That's sort of what you're saying. And then in the math department we've got less women. Dr. Wayne Winston (01:00:50): In my honors class, it was a math and sports class. Last year I had a freshman, she took it, she was pre-med and she really knew nothing about sports. She didn't even know what baseball was. Okay. But she was the best student in the class. She would tear apart everything I did and try and understand every little detail and say could I have done it this way? She was valedictorian at her high school in Southern Indiana. So she's just really smart. It could be the approach we used to teach high school math as an approach that women don't like as much or they're taught they they shouldn't like it. I think if we did some research online, we could figure out why there's not that many women... Your data point is really interesting that there are more women who are basically high level analysts, I guess would be the title, (Senior Data Analyst) then there are men. Rob Collie (01:01:52): Yeah, that's right. Yeah. Dr. Wayne Winston (01:01:52): And they probably started out pretty low level, but they just.... It would be great if you talked to them after class and asked them, how did you get good at this? Just give that as a question. And you're to all the people, the men and the women. Rob Collie (01:02:08): Well, it's always the same story, no matter who they are. Dr. Wayne Winston (01:02:11): Oh really? What is it? Rob Collie (01:02:12): It's always like essentially like some sort of battlefield promotion. It's like that whole nature abhors, a vacuum thing. If you're in a particular department or a team or an organization, let's just call it a team for a moment. Every team in every business in the world has an Excel guru on it. Dr. Wayne Winston (01:02:30): Okay. Well that black belt or something. Yeah. Rob Collie (01:02:32): They're not trained in that. Sometimes it is, but usually it's not something that has anything formalized about it. No formal training. And the reason is, it's just out of necessity. If they don't have someone who is good at Excel, they're not going to be able to get anything done; that whole team. Dr. Wayne Winston (01:02:48): That's that's exactly right. Rob Collie (01:02:50): They kind of like organically try out. Everyone gets tried out as to be good at Excel. The one person out of 15, and that's about what the ratio is. The one person out of 15 who shows some aptitude for it, that just becomes their job. That's what happens over and over and over again. The Vlookup and pivot crowd, which is its own subculture of the Excel landscape. A lot of the things that you do, Wayne, about simulations and... Dr. Wayne Winston (01:03:16): Yeah, that's minor. Rob Collie (01:03:17): It's not minor, but the Vlookup and pivot crowd is basically the world's BI. Dr. Wayne Winston (01:03:22): That's the survival. Yeah. Rob Collie (01:03:23): Yeah. All the other BI in the world rounds to zero still relative to the Vlookup and pivot crowd. Dr. Wayne Winston (01:03:29): No, I agree. Rob Collie (01:03:30): Another anecdote that I think is interesting on this front. And I know we're just a bunch of dudes sitting around talking about that there are more women in something. But even though what I've told you is true, the overwhelming majority of our job applicants are still male. Dr. Wayne Winston (01:03:43): That's surprising. Rob Collie (01:03:45): It is something that we're definitely working on. We have several women working for us now as consultants, but as a percentage edge of our applicant pool, we're not seeing that 55/45 that we would want to see. Back at Microsoft, when I was building the very, very first version of what I called the great football project, where we were building this amazing stats portal that lets you ask all kinds of questions. For instance, these quarterbacks are good, but who's good in the third quarter? Who's good at completing third and long? Who's good at avoiding third and long? The ability to answer basically any question you could formulate. I thought this was going to be hot. Dr. Wayne Winston (01:04:27): It is now. Rob Collie (01:04:27): Well, I'm not so sure. It would be hotter now than back then for sure. Dr. Wayne Winston (01:04:29): This was gambling. Rob Collie (01:04:31): Oh. Yes, yes. Dr. Wayne Winston (01:04:32): In Indiana you can gamble. Rob Collie (01:04:33): Really? Dr. Wayne Winston (01:04:34): You didn't know that? Rob Collie (01:04:35): I didn't know that. I did not. I thought I had to go to French Lick. Dr. Wayne Winston (01:04:37): No, you can put it on your phone in Massachusetts. I'm not sure. Rob Collie (01:04:42): I thought we were done talking about Bitcoin. Are we back to that? Dr. Wayne Winston (01:04:46): No, no. I'm talking about like Draft Kings or Fan Dual. Rob Collie (01:04:49): Yeah. Yeah. Dr. Wayne Winston (01:04:50): I mean that's where there's a site with pro football reference where you can query the stuff that you talked about. Rob Collie (01:04:58): Yep. So we had this portal, we were focus grouping it. So we were bringing in people who were known to Microsoft, they were friendly to Microsoft, somehow. They were part of extended social circles of people that worked with us who were also into sports. And I learned several things in this process; one that's relevant here was that the men were not interested. They did not care. Dr. Wayne Winston (01:05:22): Really? Rob Collie (01:05:22): They were not interested in what I was showing them at all. It was like they already knew what they needed to know. They thought they did. But time after time in these focus groups, the women in the group were leaning forward going, oh, but what about this, and what about that? Dr. Wayne Winston (01:05:37): In sports? Rob Collie (01:05:38): Yeah, the curiosity. Dr. Wayne Winston (01:05:40): That's the word I was looking for. Yeah. I mean, women are more curious to learn. Men are just practical and they just want to get it done and then go drink or play sports. I mean women have more of a soul, I guess. Rob Collie (01:05:56): I agree. Some of the things that we try to do are encouraging people to be more like men and I'm not necessarily sure if that's what we should be doing. But you mentioned Python earlier and you said it's got a learning curve. I'd like to adjust the terminology a little bit. I think Excel has a learning curve. Curves are smooth, right? Things like Python and C and Java, all these things. They have a learning cliff. Dr. Wayne Winston (01:06:22): That's brilliant. That's a better way to put it. Rob Collie (01:06:24): You have to climb this vertical surface. A slick, vertical surface under icy, snowy windy conditions. And we're also going to oil it, we're going to put some oil on the cliff too while you climb it before you can even put your first dot on a chart. Dr. Wayne Winston (01:06:39): That's you should write a book with these quotes. Rob Collie (01:06:41): And you mentioned this earlier, you think Python is the future. And I understand what you're saying, but I don't think so. I think the future is still Excel because of that learning cliff. Look at you. You came from a very, very rigorous academic background and it was only because of the accessibility of Excel to your audience, that you ended up drawn into it. Well, until something like Python has a learning curve, as opposed to a learning cliff, it's just never going to reach humanity in the same breadth. And this is something else that I talk about a lot, which is that Excel suffers a lot in its perception. Excel needs a better PR firm because it's grouped together with PowerPoint and Word. Everyone thinks of it as a document producer, a document container when it's really a programming language. The combination of the grid with the function language and the reference syntax, it passes every single formal test of what constitutes a programming language. Rob Collie (01:07:47): It is the most popular programming language by far in the world. In fact, the people who use all of the other programming languages combined don't come close to the number of people who are, even if we just limit it to people who are using Excel at a high level. I'm deliberately challenging something. I don't think we disagree, really. I haven't learned Python. Dr. Wayne Winston (01:08:04): I haven't either. Rob Collie (01:08:05): So there's something else we can be completely unqualified to talk about. Let's do it. What are the roles of things like R and Python? What tools do you use, Wayne, that are not Excel? Dr. Wayne Winston (01:08:15): Well, I mainly use Excel or add-ins for Excel. Like Monte Carlo simulation I use at risk and for data mining I use Dan Fouser, he bought XL Miner, which even if you have more than a million rows, it can sample. So I mean, you can fit a neural net and stuff like that. But I'll give you an example. I mean the Python people, if that's a proper phrase, they truly hate Excel. Now why do they hate Excel? I would like insight from you guys. Dr. Wayne Winston (01:08:44): But I'll give you an example. I have this book, Mathlethletics on Math and Sports. I wrote it all in Excel, because I know Excel plus high school kids can read that book and even middle school kids read my book on math and sports because they can follow it because it's in Excel. To me, one of the best things about Excel is you see your data. So if I want to like square something for regression, I say column A squared. In Python, I could say that, but I don't see it. Rob Collie (01:09:11): That's right. Dr. Wayne Winston (01:09:11): I mean it's the grid. It makes it really easy to see. So we just wrote a revision. I have a co-author who does Python. And so he did all the stuff in the book in Python, we still wrote the book in Excel. So one of the reviewers said, I love this book. It's perfect. One reviewer said, why did these stupid authors put this book in Excel? Why isn't everything in Python? I should have used the word, the cliff that you just used when I was talking to my editor. We'll put the Python on a website so if a data scientist wants to teach our book in Python, they teach our book in Python. You're right, 90% of the people who are going to use our book in class on math and sports. And it is the best selling book in that subject. Not a big market, but basically they do it in Excel. Dr. Wayne Winston (01:09:59): And I mean I can do logistic regression in Excel. There's a nice ad in Excel stat at letter X L L stat by a guy in France that does really advanced statistics in Excel. And I mean, you're right, it's a cliff, and you have to sort of master the whole Python language. And I think where Excel has tried to bridge the gap is Power Query. If you want to scrape the web efficiently for sports stats, Python's probably way better if you know it. But Power Query, if you're good at it, I think Power Query is fantastic. I don't know who developed that at Microsoft. How much of your classes do you spend on Power Query? Rob Collie (01:10:41): A third. Dr. Wayne Winston (01:10:42): Okay. Oh that's yeah, that's good. Because there's not as much to learn. I would go to a power company in South Carolina and this person she spent two hours a day doing one task. I said, well you can do that in Power Query in 30 seconds. So I told her that and by the end of the break, I'd saved her an hour 59 minutes a day. It was just filled down. She'd have something in the first row, 10 rows of blanks, then something in 20 rows of blanks, something with 30, but it was always a variable number of rows. I said just do fill down and you'll fill them in. I mean, she was so grateful that she said, boy, that saves me so much time, 10 hours a week. And there are so many people who need to know Power Query and of course Power Pivot. You can do them on a hundred, 500 million rows, right? You work for Medicare or Walmart, you have to do that. They may be your clients. I don't know. Thomas LaRock (01:11:41): So I've spent, I mentioned earlier, over the last few years, I've tried to pivot a little bit and do more data science stuff in my career. And I spent the last year taking just about every Python class that data camp has to offer. I don't know how many that is at this point, but I'm probably not qualified to answer this at all. But what I wanted to offer was, I think Rob... Thomas LaRock (01:12:03): This is all, but what I wanted to offer was, I think, Rob, you're coming from this angle of saying, "It's one or the other." And I would tell you it's both. The future has both. And I can't help you understand why, like Wayne said, Python people hate Excel. I have no idea. I don't hang around with Python people. Maybe they do. But what I see is I see a future of both, because when I'm trying to work with Python sometimes I'm sitting there going, you know what? I understand they want to teach me how to do this in Python. But if I had to do this for my job, this part would be in Excel. And then I'd save it as a CSV and then I'd bring it in and then I'd do the data sciencecy stuff I need to do with Python. Thomas LaRock (01:12:40): So to me, I see a blend and where you can see that blend, like Wayne said, I can't see my data, but if you're using R and you use RStudio, you do get to see the data frame and you do get to see the data. And that's a lot of power. I don't know of anything similar for Python, that's similar to what RStudio does, but there is a place for both the extensibility that Python offers and the usability that Excel offers. And I think the future is a blend of those two. Rob Collie (01:13:08): And you know what, Tom, we have people at our company who are good at Python. We're in agreement that it's both. I do, however, I think, have an answer to why the Python people hate Excel. Dr. Wayne Winston (01:13:18): Okay, good. Rob Collie (01:13:19): And you know what it is? This is one of those cases where the rule is, just take the cynical explanation and it's right. Thomas LaRock (01:13:26): Is that like Occam's razor? Rob Collie (01:13:27): Yeah, it is. It's just elitism. Dr. Wayne Winston (01:13:29): I think that's right. Rob Collie (01:13:30): That's all it is. And that's really disappointing because it's this other topic that Tom and I have talked about, nerd bullies are the worst. We grew up as nerds. I know I did. People don't believe this about me, but I grew up, I was getting stuffed in lockers left and right. [crosstalk 01:13:48]. I was getting abused and traumatized at school every single day. And a lot of us were, right? And then to grow up and then acquire some sort of technical expertise and then to use that as a platform to continue the cycle of violence against others. It's just one of the most repulsive things in the world to me. And so when I see that kind of behavior and that's what it is, it is elitism. Rob Collie (01:14:17): I mean, I've been around a lot of technical types my whole career. I study humans very intently. So whenever I encounter that type of thing, I'm just like, "Oh, this is the good stuff." I love antagonizing that kind of crowd. The Linux cool kids... That is the best, getting after them showing them the things that I can do. Oh, with my so uncool Microsoft tool set. I can do things that they can't imagine doing. They can't even fathom being able to do this. And I'm just like, "Yeah, well, you know, all you have to do is just be a little more humble and not be such a jerk and be a little more open-minded and you can do these things too." Dr. Wayne Winston (01:14:57): Yeah. But one point on why I think the Python crowd doesn't like Excel and partially there used to be when they ran regressions, they'd give long answers. [crosstalk 01:15:08]. And I think that started the statistics. People saying you can't trust, Excel. And so from that point, the philosophy that you mentioned, took over. I think Microsoft left themselves open a bit. Rob Collie (01:15:22): That'd be a very charitable interpretation of human nature. I'm not going to subscribe to [crosstalk 01:15:26]. Dr. Wayne Winston (01:15:26): No, but the cliff is, why should you have to like climb El Capitan or whatever when you can go up a 50 foot hill? I mean, whatever you want to call it. Rob Collie (01:15:37): If I'm a Python or whatever... A programmer, a developer, I'm not going to know Excel. I'm not going to know it. If you hate Excel, you also don't know it. Dr. Wayne Winston (01:15:46): That's true. You have to love Excel to learn it. Rob Collie (01:15:49): And vice versa, right? This is a chicken and egg kind of problem. It's just like a self worth thing. "If Excel is good, then I'm worthless," I think is the psychology, right? Dr. Wayne Winston (01:15:58): You're sure you want to be put this out on the internet? Rob Collie (01:16:00): Well, we're going to find out. Dr. Wayne Winston (01:16:03): I don't know. I'll be really interested, but I actually think you're 80% right, at least. Rob Collie (01:16:10): So Tom, we've got audio of him saying that you're right. And that I'm 80% right. Thomas LaRock (01:16:14): That is also going on my blog. Dr. Wayne Winston (01:16:18): I can just tell you that the really good MBA programs, they're now adding Python courses that a lot of the students are taking. Rob Collie (01:16:27): Does the marketing track at IU require your type of Excel class or not? Dr. Wayne Winston (01:16:32): Well, I'll give a good plug to the Kelly School of Business. The big four accounting firms. Oh, we say Kelly Indiana University School of Business. The Excel for the undergrads is far and above what any other undergrad school teaches. And a lot of the people teaching it, I'll brag on this, are my former students. We make the students take two courses. The first one is half Access and half Excel. And the second one is basically the management science class. And then if they're in marketing major, they have sort of a database class, which is in Excel and maybe a little bit of SPSS. But then they just add in the undergrad program last year, Python class, which a lot of kids are taking or they could take in computer science, but business students hate that class. Dr. Wayne Winston (01:17:20): We've got somebody really good teaching in the business school who teaches Python. The people I've talked to, who took that course, are very happy, but like I can tell you, Columbia, University of Chicago, Northwestern, they're adding Python electives to their MBA program. And that's just a trend from the last year or two. It's going to be a non zero percentage now. Like if you want to be in sports analytics, you couldn't get an interview if you didn't know Python. Rob Collie (01:17:47): I was just going to say, Mark Cuban is learning Python. And that's all I needed to know. Dr. Wayne Winston (01:17:52): Oh, okay. I didn't see that. But I've talked to a bunch of the analytics people in sports and no matter how good you are, if you can't show you know Python or R you won't... And he sponsored AI for underprivileged kids, a seminar this week. Rob Collie (01:18:08): We're not going to talk about this in any depth here, but we'll reserve this for some other time. Dr. Wayne Winston (01:18:11): Yeah, I'm glad. Anytime. This has been wonderful. Rob Collie (01:18:13): A friend of mine just bought an actually relatively prominent known soccer team in Europe. He wants to do the analytical approach. He wants to do all that kind of stuff and he wants our help. So we're going to need to put our heads together a bit. Dr. Wayne Winston (01:18:28): I'll talk to you about that. Rob Collie (01:18:30): Super exciting. By the way, you have a chapter on hedge fund performance in your new book. This guy is also a former and, I think, somewhat current hedge fund manager and a former professional soccer player and a former physics major who will be happy to talk to you about relativity. Dr. Wayne Winston (01:18:49): You know it more than I do. Rob Collie (01:18:49): I'm going to see if I can eventually get him to come on this show, but he kind of likes to stay in the shadows. We'll see if I can get him to pop cloak, but for now he's going to remain a mystery. Dr. Wayne Winston (01:18:57): But can I say a word about analytics and soccer and I'm sure you see this in your work. Okay. What data should the people be working with? To run their company well, what data do you need to do your consulting on? Do they usually know the answer or do you help them get to that answer? Rob Collie (01:19:16): It really depends. A Lot of times when we get brought in, they have a pretty good handle on what it is that they need. It's just really hard to get there. And Power BI makes it super, super easy, at least compared to traditional methods to get there. And so we're just like lightning in a bottle for them. Dr. Wayne Winston (01:19:33): Oh, I see. Rob Collie (01:19:33): At the same time though, there's another principle that we've developed over time. And then this is just the absolute truth, which is human beings don't know what they need until they've seen what they asked for. Dr. Wayne Winston (01:19:45): Oh, that's another aphorism for your book, because I'll tell you the thing I'm bad at teaching or I don't spend enough time on it. What data do we need to solve a problem? You guys are probably the best qualified to write that book. And it would get used in schools. Getting back to your soccer. The key in soccer is digitizing the video because nobody ever scores. And baseball they just came up with Statcast. So you watch a baseball game, they'll say, he hit it a 105 miles an hour and it went 450 feet. And the launch angle was 30 degrees. So they digitized the video. And basically until five years ago, they never knew that stuff. So the baseball data set based on the cameras in the stadium is for this whole season, a billion rows. Now I think Power BI could, and. Dr. Wayne Winston (01:20:32): I would bet teams are using your power pivot without... Maybe they're your clients, I don't know, but basically I'll bet you there are big Python people, but I bet, to generate their reports on what they need, they would be using Power BI if they weren't to do a lot of their stuff. And it's like, "Is a pitcher going to get a sore arm"? You look at his spin rate and velocity over time and see if it's dropping and then you should rest them. I had an Uber driver in Florida, it's a tragic story, he pitched for the Red Sox in the year they won the world championship and then he pitched too many pitches in one game, and the next day his rotator cuff was torn. He was 23, end of career and he is driving Uber. If they had understood then that he thrown too many pitches. Dr. Wayne Winston (01:21:16): He might have pitched 10 years in the major leagues. But now they took that guy out in the Dodger game and they shouldn't have taken him out because he was unhittable and the analytics said to take him out. So analytics can only be a piece of the puzzle. You have to recognize non-analytic things in your business and I'm sure you run into that. Rob Collie (01:21:39): Yeah. Maybe they made the right decision for the pitcher, but the wrong decision for the org. Dr. Wayne Winston (01:21:44): Oh, that's a very good point. And they only scored one run so they weren't going to win anyway. Rob Collie (01:21:48): Yeah. [crosstalk 01:21:49]. Dr. Wayne Winston (01:21:49): So I don't know. I don't know. The key in sports is what data do you need? And we have huge data sets now, which is why the Python people are needed. So that baseball data set is a billion rows. And they're just trying to figure out what to do with those billion rows. If you look at the Dodgers webpage, I think they've got 20 analysts. Now American football is the last sport to really have good analytics. And I still think they're working. The Colts have an analytics team and we had an interview with them, but they thought they were good enough. Sports is probably the best field to teach analytics in to people who know sports, because the problems are sort of well defined. And you can say, "What data do we need"? I know enough about business that I can go into a business and I can probably say what data you need to be keeping. And you may not be keeping it. And then what to do with the data. I'm good at that. But basically in sports it's been... We wrote the Math and Sports book 2007. It was obsolete before it came out. And every day we sent the final edition of our Math and Sports second edition. But it'll be out of date by the time it's out in the year because the field's changing so much. Rob Collie (01:23:04): Well, the NBA was the first to add the sport view cameras, right? Dr. Wayne Winston (01:23:06): I believe that's true. Rob Collie (01:23:07): It wasn't even league wide to begin with. It was only certain arenas, right? [crosstalk 01:23:12]. Dr. Wayne Winston (01:23:11): Teams had to pay. You're right. Rob Collie (01:23:12): Yeah. So I assume that the Mavs were probably in that first wave. Dr. Wayne Winston (01:23:17): Mavs and the Rockets. And the Rockets' GM just left for Philadelphia. Rob Collie (01:23:21): That's right. Dr. Wayne Winston (01:23:21): The rockets never did that great, no offense. Rob Collie (01:23:23): So it's only fair if Hinkey now goes to the rockets, right? Dr. Wayne Winston (01:23:27): Right, Hinkey. Yeah, trust the process. He was fired by Philadelphia. Rob Collie (01:23:31): Yeah. Trust the swap. Dr. Wayne Winston (01:23:32): Yeah. Well, and the funny thing is Daryl Morrey believes in three pointers, which is the right thing to believe in. He's now through a team where they have the worst three point. Ben Simmons has made two, three pointers in his career. Rob Collie (01:23:42): Yeah. Enough that both of them made SportsCenter, right? [crosstalk 01:23:47]. Dr. Wayne Winston (01:23:47): Probably. [crosstalk 01:23:47]. Rob Collie (01:23:47): A Hundred percent of his threes are on Sports center. [crosstalk 01:23:49]. Dr. Wayne Winston (01:23:49): I couldn't believe because Daryl said he quit to spend more time with his kids. And so now he takes this other job. The owner of the Rockets is a difficult guy to work with. [crosstalk 01:23:58] Rob Collie (01:23:57): Maybe Ben Simmons is his kids now, you know? Dr. Wayne Winston (01:24:00): Yeah, Ben Simmons will be traded. I think he could bet on that. Rob Collie (01:24:02): I've researched the Sport View stuff. It's rampantly deployed in soccer worldwide, those cameras are everywhere. So they're digitizing just like they did for the NBA. I've all also had some sessions with their sales team where we've gone through what's available and what they've got and everything. It's really impressive. But at the same time, the same sort of questions you're asking, I think, they're still pretty early into figuring out what's actually relevant. Dr. Wayne Winston (01:24:26): I can tell you what's relevant in soccer. There was a paper and actually was written by a woman who worked for Microsoft. The problem with analyzing soccer, and to be honest, why I don't watch it that much, is you can go to bathroom and miss all the scoring. Okay. So basically if you want to evaluate players, you have to sort of value the movement of the ball. Like if somebody steals the ball from me, that's worth minus something goals. If I make a penalty kick, that's worth plus so many goals. And the key to doing that is what's called a Markov chain, which you may have studied in undergrad. Some of you- Rob Collie (01:24:58): In computer science. Yeah [crosstalk 01:24:59]. It's queuing theory, right? Dr. Wayne Winston (01:25:01): It's a state. You have states of the system and where the ball goes as a state and where it goes next is the next state. Rob Collie (01:25:07): Yep. I read a book one time where there was a character in the book named Markoff Chaney. He represented the random element in the story. Dr. Wayne Winston (01:25:15): Was it science fiction? Rob Collie (01:25:16): It was like 1960s, hippie Robert Heinlein inspired- Dr. Wayne Winston (01:25:21): Oh, like Stranger in a Strange Land? Rob Collie (01:25:23): Yeah. Yeah. It had that kind of vibe to it. Dr. Wayne Winston (01:25:26): No that's cool. Figuring out you needed to digitize. So in the NBA, the cameras, I think, look at the ball and the players 20 times a second. That's the humongous data set for just one game. And then they all now have the same data in baseball and basketball and NFL. They just hired a director of analytics. And so they're really getting better camera data in the NFL. But you'd think with that being the most important sport in this country, they would've had that data years ago. But basically in hockey has, because they played in the basketball arenas, the analytics of hockey has been coming up recently to be very important. I talked to the Indiana University, swimming coach. Dr. Wayne Winston (01:26:04): He thinks, are there things he can do with analytics and swimming? You got to know the sport and you got to know... Some people are good at thinking about data, "Like what data do I need to solve a problem"? Most people are absolutely horrible at that. I wonder if the people who take your class, do they have the knack for knowing, when they come into your class, what data you need to show them how to work with? I don't do a good job teaching that in class, because I run out of class time. But it's real important. Rob Collie (01:26:33): You've heard me express some cynical viewpoints, but this is one on which I'm pretty optimistic. The people that come to our classes, in general, like the VLOOKUP and Pivot crowd. These are just really, really, really good people. Dr. Wayne Winston (01:26:46): Right. And they know their business. Rob Collie (01:26:48): I love them. Like if the world were only VLOOKUP and Pivot people, we would live in harmony. It would be such a better place. So I have a really relatively high opinion of this demographic. It doesn't mean that we're perfect. It's always a work in progress, right? Dr. Wayne Winston (01:27:04): No, you're absolutely right. You were talking about like, you've got to know the sport, but you've also got to know data. You've got to have that data gene that we talk about. Dr. Wayne Winston (01:27:11): And that's another great phrase. Rob Collie (01:27:13): It's that decathlon thing again. [crosstalk 01:27:15]. If you have people who know the sport and you have people who are good at the data, but that's not the same brain- Dr. Wayne Winston (01:27:20): Usually not. Rob Collie (01:27:20): ... You're going to be very inefficient and really ineffective. And this is true in business. This is why we're seeing BI migrate outwards from the IT hub into the business units in the form of Power BI for instance. And it's the VLOOKUP and Pivot people that are being deputized in this game. Because they have that business knowledge, the tribal knowledge, all that kind of stuff. Dr. Wayne Winston (01:27:44): And they have the data gene. Rob Collie (01:27:44): And they have the data gene, which again is not rampant in the business, but there's like one out of 16 that have it. And when those two things, the ability to execute and the knowledge of the business, the domain. Same thing in sports, right? I think most people get into sports analytics, especially these days. They like sports and they like data. But for a while there, they were just grabbing, like Wall Street Quants. Dr. Wayne Winston (01:28:06): That's who runs the Dodgers. Rob Collie (01:28:07): Yeah. It's worked. So- Dr. Wayne Winston (01:28:09): But they knew sports. Those people, I think. Rob Collie (01:28:12): Tom, do you have any pardoning questions? Thomas LaRock (01:28:14): The one question I have for Wayne, because he made that bold statement that Mark Cuban is going to run for president in 2024. Is that something that we can use when we promote the podcast? Like as a question, "Will Mark Cuban run for president"? [crosstalk 01:28:26]. Dr. Wayne Winston (01:28:26): I'm just putting two and two together. I said him, the chapters of my book on politics and he was interested. And basically he did this thing on ranked-choice voting. See the way the country works, the Democrats and Republicans, they have almost nothing in common except one thing... They want nobody else on the ballot. They've made it very hard to be on the ballot. And I think he is going to spend the next couple year, I'm not saying he'll run, I shouldn't have said he'd run, he's going to spend the next couple of years thinking how you can get a valuable third party option and ranked-choice voting is the key. If we get third party on the ballot, the country would be much happier if they were able to rank the candidates. Maine, and one of its congressional districts is doing that. Dr. Wayne Winston (01:29:13): Minneapolis does it in all their elections. I'm going to teach that in class later today. And I think it's very important, but I think, Mark, first of all, he hates Donald Trump from a long time back. I mean, he's not alone in that. I'm an independent, I'm in the middle and the people in the middle, we have nothing. [crosstalk 01:29:36]. Ranked-choice voting, 35% of the population classifies themselves as moderate. But the primary system, basically, you get either the far-right or the far-left. That's how it works. Rob Collie (01:29:46): I lasered straight to this chapter when I saw the table of content. This is where I went immediately. Dr. Wayne Winston (01:29:52): The voting chapter is really interesting. The math of voting. Rob Collie (01:29:54): I agree. I think you're totally right. That the primary system is part of the problem. Dr. Wayne Winston (01:29:59): That's exactly right. Rob Collie (01:30:00): You identified yourself as a moderate in there, which you know that, that middle 35% [crosstalk 01:30:06]. Dr. Wayne Winston (01:30:05): Probably a bad thing to do. Rob Collie (01:30:07): I really want moderate to get a brand makeover. You and I could both identify as moderate, somewhere between left and right. But in our day to day lives, would any of us really describe ourselves as moderate? It's such a vanilla milk [crosstalk 01:30:23]. Dr. Wayne Winston (01:30:23): Need a better term. It's like- Rob Collie (01:30:24): Yeah, I think we need like radical moderatism. Like the analytics driven. We need to disrupt, that there's something needs to change. And you don't typically associate moderate with a need or a desire or an enthusiasm to upset the apple cart. But with this apple cart needs to be upset. Dr. Wayne Winston (01:30:44): Right. We need compromising. There's just no compromising. Even under Reagan, there was compromising and basically these parties hate each other. Do you want my prediction for the election? Rob Collie (01:30:56): Sure. Let's let's put you on record. [crosstalk 01:30:58]. Dr. Wayne Winston (01:30:58): And it's horrible. My prediction is, Trump will be ahead of election night, but when all the mail-in votes come in Biden will be the winner. So we told the guy building our pool in Florida, what's going to happen. And he said, "Well, if Biden wins, Virginia has the fifth largest militia in the world and they're heading to DC. I mean, that symbolizes where we're at. [crosstalk 01:31:19]. It's going to be a disaster if, the scenario I mentioned whoever you're for, if Trump's ahead election night, but the mail-in votes, they're going to fight every mail-in vote for a month. And then it'll make 2000 look like a walk in the park. Rob Collie (01:31:36): We closed the previous podcast with Michael Salfino, by saying, "Yeah, we'll get together again right after the election. That is, if we're not out fighting the second American civil war on the street." Dr. Wayne Winston (01:31:46): You got that right. What is his job? Rob Collie (01:31:49): You'll really like Mike, he writes for The Athletic and FiveThirtyEight about sports analytics. Dr. Wayne Winston (01:31:54): Oh, I'll look him up then. I can find him. Rob Collie (01:31:56): The two of you haven't quite crossed paths yet, but you're going to need to. Dr. Wayne Winston (01:31:59): The second civil war in the streets is, if that was your phrase, you're not far off because, I mean, I can tell you, I've talked to people on both sides and as all of you have, no matter what happens, it's going to be bad. I mean, whoever you're for, it doesn't matter. It's going to be bad. Rob Collie (01:32:18): Wayne, thank you so much. [crosstalk 01:32:20]. This was a real pleasure. Dr. Wayne Winston (01:32:21): I'll come up there sometime to see you. Thomas LaRock (01:32:22): Dr. Winston. Thank you. Dr. Wayne Winston (01:32:24): Bye. Thanks. Thomas LaRock (01:32:25): Thanks for listening to the Raw Data by P3 podcast. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Interested in becoming a guest on the show? Email LukeP, L-U-K-E P, @powerpivotpro.com. Have a data day!

Brad Miller and Kai Hankinson from Agree Media were Rob's first clients at P3 before it was even called P3! These guys have been close friends and business partners for a long time, and they know a lot about data. We cover: Their ups and downs in various business ventures How they wrote and used AI software to win at online poker, while living in India Organic farming in Hawaii, also known as WWOOFing Their connection with Snoop Dogg How Agree uses Power BI The Lead Generation Business The differences between Facebook and Google advertising Episode Transcript: Rob Collie (00:00:00): Hello again everyone, one of the gifts of my career is all the interesting places it's taken me during some interesting times and all the interesting people I've met along the way. Well, today's guests, Brad Miller and Kai Hankinson are two of the most interesting people I know. Ivy League MBAs, entrepreneurs, rabid users of Power BI. Along the way, they've written poker bots, lived in India, they've been sued by Facebook, and they've lived off the land in Hawaii. But they're more than just interesting people to me. I'm really grateful to say that they're also my friends. Friends who've been along for the whole ride. They were there before P3 was even a company. And really they were the guinea pigs for the whole idea. When I had a full-time startup CTO job, they'd hire me to consult with them on the side, even though I wasn't taking consulting terribly seriously yet. Rob Collie (00:00:49): So they watched P3 develop from literally zero into the nationwide firm we are today. More than watched actually, they helped and vice versa. I've been along for their journey as well. So what I hope you hear is an authentic conversation between friends. A conversation revolving around data, for sure. But again, that human element looms large, the filter was completely off for this one. We talk a lot about failure, rebounding from it, learning from it. We laugh a lot about how hard it was to get them onto the Power BI train. Even though today, they can't live without it. And I think you'll see there's no shortage of personality in these two. It's a long one, but I don't think there was a single dull moment in the conversation. So let's get after it. Announcer (00:01:33): Ladies and gentlemen, may I have your attention please? Announcer (00:01:37): This is the Raw Data by P3 podcast with your host, Rob Collie. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Raw Data by P3 is data with the human element. Rob Collie (00:01:53): All right. Welcome. My California startup entrepreneur friends. Thanks both of you, Brad and Kai for coming on. Really been looking forward to this. You guys have been in business together for how long now? How long has Hankin Miller Enterprises been a thing? Brad Miller (00:02:10): I think we got started what, 2005, Kai? Kai Hankinson (00:02:14): I think we got started in 2004 actually. Brad Miller (00:02:16): 2004, so it's a long marriage. Rob Collie (00:02:20): Yeah, I was going to say, and one of you has already forgotten the anniversary. One of you's already forgotten how long. So it's a realistic marriage, isn't it? Brad Miller (00:02:30): It's a very realistic marriage on a lot of fronts. Rob Collie (00:02:33): Yes it is. So you guys met in business school, right? Kai Hankinson (00:02:37): Yep. Rob Collie (00:02:37): And not just business school, right? Ivy League business school. That's where you guys met. I'm an engineer sort of originally like that sort of what the machine churned me out as, as a 22 year old. From my perspective, you guys were in a completely different world. You were in the essentially like the financial and business world with almost like this made man sort of circle, in a way. Right? But today I think of you as data professionals in every bit as much of the real sense as I am. So Wall Street, private equity, my expectation as a borderline Vulcan nerd is that once you're in that world, you would just stay there. You know, you would just stay in that world. You would stay circulating in that Wall Street and private equity world. You'd have other people doing your data for you. You wouldn't be getting your hands dirty in that. You'd just be too important for it. That's my expectation as an outsider, but that's not where you find yourselves. Rob Collie (00:03:39): And so I just want to start here, why did you leave? Why did you leave that world to go do your own thing? And some of the things you've done, as we're going to get to, are very interesting. What led you to branch out like that? Brad Miller (00:03:52): Kai, you want to take that one? Kai Hankinson (00:03:54): Yeah. Brad and I, we met at the first day at Columbia Business School, and we were assigned this little group of four students together that did a semester long project. Brad and I hit it off as friends, and it wasn't long before we started up a weekly poker game at my house where we would take money off of the rest of our business school friends, who mostly came over to socialize and drink the free beer we bought them. We were taking the poker much more seriously just because that was our thing, and we wanted to win. Rob Collie (00:04:29): I've had friends like you at poker games. You ruin everything. Kai Hankinson (00:04:37): Yeah. So we had a lot of fun playing poker through business school. It was around the time that the Rounders movie came out. No, we went to school 99 to 2001. So we were in Manhattan and we'd occasionally hit up those Rounders style clubs in Manhattan that the movie was kind of written after. Those places used to scare the shit out of us, in a good way. But we would just walk into places, scared to death, just these little business school dorks. Trying to look like we knew what we were doing, and we pushed some chips out there. Rob Collie (00:05:09): And then Teddy KGB. Kai Hankinson (00:05:11): Yeah. Rob Collie (00:05:11): Sitting across the table. Yeah. Kai Hankinson (00:05:14): Yeah. You don't want to mess with Teddy KGB. Anyways, we graduated business school finally and went to for the man. I was on Wall Street for Merrill Lynch and Brad went out to California with Deloitte, but we just kept in contact about the poker because we were really interested in it. We started studying the artificial intelligence projects going on in poker at some of the institutions like University of Toronto, I think was a big one. This was back in the day when the AI had beaten the chess problem, but hadn't yet really made a dent in the poker problem because it was a tougher problem. Chess is a perfect information game. Everyone can see all of the board and poker isn't, and because of that, you get the whole component of bluffing and everything else. Kai Hankinson (00:06:04): And so it's just a harder problem for AI, but we were really interested in their attempts to solve it with AI. We ended up getting this idea that we could break the poker hand up into the parts that were best for AI and the parts that were best for human, and just divide up the workload that way. And together build a best case player. We ended up raising some money. We actually raised the money in 2004, which was the reason for my answer, and we quit our jobs in 2005, which was a reason for Brad's answer. We raised the money and left our gigs, and started up a company dedicated to playing online poker with artificial intelligence assistance. Brad Miller (00:06:51): I will say that it was easier for me to leave my job than it was for Kai to leave his job. Going back to your question, I had always considered myself an entrepreneur and I knew that's what I wanted to do. So my time in management consulting was time I was putting in there. It's also very difficult to get rich charging by the hour. Right? Which is the consulting model. So I knew wasn't really my long term goal. Kai, on the other hand, had landed a very plush gig, trading equity derivatives for Merrill Lynch. It took me a while to convince him to leave. I think, at the time, his wife, at the time, wanted to move back to California, and the New York lifestyle might have been killing him a little bit, and so that kind of helped the cause. Brad Miller (00:07:41): During the whole time I had been playing on the nascent online poker rooms, and there was some really basic software that was available at the time, I forget the name. There were all these upgrades that I wanted to make to the software that I was like, "the guy's missing this, he's missing that, he's missing this." And so I'd keep telling Kai about it, and Kai would agree and be like, "what you're describing is a lot like kind of the software and the UI that I use at Merrill to trade derivatives." And so we started to design what this ultimate software product might look like? Rob Collie (00:08:20): Did you hear how Brad started that? He said, "it's really difficult to get rich charging by the hour." You had to throw that in there, Brad, you had to cut me. As the owner of a consulting firm, don't I know it? You know? Brad Miller (00:08:33): I think the difference is you're the owner, so if you've taken that step and taken the additional risk, right? Rob Collie (00:08:40): That's true. I'm still aspiring to be you guys when I grow up, and you're still aspiring to be someone else when you grow up. It's how the game is played. If I can paraphrase the last couple minutes, we really enjoyed playing poker with our business school friends that weren't as good at it, and taking their money. And then we said, what if we could take that and really scale it? What if we could do that online? That'd be great. Brad Miller (00:09:07): I think that's right. One more thing I'd want to throw in though is that I think both of us live by a kind of philosophy where if you're not willing to publicly admit that you do something, then you probably shouldn't do it. And so it's kind of a motto we live by. I know I definitely felt like I was living two lives at the time. I was living Brad, work Brad, for a consulting firm. And then I was living private Brad, and I just really didn't enjoy that. I was like, I want to be one person, I don't want to have to put on an act, I don't want to have to satisfy the man. Brad Miller (00:09:49): Things like Kai went to this mushroom ceremony a couple months ago and had an amazing experience, and wrote an email to our entire team about it, explaining the ceremony and what he had learned. You can't do that at Deloitte. If you want to go run around naked at Burning Man, you should be able to go run around naked at Burning Man; but Deloitte customers probably wouldn't appreciate it. So, that's what I mean by kind of two separate lives. Rob Collie (00:10:17): There's all these like workplace personality assessment, there's a million companies that do this. The ones that I've done, I've done the same one a couple times at different points in my life. They give you two pictures. They give you one of yourself in the professional environment, and one of yourself in the personal environment. There's almost also this distance score between the two. Rob Collie (00:10:41): The mediator in the first session, when I took did this at Microsoft, told us that those of you with a large distance between these two, the most authentic moment he could offer, he was just going through script for all day. Right? But this one moment you could sort of tell he was telling you something that was real. He was sort of dropping the script and he was saying, "if you have a really large distance there, it takes its toll." He said it that slowly, and then he repeated it, "it takes its toll." I'm sitting there looking at my sheet, and me and one other guy and this room of like a hundred people, like we're these outliers with this most dramatic difference between our personal and our professional. And I'm just like, "Ahh!" You know? Took it again recently, but my distance isn't as great. Both sides of me has changed over the 10 years intervening. They've sort of moved towards each other, and I feel a lot better. Brad Miller (00:11:35): Yeah. Rob Collie (00:11:35): I can certainly appreciate that. Did you get rich with the poker thing? Brad Miller (00:11:39): So scale, we did. We ended up moving the company to India because we wanted to use one of the outsourcing firms there. We were working with one of the outsourcing firms. We ended up putting together a group of a couple of hundred Indians that would use our software that we had designed and developed to play online poker at the various rooms. It was pretty crazy. We were doing things where the software was recording every hand online at these poker rooms. So we had data on every handle that was playing. So that was accessible while you were playing against that opponent. Our software allowed you to play upwards of 10 to 20 hands simultaneously. We then started doing things, because it was all through our UI, we would break the hand down into its various components. For example, we would have an expert that would only see hands that had pocket pairs in early position. Brad Miller (00:12:45): Whenever we got dealt up a pocket pair in early position, we would route that to Indian call center agent number 12. So, on his screen at any point in time, you could have 12 hands, only from the flop forward, only pocket pie pocket pairs in early position. That's all he knew. He was an expert. That's what he did. So we just designed the whole thing kind of similar to how he would design a call center, and how you would route different phone calls. We would route poker hands the same way. Kai Hankinson (00:13:19): Yeah, in order to do that, guys and girls would get thrown into the middle of poker hands. So in order to do that, and also to facilitate the ability to play so many hands at the same time, Brad invented this graphical representation of a poker hand to where, in a quick instant, you could catch up on all the action that had occurred up to that point. It really was one of the coolest things we came up with, I think. It was just so powerful to be able to, even if you were playing in the hand the whole time and you forget that a lot of the little stuff, and with this, you can at any point, look back at every little bit and be like, "oh, that's right, he did call that cold on the flop." Rob Collie (00:14:02): Do you have a screenshot of that laying around anywhere? This is a data visualization technique, right? It'd be a really interesting novel thing. We won't actually do this, but it'd be a neat thing to build the Power BI custom visual that reimplemented what you designed. If we were just laying around with nothing to do, we would absolutely do that. Brad Miller (00:14:22): Yeah. It'd be cool. I'm sure we have it somewhere. It's also just such a neat experience because they don't play poker in India. And so we had to train everyone up from the get go. We also couldn't let them that they were actually playing real poker because they were betting, in some cases, hundreds of dollars per hand, they were only making $10 a day. If they knew that they had hundreds of dollars on the line in any given hand, they would just freeze and wouldn't play optimally. So we basically had to tell them that they were just testing the software and they were trying to train the software up. So it was a crazy experience. Kai Hankinson (00:15:03): That was a wild experience. You would walk around that floor, that call center floor, overlooking the shoulder of all these little kids. They were like 20, 21 to 24 years old, just so innocent and fun. You'd look over their shoulder and you would know what was going on, and I'd be nervous as crap like, "oh please, please don't screw this one up. You got to raise him here, you know, right?" I'm thinking, but I don't want to interrupt him because he is also playing four hands at the same time. What a wild experience. Rob Collie (00:15:40): Now, Luke is usually just the silent producer on these things, but in the same timeframe, Luke was playing a lot of online poker. Luke, now do you see what you were up against? Luke (00:15:51): Well, yeah, I might have played against you, and that's the reason why I sucked so bad. Brad Miller (00:15:58): Now, we were playing primarily at the lower limits in the limit games. So, once you get to the high limit stuff, the software wasn't able to compete. So we were mostly in it for the quantity of the quality. Kai Hankinson (00:16:14): We were playing quarter million hands a day, quarter million hands of poker a day at the end. So, maybe we should say though that eventually we got with the Indian operation to where we were highly poker profitable, but the company wasn't yet profitable. We weren't covering our full operating expenses. We were running out of our runway from the money we had raised. We decided to make a bet that we could automate the rest of the hand, and both cut down our cost structure quite a bit and play even better poker than we were with the human training. Kai Hankinson (00:16:52): We eventually packed up from India. We came back to San Diego, boarded ourselves up in an office, and set out on a six month endeavor before we ran out of money to fully automate the hand. We made it just in time, at the end, we got up to a quarter million hands a day, when it was just servers, no operational nightmares, no humans to maintain and get everybody aligned up, logged in, just servers, just pounding out hands on every table we could get a seat on online. You say we were primarily lower limit, I think the truth is that's only because that's primarily what's played. We had guys at every limit poker seat. It was just that there's very few high limit tables. Rob Collie (00:17:38): Did you ever suspect that you weren't alone in this? That there might be other similar operations going on, and some of your hands you were playing, you were also playing against other bots? Kai Hankinson (00:17:50): You definitely wondered. It's just so unknowable. Rob Collie (00:17:55): Yeah. Kai Hankinson (00:17:55): People would accuse us of being bots regularly. I think that's just an accusation that probably flies around online quite a bit. We had a whole setup where when one of our players was mentioned, it would get routed to a human whose job was to like pipe in on the comment thread and come back over the top, "your mother is a fucking bot." Rob Collie (00:18:20): You had a specialist for that as well. Kai Hankinson (00:18:25): Yeah. Yeah, we did. We didn't like being called bots. Rob Collie (00:18:28): Yeah, those accusations that are true, those are the ones that hurt the most. You know? Kai Hankinson (00:18:34): Yeah, we have a poker reputation to maintain as a big bluffer. Rob Collie (00:18:41): At the same period of time when you guys were doing this, I was working at Microsoft and playing a lot of World of Warcraft because, like you were saying earlier, you get this difference between your personal and professional self. There's a point at which you don't really pour as much into the Microsoft job, right? You start staying up late playing these games. There's bots all over those games. Those games are loaded with bots that are running around farming in-game currency to turn around and sell it for real world currency. You can report the bots, and if someone tries to talk to them and if the bot seems human-like in its response, the bot gets to go on its way. Look at the sophistication of the CAPTCHAs that we have to navigate on the internet these days. We're reaching the point where the CAPTCHAs are harder for us than they are for the bots. We're not too far away from the point where the bots are better at solving the CAPTCHAs than humans. Then what do we do? It's sort of game over. Kai Hankinson (00:19:42): The best defense is if you can pass a quick Turing Test. Rob Collie (00:19:46): Oh yeah, of course, everyone knows about that. The Turing Test, if it turns out that, in those situations, the bots are just routing to a real human to pass the test, what are you going to do? Game over. I asked, "Did you get rich playing with the poker business?" You said, "Well, we scaled." Brad Miller (00:20:08): We did not, we got very poor with the poker business. There were two things that went against us at the poker business. One was in around 2008, they passed some legislation that basically killed the industry for a while. All the poker sites were based in offshore jurisdictions that the US government couldn't touch. So what they did was they regulated the bank transactions between the US banks and the poker sites. So there was no way to get money in and out of the poker sites anymore. Which effectively killed it. Which was a pretty smart way to kill it, if you're going to do anything. The other one was we thought that the poker software would kind of go the way of the market trading software, the stock market trading software, the kind of stuff that that Kai was using at Merrill Lynch. We thought that if you could build better software, great, you deserve to win just like in the market. If you can build better software, you're a hedge fund, you can build better software, great. The poker sites don't like that. We didn't quite realize that at first. The reason they don't like that is because what happens is you take it to the extreme. Brad Miller (00:21:20): Let's say this one guy at the table and he kills everyone else. Basically all the money goes to him. He stands up and he walks away, right? The poker site doesn't make any money. If everyone at the table is of kind of equal capability, then what happens is they just push money around hour after hour. Everyone's just sitting at the table, money's going back and forth. And the poker site takes their percentage out of every hand or out of every hour or whatever it is. So the poker site has a lot of incentive to make sure that everyone is kind of at an equal playing field, an equal level. And so they were very anti the kind of software that we were building. So we had those two strikes against us with were too big to overcome. Kai Hankinson (00:22:10): We were making about $5,000 a day in pure poker profit, and we were on our way to getting rich. We just didn't get the money out. Rob Collie (00:22:21): So you got poor instead of rich playing poker. Which, when you say it that way, makes it sound like it was predestined. I'm sure you both took that really well. Right? How many years did you have sunk in it at that point? Brad Miller (00:22:37): Well, that was about 2008, 2009 around there. So... Rob Collie (00:22:41): Wow. That's longer than I realized you guys were at it for four years. Brad Miller (00:22:46): Yeah, roughly four years. Kai Hankinson (00:22:48): Yeah. Yeah. The hardest part for me was we had raised all our money for that endeavor had come from friends and family. Rob Collie (00:22:56): Yeah. Kai Hankinson (00:22:56): So we were a total loss for our investors. That's one thing that I love about our partnership, Brad and mine, is that we've been through thick and thin together. You know? That's not the only company that we had a bad outcome on. We're on our fifth startup together, and we've had some depressing bankruptcies and some exciting exits, and we've been through it all. I've seen a lot of partnerships break up on the downside. I've seen a lot of partnerships break up on the upside, even. When, all of a sudden when you win and you exit, now there's something to fight over, and partnerships break up. We weathered all that together, we're on our, onto our fifth company together right now. Rob Collie (00:23:36): Yeah. That's amazing. Now, just a random side question. What do you know about organic farms in Hawaii? Kai Hankinson (00:23:46): Well, after losing all my friends and family's money, I got pretty down on myself, and Brad got right back on the horse and went right into private equity and was off to work the next day. I took a year and a half off of work just to kind of regroup and find myself. I ended up during that year and a half over in Hawaii doing something called WWOOFing. Which is working for free on organic farms. That was a great time in my life. I did a lot of meditating, lot of soul searching. Rob Collie (00:24:21): I now know too WWOOFers. Paul, at our company was telling me a similar story one day about, "Well, you know, there's this time where I ended up, you know, over in Hawaii and like, you know, living off the land on this guy's farm that he let us..." And I'm like, "Whoa, whoa, wait a second. I've I have heard this story before." What are the odds? I'm a WWOOFer magnet, apparently. Brad Miller (00:24:46): I decided to buy a one way ticket to Hawaii. Bought one of those kind of load of the ground trikes. I don't know what you call them, but... Kai Hankinson (00:24:55): Recumbent trike. Brad Miller (00:24:56): Recumbent trike. Folded it up, put it in a baggage claim, and then was sitting in baggage claim in Hawaii... Kai Hankinson (00:25:03): Baggage claim. And then, it was sitting in baggage claim in Hawaii, putting it together. And then as soon as he put it together, he just hit the road and stopped where he stopped. Brad Miller (00:25:11): Lived out of a tent. I went months without spending money. It's crazy over there. You can just live off the land, there's avocados and nuts growing everywhere. Rob Collie (00:25:22): I really need to try that someday. You're making it sound so good... All right. So poker, Hawaii, private equity and you did something else. What was the next one? Now we're really starting to... AI and ML definitely qualifies as a data thing. There would've been no real reason for our paths to intersect, especially back around 2010. So what were you guys doing in 2010 when we met? Brad Miller (00:25:57): At that time, we had gotten back together. I had come out of my hiatus and at the time when we first met, we were running a Facebook app called Page Rage. It exploded in popularity, it was back in the Facebook app days. That used to be a huge thing. I don't know that they still are. And Page Rage was this silly little app that let you color up your Facebook profile which was, as they are today, very uniform with everyone else's and all. And make it look a lot more like a MySpace page, which was the big competitor back then. People loved it. Brad Miller (00:26:34): That app exploded and we found ourselves in our second startup together with a lot of big data from all those Page Rage installs. Kai Hankinson (00:26:43): I think how did it go? Robert was like, "We just needed a better BI solution." I don't even know if we knew the acronym BI at the time. We just knew we needed a better tool set to help us understand this data. We were using Excel for everything that was after Excel 97. So that's when it went to like a million rows or wait, no. Am I getting the years wrong? Rob Collie (00:27:05): 2007. Kai Hankinson (00:27:07): Right. Rob Collie (00:27:07): Excel went to a million rows per sheet. Kai Hankinson (00:27:10): That was a game changer for Brad and me. I mean, we used Excel for everything and it was limited to 70,000 rows or something prior to that. But then we quickly outgrew the million rows and that's when we started circling around and I ended up finding you Rob online. And I called you and as it would turn out, you were the program manager on our favorite software tool, that Excel 2007 version, I think. Is that right? Rob Collie (00:27:38): Yeah. You see, this is great. You tell people the truth about yourself, like what you did at Microsoft. And then you just sit back and watch as your legend grows. There's an important word, he says, "You were the program manager." Well, I was a program manager on Excel. There were like 15 of us. Now, I was pretty senior on that team and I had things I was in charge of. I was in charge of the BI features in Excel, that kind of thing. But it does sound a lot better when you say it. "You were the program manager for Excel." And so when you introduced me that way and you've done that now a million times with other people, I just let it go. Rob Collie (00:28:18): I mean, of course, why would I interrupt you while you're... I said a program manager, you heard the program manager and you were dually impressed. I was working as the CTO of my first startup at the time. I had left Microsoft among other things, I had seen the promise of this Power Pivot thing that we've been working on. And so I was at a startup that was leveraging Power Pivot, as its primary software. They told me as part of my employment deal, that I was allowed to do some consulting. Okay, great. But I hadn't done any, I hadn't actually done any. Rob Collie (00:28:55): And so you guys and you know this, but for the benefit of the people listening, you guys were the absolute first clients of P3. And I didn't tell you that at the time. In fact, I even tried to get you to hire someone else instead of me, that's how good of a businessman I am, right? Like, "Nah, you don't want to hire me. There's a guy in San Diego who'll do a great job." And so I just point you his way. I tried to have you not hire me and you hired the other guy, didn't you? It didn't go well. Brad Miller (00:29:26): Hired and fired. Rob Collie (00:29:28): You called me back and said, "No, really, you should take our money, Rob. Really, you should take it." And I said, "Okay, fine." And I was just absolutely stunned at what you guys were doing. It was just the three of us in a conference room for two days. There were other people in the office, around and about, but it was just the three of us the whole time. I remember this moment where it started to dawn on me, what you guys were doing. And the scale of it, when we were looking at the number of active add-ons. The number of Page Rage add-ons that were out in the universe, calling home, letting you know they were still alive. Rob Collie (00:30:09): I don't remember what the number was, but it was like double digit millions. In 2010, it was October of 2010, something like 60 million. Was it 60 million? Was it 100 million? I don't remember, but it was just I had never heard of it. I had never heard of Page Rage. And yet there were like 100 million humans or 100 million devices out there in the world running your Page Rage software. I was just like, "What?" Kai Hankinson (00:30:37): Yeah. It was some mind boggling scale. It never ceased to me, how big the world is. It's hard to wrap your brain around it. Rob Collie (00:30:46): Well, wrapping your head around how large the world is, is one thing. Wrapping your head around, there's a reasonably good chance that if I meet a human being on the street, they're running my software, that's a different thing. You guys were paying what? You paid Snoop Dogg to tweet about how good his Facebook page was looking today, thanks to Page Rage and stuff like that, right? I mean, you guys were doing something next level. It certainly got my attention. Kai Hankinson (00:31:15): We had just got a written, the code tails basically, of Facebook. Because, this was right when Facebook was just exploding. And so we were writing their code tails. I think at our peak, we got up to 250 employees. And it was fast, it happened in a matter of two, three years. The Snoop story is funny. That whole influencer model is a lot more prevalent today than it was back then. But he was one of the OGs back in the day. And you would pay him to tweet something out, but he would make you come up with the tweet. So we had 50 white software engineers in San Diego, coming up with Snoop's tweet, which was pretty hilarious. Rob Collie (00:31:59): I'm sure it sounded very authentic. Brad Miller (00:32:02): We had experience passing that touring test, though. Rob Collie (00:32:05): Yeah. There you go, there you go. Are you Snoop? Check here if yes. So the financial transaction in that engagement, went my direction. Money came my way, you guys paid me. But I definitely benefited more from it than you did. Brad Miller (00:32:20): That's an understatement, Rob. Rob Collie (00:32:21): Yeah, I know. We'll get to that. We're going to drag my name through the mud, don't you worry, we're going to do a thorough job. I have a saying that I have been saying now for 10 plus years. Which is that, human beings do not know what they need until they have seen what they asked for. And the very first time this thought occurred to me was when I was working with you guys. We sat down with what has ultimately become our jumpstart methodology. Which, boil it down in layperson's terms is really just, just get started, is the philosophy. Rob Collie (00:32:56): Which we loaded a bunch of your data and the Power Pivot engine, the DAX engine, you did a great job chewing up tens of millions, hundreds of millions of rows. It was great. It was doing everything that you wanted in that regard. We got to a point where we had the chart, the pivot table, whatever that you guys had wanted that you'd described from the beginning. And then as soon as you saw it, you're like, "Ah, no. That's my not quite right. We need to blah, blah, blah, compensate for this, filter, that." That kind of thing. And we tore through four, five, six, maybe even 10 iterations like that in rapid fashion. Rob Collie (00:33:33): And got you to a place where you were looking at something that you wouldn't have even known to have asked for. And it was clear that you guys were really sharp, so it wasn't a failure of brain power. It was just a basic limitation of human capability. You need to see something that you asked for and then realize, this isn't good enough. There was one other thing that happened at the end of that, which was that you guys realized at that 10th iteration or whatever, which didn't take very long. That, "Oh my gosh, we're not even collecting the data that we need to answer the problem." Rob Collie (00:34:10): You probably don't remember this, but it was a really formative moment for me. Where, you guys realized that you needed to have an extra column of data that wasn't being reported by the add-ons. So you had to go back and modify the add-on code. So that in the future ad, when they called home and sort of reported on their health or whatever, their activity. That you'd have the raw data to do what you needed to do. And it was just crazy to me and how fast that happened in the course of those two days. We went from thinking you needed X to realizing you needed X to the 10th derivative. Rob Collie (00:34:45): And then even that going, "You know what? Not good enough." These ever rising standards, the ability to be greedy and truly chase some thing to the end of where it needs to go and watching it play out. And it just had never, ever, ever worked like that in traditional BI. In traditional BI, you don't even make it to derivative two, you just never get there. You get exhausted, you get worn out, there's just too much time, too much expense. And you certainly never get there with regular Excel either. Because, of all the elbow grease that has to go into every single thing that you do. Rob Collie (00:35:21): And I was just like, I left there even more in and I'd already chosen to bet my career on this stuff, right? But I left San Diego even more in. But of course, it didn't take with you guys, right? A week later, how long was it before you stopped thinking about Power Pivot? Kai Hankinson (00:35:41): We just couldn't do it without you in the room. I mean, it was just so hard for our company to adopt those early stages. And I don't know if it's because the product was a little bit immature, it's evolved quite a bit or we had a shitty teach. Rob Collie (00:36:00): You mean the first guy before I came out there? I agree. You totally did. Kai Hankinson (00:36:04): I don't know. I'm thinking about your second consulting engagement. It didn't take the first time, we're like... But you sold us on the revolution, you sold us on the vision. We wanted what you were selling. We just didn't pick it up on that first trip. So we brought you out for a second trip and it was a two day consulting engagement. And the first day happened to coincide with the day I was throwing a company party at my house. Rob Collie (00:36:32): Oh no. Hold on now, you've got this all wrong. You're talking about the third time that I came out to try to make it stick with you guys. There was already a second time- Kai Hankinson (00:36:43): Okay. Rob Collie (00:36:43): In between. You're talking about failure number three. Kai Hankinson (00:36:47): Okay. Rob Collie (00:36:48): The real problem with you guys and your adoption of this, was that you just had too many resources. You had too little time, which is common to everybody. You had too little time to learn something particularly new, right? If you had to learn Excel from scratch on the spot, it would be hard to take that as well. Takes years to get good at Excel. But the other thing is that you just had developers laying around, left and right. You just had too many technical developers that you could go to them and say, "Hey, I want you to write some code to take this data and turn it into a chart." Rob Collie (00:37:21): And it's like addiction in a way. It always seems like it's the cheapest move, to go ask the developer that you've already got to do something for you. And it is. At that moment, it is the short term cheapest move. But you're stuck in this relative minimum. You're just stuck in this gravity well. Kai Hankinson (00:37:39): Yeah. Rob Collie (00:37:39): Where, if you do take the time to escape it, you end up in a much different place in a much better place. But I just couldn't break you guys of that, we're surrounded by developers crack pipe. I didn't have the right treatment plan for you. So that was the second mission, right? I came out the second time and you had even more people there. And my job was to convince, not just you, but also the new guy. And so we did that, he wasn't really all that convinced. And then you still had the developers. Because, you guys just kept getting back on that horse. Rob Collie (00:38:11): You kept falling off and getting bruised and then you're like, "Oh we should call them out again." And I'm like, "Okay, great. Let's go. I always enjoy visiting you, let's do it." So yeah, the longtime blog readers have even seen a picture of the shame that Kai's about to describe. If you've been around reading the Power Pivot Pro blog from the beginning, I sneaked a picture in of this. So go ahead Kai, embarrass me. Kai Hankinson (00:38:38): I'm sure you had a lot of anxiety. This was your third attempt, you had failed miserably the first two times with your first client. We invited you out to my house with the whole company and I'm just going to leave it at, you drank more than anyone in my company. Rob Collie (00:38:57): Lifetime. Kai Hankinson (00:39:01): You weren't anxious at the end. But I ended up having to drive you home and then pick you up the next morning to come back for day two of your consulting engagement. Rob Collie (00:39:10): In my rental car, by the way, right? You picked me up the next morning in my rental car. Kai Hankinson (00:39:16): When I saw you that next morning, I understood that day two was off. You were just going to be physically present. Rob Collie (00:39:26): And you paid me anyway. Kai Hankinson (00:39:28): It was the right thing to do. Rob Collie (00:39:31): And you invited me both days actually that I was there, to your after or noon KPI meetings. Which again, I learned something you were paying me to learn. It was awesome, I really appreciate that. You guys are the best. You had like 20 charts that you looked at every single day at four o'clock in the afternoon. And it was the same 20 charts, every day and there were like 10 people in the room. Kai Hankinson (00:39:57): About 20, easily. Rob Collie (00:39:58): Okay, yeah. It was a big room. Kai Hankinson (00:40:00): Huge meeting. Rob Collie (00:40:01): And there was no interactivity, you couldn't drill down on a chart. That's why there were 20 charts. It's because you probably had started with five charts. Brad Miller (00:40:08): There were probably 50 times four. There were probably 50 slides with four on slides, probably 200 charts. Rob Collie (00:40:17): Right. You didn't start with 200 charts on the first day of the KPI. There's 10 charts and you're like, "Ah man, these don't answer the questions that we need to follow up." So clearly the answer is more charts. And this is how, by the way, traditional reports explode in traditional BI as well. Like, you got a question? You need a new report. Got a question? You need a new report. Organizations sometimes end up with tens of thousands of reports. Many of which have only been looked at once. Some of them have never been looked at, they were asked for built and never used. Brad Miller (00:40:48): Yeah. Rob Collie (00:40:49): And it's all super slow,.but you guys, I was really, really, really rapidly against this. This idea of you'd have 200 static charts. It was in some sense, offensive to me. Because, I was so big on and I still am... On the concept of interactivity in the ability to dynamically ask questions, as opposed to having to build all the charts to answer all the questions, which is a losing game. You never get there. At the same time though, it was a good meeting. I could not say to you that it was a bad meeting or that it would've been better. Because, it was so weird to me that it was the same I guess 200 charts on Thursday that it had been on Wednesday. Rob Collie (00:41:36): How much could change in your business in one day that you would look at the same charts? That the conversation was completely different both days. Kai Hankinson (00:41:44): Yeah. Rob Collie (00:41:44): It was super valuable. Kai Hankinson (00:41:46): Rich conversation every- Rob Collie (00:41:48): Yeah. Kai Hankinson (00:41:48): Every day at that meeting. I mean, it would just be like either the charts would be stimulating. The changes in the charts would be stimulating the conversation or we'd come to the meeting with business issues that we would then turn to the charts to help us think through. Rob Collie (00:42:04): Yeah. Kai Hankinson (00:42:05): And a quarter of that room was dedicated to supplying that product. That's how it went from five slides to 50 slides. Is like almost every meeting we would put in a request that we'd usually be looking at the next day. Rob Collie (00:42:20): Yeah. Kai Hankinson (00:42:21): Because, we had so much expensive manpower thrown at the problem. Rob Collie (00:42:25): I mean, you were in essence, an IT organization from top to bottom. And so when you have that many resources laying around at your disposal, you tend to take that path. But I did, I learned something from that as well. I came back from that experience, having seen the... I also learned, don't eat chocolate cake, birthday cake at the office and that'd be the only thing you eat all day and then go straight to the Cinco de Mayo party at the CEO's house. That's another lesson, don't get into land war in Asia, et cetera, et cetera. But I came back, I realized that I had been too much into the concept of interactivity. Rob Collie (00:43:06): I refined something about my stance about all of this. Which was that, any report you produce needs to be useful before you touch it. Before you interact with it, it needs to already be starting the conversation. Whereas before, I was being lazy essentially and I was saying, "Look, if the thing is interactive and it can answer any question, it can just sit there like a slug at the beginning of your relationship with it every day. Staring at you saying, 'Yeah, I know I'm telling you something that doesn't matter to you, but you can ask me the things you really care about.'" Rob Collie (00:43:40): And so it was almost like interactivity was required before you could derive anything from a lot of the stuff that I had been producing. That's much closer to where I am today, by default useful. I'm not as obsessed with interactivity, it's table stakes that you'd be able to drill down. But if you have to interact with it before it tells you anything at all, it's a bad design. And so again, you paid me, I drank your booze, I came home wiser and you guys didn't adopt it. Kai Hankinson (00:44:10): Yeah. I mean, you certainly helped convince us that we were on the right path to be using the big data, to answer our questions. And we were sold on the vision that you were high priesting. Rob Collie (00:44:21): You say 'we', but let's be clear. You weren't both as equally in, were you? One of you is a little bit more skeptical than the other, that's true, yes? Brad Miller (00:44:31): I think that's true, but I will give you some credit though. I will say that, you have two main parts to your business, right? And we talk about this a lot when we work on the search advertising for you guys. You have your training side of the business and you have your consulting side of the business. And so, when we kept bringing you back, it wasn't just for your fun personality. It was for the consulting side of the business. And so while we weren't yet sold on being trained on the tool, there's a whole side of your guys' business, where you do provide a lot of value just through consulting. Brad Miller (00:45:11): You guys are experts in your field and so when Kai and I are able to sit in a room with you and talk to you about data and BI for hours at a time, that is super helpful to us, tool or no tool. Rob Collie (00:45:25): I appreciate that, that truly actually really is flattering. I always felt like I was learning, interacting with you guys more than you were. And that's just sort of the nature of imposter syndrome, but you're right. That's another thing that's different; if you all met our company today, things would go differently. Even if you were in exactly the same spot that you were in, in 2010. Things would go differently today because at the time, I had a day job, right? I couldn't devote a whole lot of time to you. So it was just like, "Look, I'll come out and tell you some things and leave you some stuff. Rob Collie (00:45:55): And then you're on your own." That's how the company started. The first three years, I was still full time employed elsewhere. But now this is what we do. This is what we do all the time. And so we would have the time to dedicate a consultant or maybe even multiple consultants to helping you on an ongoing basis. Even with fractions of their time, it wills till be a lot more than what I could have allocated back then. And we would just do stuff for you now. Brad Miller (00:46:21): Yeah. Rob Collie (00:46:21): There'd be more continuity there and I think it would've gone a lot differently. Brad Miller (00:46:25): Yeah. Kai Hankinson (00:46:25): Was it next that you got accidentally added to Brad's bachelor party invitation? Rob Collie (00:46:30): Yeah. He's been really kind about never admitting that mistake. Kai Hankinson (00:46:33): And I'm pretty sure it was on purpose. But you were convinced it was an accident that you got invited, to Burning Man. Rob Collie (00:46:39): Honestly, the reason why I was like a Tinder box ready to explode on that fateful Cinco de Mayo, is just, it wasn't just a professional decision leaving Microsoft. I'd been through some really, really tough divorce and family stuff. And I was away from all of my friends and I wasn't in Seattle anymore, I was in Cleveland. Essentially by myself and hanging out with you guys every time you guys would bring me out there. Hanging out with you guys was the best thing that had happened to me in years, it was just great. I've told you guys this, but I would have dreams in which I was out there working with you guys, living out there, hanging out with you. That was how much, that's how strong the pull was. Rob Collie (00:47:24): Those were tough years. Out there hanging out in San Diego of all places. Leave Cleveland, get off the plane in San Diego and be like, "Oh my God." And then it just felt like living a completely different life. "Oh yeah, of course. I'm going to go to the CEO's house for Cinco de Mayo party. It's on the Pacific ocean?" "Yeah. It's on the Pacific ocean. We're just hanging out in the front yard on the Pacific ocean." "Hand me that margarita." Everything was predestined from that point forward. I mean, I was in need of an outlet and boy, did I find one. When I got the bachelor party invite, I mean, it was just too good to be true in a way, right? Rob Collie (00:48:07): Brad wants me to go a Burning Man with him? And there were so many people on this email. I remember it being, I just assumed that Brad has 1,000 friends, it probably wasn't that many. But I just remember it as being like 300 people on the email or something. And so it was so easy to fat finger that and include the wrong Rob or something. Brad Miller (00:48:32): I definitely did not include the wrong Rob. There were a lot of people on the email because I wasn't sure what the response was going to be when I had my bachelor party at Burning Man, because it was 2015. And so, I just invited everyone from my life, going back to high school that I really liked and really wanted to be there. Even if I hadn't seen the person in 10 years, I wanted to invite them. Because if they wanted to go, I wanted them to be there. So there were a lot on there, but they weren't 300. Rob Collie (00:49:04): Okay. And this is really something I appreciate about you, Brad; is just, you're really good at setting aside everything that doesn't matter. And just focusing on what would be the most human and fun thing? You don't get hung up on, "Oh, is it weird to invite these people, I haven't seen you in a while?" You just set that aside. That didn't get in your way. Brad Miller (00:49:28): Yeah. Rob Collie (00:49:28): And so then I did the responsible thing and not read your emails. As soon as it started to become real, I started to get terrified. Like, "Oh my God, I'm not Burning Man material." And you talked me into it. Kai Hankinson (00:49:41): What a trip. Rob Collie (00:49:43): Yeah. Brad Miller (00:49:44): [inaudible 00:49:44] with 18 of us going and only one had ever been before. He had only been one time with his 80 year old dad back in the day. Rob Collie (00:49:53): That was when we finally dispensed with the idea of, this isn't a professional client relationship. This is a friendship. Kai Hankinson (00:50:03): That saved us... Rob Collie (00:50:03): Well, client relationship this is a friendship Brad Miller (00:50:03): That saved us a lot of money. Rob Collie (00:50:05): It did. A lot of failure, you got out of the Facebook business. Facebook had something to do with that, right? You were riding their coattails. One day they turned around, and said, "Hey, get off my coattails." Brad Miller (00:50:19): Yeah. They did not like our business model. They did not like us covering their ads with our ads, that was coming to an end, but before we ended up selling that company, and we had a nice exit from that one. We had these 60, 100 million add-ons that had downloaded our software. We were looking for ways to pivot, we were looking for ways to expand, and so at some point, Kai said, "Brad, I'm giving you three or four resources. I'm sending you down to the basement for three months. You're going to do essentially a consulting project, and figure out where we're going next. You're not allowed to talk to anyone else in the company, you just of focus on this." We went down, we spent three or four months on it, and we came up with two ideas. Brad Miller (00:51:08): One was a data analytics company, because we just had so much data coming down from these add-ons that like, there was something to do with this data. We ended up exiting that company, and selling it to a company in New York. Then the other spinoff was what's today Agree Media, which is in the lead generation space, having so much data, one of the things we could do with that data is identify possible leads for various industries and verticals. That was the original seed of Agree. After we sold off the main company, the big company, Kai, and I bought out our other partners, and so now we're just owners in Agree, 50-50. Rob Collie (00:51:50): Yeah. That timeframe, when you were in the basement, I remember this because this is when I was out there, when I was contemplating, and in the final stages of contemplating leaving the startup I was at and turning P3 into its own distinct entity full time commit to it, that kind of thing. This was in like 2013. Does that sound about right? I just remember when I was out there pitching you guys on being an investor in my company, which in the end we decided we didn't need any investment, but you guys had such great success pitching your friends and family on investing in your companies. I just said, Hey, I'll return the favor. Agree Media. This is why we're here. What is it? We've taken an hour to get around to the crux of it. Rob Collie (00:52:39): When I talk about you guys in Agree Media, to the people who don't know you, I describe you as experts in data driven advertising. People who know the internet advertising space really better than anyone is how I queued it up. You say, it's lead generation. You didn't say, Hey, here's the idea. Let's get really good at advertising. Let's get really good at digital advertising. Let's get really good at digital marketing. That wasn't the goal. That's not what you came out of the basement with. Right. You came out of the basement with, we should be a lead gen business. Brad Miller (00:53:16): Yeah. Rob Collie (00:53:17): Tell us a little bit about the advertising operation. Brad Miller (00:53:21): Yeah. I mean the lead generation business is interesting because one example where it struck home with me was like, you go to various conferences for whatever industry you're in. Right? With the prior software development company that we were working at, that we owned, we'd go to these conferences, and it would be guys in their early twenties, and hoodies, and the whole Mark Zuckerberg look right. Everyone at the conference, the first lead generation conference we went to, the big one is called LeadsCon. We show up, and everyone's either in a suit or a sports coat with tie and shirts. We're like, what's going on here? They're like, we've never seen this before. I've never been to a conference, like where people aren't in hoodies. Brad Miller (00:54:10): It turns out that the lead generation business had been around long before the internet, right? I mean, guys were sending out flyers in the mail, and there was a bunch of other ways to advertise pre-internet, and lead generation business has been around forever. That's the Genesis of where most of those folks had come from. Whereas our background is much more finance, quantitative analysis, software development, artificial intelligence that kind of thing. Instead of coming at it from a marketing bent, we're coming at it from that data analysis AI bent, which has given us, I think, a competitive advantage in the industry. We use that primarily on our own in-house proprietary systems, as well as when we're hooking up with Google, Facebook, Instagram platforms. Rob Collie (00:55:03): I've never heard that part before. That's new to me. It reminded me of, have you seen Glengarry, Glen Ross? Brad Miller (00:55:10): Not in a while, but yeah. Rob Collie (00:55:11): They now disgraced Kevin Spacey, is holding like a box in his hand and says, "No, these aren't, those other leads, those crappy leads. These are the Glengarry leads. You're not worth it. I'm not going to even share them with you. Because you're so rookie." What a hardcore movie that was really. Yeah. I'm pretty sure that the Glengarry leads did not come from a digital system. I'm pretty sure, they came from some other shady thing. I have no idea what it would be, but he holds them like they're gold. Brad Miller (00:55:44): That's the other thing we love about the lead gen business is that other forms of advertising, whether it's television, billboards, print, whatever it was, it was like, there was no real way to quantify the success of those campaigns. Right? Most of where the money was made was like, you had a good salesman who was taking people out for lunch, and drinking martinis mad men style, and convincing them that their campaigns are working. They were rough numbers. You really couldn't see anything down to any level of detail, obviously, because it's all offline. In the lead gen business, it doesn't require much sales, sorry, Tim, our sales guy, I don't mean to throw you under the bus, but like Kai and I, that's not our forte. Right. In the lead gen business, if the numbers back out, and you're sending the quality, you will get the bigger order, and they will pay more. If the numbers don't back out, they won't. Taking out that salesy middle man was really a feeling to us. Rob Collie (00:56:50): This is a theme across all business, right. Is that in the world of offline lead gen where you just have a good set else, person, you can't lose. If you're the lead provider, if you're the lead consumer, you can absolutely lose. You'll never know it, but it's like, I got you some leads, and your business improved. Well that's me. I did that. Oh, I got you some leads, and your business didn't improve. Oh man. Imagine how much worse it would've been. If I hadn't gotten you, those leads he just got a story for everything. A lot of people are even though no one wants to talk about it or admit it, there's still a lot of very important decisions being made in the business world every day that are basically done like that. There's still a lot of gold rush opportunity everywhere. Brad Miller (00:57:35): Yeah, probably about, I don't know, two or three years ago for the first few years of the company, we were primarily Facebook in terms of our advertising. We were strictly social advertising, Facebook, and then Instagram, maybe two or three years ago, we realized that we had to diversify into Google search for two reasons. One for diversification, two for quality. Facebook is much higher in the funnel. You're hitting someone up in their newsfeed and they're not looking to go back to school at that moment, but something catches their eye, and then they might go check it out. Whereas, Google someone is searching. I want to go back and get my cosmetology license. There's a ton more intent there, much lower in the funnel. The quality of those leads are much higher. We're primarily in the for-profit EDU space. Education vertical, so our customers are names, University of Phoenix, DeVry Art Institute, those kind of schools. They were looking higher quality, and so we transitioned to Google. Now we're about, I'd say 20% Facebook, Instagram, and about 80% Google. We used to be Google AdWords. It's now Google Ads. Rob Collie (00:58:52): You guys have put Sally Struthers out of a job. Remember those old ads on TV where she would go, "Do you want to make money? Sure. We all do." Then she'd run through all of these things that, there was always gun repair, gun repair was always one of the options that was on that she was advertising. You talk about a primitive model, right? How did they ever, that these ads worked, but boy, we saw them for years. Didn't we? Brad Miller (00:59:17): Yeah. We gave you what, like your fifth crack at us, at Agree. We like, come on, Rob, come sell us again on this vision for Power BI. Rob Collie (00:59:26): That's right. I mean, actually, even at Agree, there were multiple failed attempts. We'd given up on the notion of it being a financial transaction, right. It wasn't how it worked. I'd just come out there, and hang out and we'd spend some time and chit chat. We'd both learn that kind of thing. I mean, even once we get to Agree, there's at least two or three totally aborted attempts at trying to get this stuff to take. Then Brad goes off and he finds self this thing called Periscope it's just traditional reporting with a slicker new look on it. You guys brought me out. I remember saying, "Hey, can we replace these Periscope reports with Power BI?" I'm looking at them in like on the average dashboard, like three of the modules on the dashboard we're reporting, oh, sorry. We failed to load the data in time we timed out. Rob Collie (01:00:15): I'm Like, I could probably replace these, but I'm might have a hard time with replicating those performance failures. My stuff's just going to work. All the stuff's just going to return the answers. I suppose we could roll some dice or something behind the scenes, but because you guys now are, are using Power BI very aggressively. Brad Miller (01:00:32): Every day. Rob Collie (01:00:33): Even here I don't get to claim credit for it. It was almost, we had to maintain the joke or something. Like if you guys were going to start using it, we would need someone else to have helped you so that we can say that I still was O for 10 with you guys. Ultimately I think Austin was really crucial and really nudging plus the fact that the matter is, is that just because I was first. I was, I was like the first professional in the world dedicated to what is now called Power BI. Rob Collie (01:01:04): I was the only person that was like saying, this is the only thing I do. My only source of money. I was first. That doesn't mean that would never translate to being best. I was pretty good. I was pretty good, but people we hire are better. I am absolutely the worst at these tools of the people at my company. I think that's great, takes a lot of pressure off me. Austin was definitely one of those. He's better at it. He was able to make progress with you quickly. His goal was to build stuff with you, which I never had time for. Brad Miller (01:01:37): Yeah. Austin was key from your side, and then Sean, our head of BI slash CFO was key from our side, so putting those two together, really helped us turn the corner. Rob Collie (01:01:52): By the way, I love that head of BI slash CFO. That's the future, business intelligence owned on the business side or at least driven from that side, supported from the IT side. For sure. You guys don't have a huge footprint of people, but that's your model you're already running on the model that the world is inexorably heading for. You're seeing so many more of these hybrid titles that you just would never have seen before. Rob Collie (01:02:23): I think that's really exciting, and yeah, Sean has really, really, really, really taken to this stuff. What are the sorts of things that you look at in Power BI? Because you know, Hey, I was going to ask the juicy softball question. I mean, but seriously doesn't Google just provide you with all of the reporting that you need. I mean, AdWords has got a reporting portal. Why do you need Power BI? Kai Hankinson (01:02:45): What don't we look at in Power BI these days? It's really got to the point where we almost exclusively, answer our questions with Power BI. What we found is that, sometimes you'll have these situations where a DBA could bang out a slightly quicker answer with a sequel query, but we've learned the hard way that, if you just put the little bit of extra time, and it's not much, but just a little bit of extra effort it into answer that question with a Power BI model, it will just pay dividends. Kai Hankinson (01:03:20): You will be able to keep asking it questions, and you'll be able to ask the next version of the question, and they get easier and easier to build each time. We are using Power BI for all our BI at this point, and we love it. It's like we're finally living the revolution you promised us back in 2010, it took us a long time to get there. Kai Hankinson (01:03:46): One of the reasons the AdWords portal isn't sufficient is that, we need to stitch that data together with what happens on our side, what happens those ad clicks once they get to our website, which ones drop off, where do they drop off? Which ones become leads, where do they become leads in? Which schools are they matched to? Even then we get the offline data that we have to collect and bring back in and import into Power BI, which is, well, what happened after we sold the lead, which leads applied at the school, which leads enrolled in school, which leads actually started. Kai Hankinson (01:04:21): We get data all the way down to, they pass their first class even. So we need to stitch that full conversion funnel if you will, together. And AdWords only has some of it. Rob Collie (01:04:32): That is the thing so often is that it's so rare that there's a report that's useful that truly helps you. That is sourced from only one system. Anything valuable is almost always going to span at least two systems. You're talking about just in that quick survey, there were at least three systems and one of the systems doesn't even belong to you. Rob Collie (01:04:59): It's your customer's systems that are reporting back to you about like, well, you sold us a hundred Glengarry leads, but they actually felt more like Glen Ross or vice versa, right? So you get a quality feedback on a longer time scale. But, but you need to use that to steer because if you deliver poor quality over time, your customers are going to stop buying from you. Rob Collie (01:05:22): And so if you can't span silos with your BI, if you can't span silos with your reporting, you're all always looking at one part of an elephant going like, "Is this an elephant? I don't know. It's gray. Could be a rhino, no idea.@. Rob Collie (01:05:39): And so I'm really happy that you guys have gotten to that. And you and you operated at a pretty impressive scale. You guys can't even use certain commonly available integration tools like stitch for instance, because they have some cap that I'm sure when they set that cap, the engineering team said to themselves, no one's ever going to need more than this. They said it to be 10 times what we think anyone would ever possibly humanly need, right? It's like 10,000 ad groups or something like that. Is there limitation and you guys are like, please 10,000 ad groups. Kai Hankinson (01:06:19): We have some proprietary technologies that we developed to leverage the AdWords platform. It's driven us to have to come up with some elaborate structures over at AdWords that involve hundreds of thousands of ad groups and campaigns keywords. And in order for them to be able to support the kind of integration with our technologies that are... we live by a philosophy of AdWords whereby we think that the smart bidding technology artificial intelligence is that Google has built is some of the smartest AI in the world. Kai Hankinson (01:06:57): I mean, it is just so, so powerful at doing what we need it to do, which is to go out and bid the right amount on the ads that we run. Bid the right amount for the right user at the right time on the right device, all those things. And we believe wholeheartedly that if you're going to succeed, you have to be leveraging their smarts to the max. Kai Hankinson (01:07:20): But then there's also a component of, there are things that we know, not many, but there are things we know that Google doesn't know about our business. We know that that customer just canceled this other customer, close their calls there for labor day. We know we just got a double size order on our Chicago school. Kai Hankinson (01:07:40): We've set up the mechanisms whereby Google's Smart Bidding will learn those things over time, but that's just it, it's going to take time. So we will elaborate technologies that us to leverage their smarts while still injecting ours. And that's a tricky thing to do because it's easy to clobbered their smarts and be like, we're switching Google onto CPC bidding instead of Smart [TiBO 01:08:03] as bidding or whatnot. Kai Hankinson (01:08:05): But now you've just clobbered their smarts and you're not leveraging the best of both. So, that's been one of our driving philosophies there. That's driven the elaborate structures that have taken us beyond some of the caps in many of the products out there, like stitch being one of them, Stitch data. Rob Collie (01:08:20): I even love the names you guys have come up with for some of your proprietary stuff. It is just as a terminology guy. I got mad respect for things like probabilistic pixel and denominator, even Trillions with capital T. I mean, it's hot. You guys have got good naming game Brad Miller (01:08:41): [Redon 01:08:41], don't forget about Redon. Rob Collie (01:08:43): Ooh, I never heard about Redon. You we'll have to talk about that later. I need to know is PG 13, is it Patrick Swayze? Is it, I mean, just let's leave that. But, one of the things that's impresses me about you guys and there's many things obviously, but you guys are just so into the weeds, into the Twilight zone of all of this. Your intuitions and everything. You've spent so many years building them up and learning the way in building all this technology and proprietary techniques and everything for optimizing your ads, you still keep hiring experimentally. Rob Collie (01:09:19): You keep hiring third party ad agencies, ad optimizing services to see if they can do better than you. I mean, so many times when I visited you, like you've been on the phone and interviewing like just some... Oh this is this other firm we're thinking about trying out, just the humility and also just the stamina that it takes to keep doing that. Rob Collie (01:09:46): And it's worth noting that you're still doing that. You're still trying it. If you could get out of the business of managing ads and just be like the leads clearing house, you probably prefer that in a way, no one's ever come along that's that's been able to do better than you. And so you just keep getting better at your own stuff. Brad Miller (01:10:05): Yeah. I don't know if we would prefer to get out of it. I mean people tend to do what they love. If I was kind of retired and really rich and I wanted to spend a few hours a day doing something like this is probably what I choose to do, right. Is to dive into some huge, powerful platform and start pivoting stuff and figuring out. It's my version of a, that's the way I would spend my hobby time. It's like, but I get to do it at work. Kai Hankinson (01:10:37): I love my job. I mean, one of the greatest things about working at Agree is we're only a team of 12 right now, but we've pulled everyone forward from a prior company. In some cases we pulled them forward from two prior companies. So it's the best of the best that we get to work with. Kai Hankinson (01:10:56): We've probably employed like 600 people together over the years, or maybe more. And these are some of the 12 best. We get to spend all day every day, solving fun problems with good people. It's a dream come true. I've learned over the years with our startups that we both love and do best at a specific type of business challenge. We do best when it's like a high volume of transactions with a quick feedback loop because those are the pro that you can just really work the data to kill. Kai Hankinson (01:11:31): It's like, I need that quick feedback loop or else... is just going to be such a slow cycle of improvement. And I need the high volume of transactions to drown out the luck factor. I learned that at the poker company, I just couldn't to believe I used to love table poker prior to running that poker company. And it almost ruined table poker for me, because we would see these runs, these bad luck runs that would just blow your mind. We would have different versions of our software. So we'd be like running version 53.2 for a week. Kai Hankinson (01:12:07): And kicking, come Monday, all of a sudden what happens? We're losing money. Why are we losing money? We're just spending all day diving to the data. This doesn't make sense. How can we be losing money? Kai Hankinson (01:12:18): We're running the same version that crushed last week, nothing's changed. We're checking all these things. Tuesday get killed again. This doesn't make sense. We made money every day. Wednesday, all of a sudden, after 300,000 hands, our luck turns. It goes back to the whole way of just churning out the profit day after day. Luck is such a factor and you need that high volume of transactions to drown it out. If my success depends on whether or not this one big commercial sale closes this quarter, Ugh. I just don't like sleeping with all that. Brad Miller (01:12:54): To answer your question from earlier, we're currently bidding on 17 million keywords in 2 million ad groups. Rob Collie (01:13:01): Wow. Yeah. 10,000 is cute. Isn't it? It's so cute. Yeah. If you let us have 200 accounts, merge them somehow we can use your service. So let's see if we can check the check boxes. First of all, you guys are like, "Ah, I don't want to be in any sort of hourly business," like that professional services business. So that's something you want to avoid. You want tens of thousands of trials a day and you also need almost a near immediate feedback. You have now described the exact antithesis of our business at P3. Brad Miller (01:13:44): I got one more for you that'll fit in that category too. Rob Collie (01:13:47): All right. All right. Bring it. Brad Miller (01:13:49): When we developed the poker software, we could have licensed it to someone else. Right. And let them play poker with our software, but we decided if it's that good and we're building it, we might as well use it to trade ourselves. Brad Miller (01:14:04): And so up until now, and I know we're working a little bit together with you, but up until now, we're like, "Look, if we're that good at AdWords and Facebook and digital marketing, let's just use it ourselves as opposed to doing it for someone else." Rob Collie (01:14:16): That's right. Yeah. We've been sort of inching our way towards as I say it finding a reason to risk our friendship. That's what you do. When you get into business together. You want to ruin a personal relationship. You really want to ruin it like permanently. I suggest going into business together. And so like moths to flame, we've been slowly inching towards this really slowly, but at least with our eyes open. Rob Collie (01:14:42): So we've been collaborating, we've been providing Agree with additional Power BI support and you all have been helping us by architecting and advising, et cetera, et cetera on our advertising campaigns, which is somewhat unusual for professional services firm. Kai Hankinson (01:15:00): Who did you send to help Agree? Of course... Kai Hankinson (01:15:03): This is firm. Rob Collie (01:15:03): Who did you send to help agree? Of course, your woofer. Kai Hankinson (01:15:05): That's right. We sent Paul. We're like, "Okay, we've got an in here. These are two guys that know how to dumpster dive in Hawaii." Rob Collie (01:15:15): [inaudible 01:15:15] has been great. He's really taken us to the next level. Kai Hankinson (01:15:19): Well, like you said, you've hired 600 people over the years and you're down to like 12, that's essentially like our interview pass rate as well. We're like in the 1%-2% range. Rob Collie (01:15:31): Wow. And so, that's that similarly filtered group. We all say it, we're the only office we would ever go back to. And we're all over the country. With a 1% pass rate, you can't hire in one spot. You can't say, "Oh, it's got to be in Indy, wherever I happen to be living at the moment." People all over the country, full-time employees, no office. And if there were an office with all these people in it we'd actually want to go to it. But yeah, we've been saying, "Hey, let's take your advertising expertise and let's apply it, both to help the P3 business but also as a completely different test case than the industry that you've been in." Rob Collie (01:16:13): Let try it with when there isn't the 17 million keyword volume. There isn't that massive volume. Let's try it. When there isn't instant feedback. We're not running an e-commerce business. It's ultimately a relationship business. We have this thing at our company. We like talk about how people say nobody believed in us. Nobody gave us a chance. I don't know how many times I've been told you can't run a professional services business this way. You can't start relationships over the internet, but we can because once you try us you understand that there's nothing like us. Rob Collie (01:16:50): How do we get people to try us? How do we get more people to try us? And it's been very effective for us, for our business model. It's not been easy. And the whole time we were talking about all these hard lessons you had to learn about bad luck, and needing volume, and all of that. They're really echoing in my head because, I think Brad's favorite way to start a sentence with me is something to the effect of, "Well, you'd think that would be a good idea, but..." Right? This is what I do, I come up with ideas or suggestions and then, way more often than a coin flip would indicate, my ideas are bad. Rob Collie (01:17:28): And so, because that subject matter expertise that you... You're not just looking at spreadsheets, or Power BI reports, and turning knobs like robots. You're having to blend this with, what's basically, a decade and a half of experience being in this digital marketing world. This digital world. It's hard. It is hard. And I know that I take a toll on Brad with my, "Oh, come on." Right? Like, " Shouldn't we do X, Y, Z?" And he's just so exasperated with me. Brad Miller (01:18:05): Yeah. What we spend most of our day doing, or at least what I spend most of my day doing, is trying to figure out what's going on in the black box over at Google. Those guys have built something that's essentially taking over the world. The robots are coming. No one really knows. No one knows what they've built over there. It's taken hundreds of engineers and a bunch of marketing people to write up the documents. But we've done several trips over there and tried to track down someone that has a thorough understanding of what's going on under that hood. And the AI is just, it's taking over. So it's kind of figuring out what it's doing and trying to play nice with it. Rob Collie (01:18:53): I read recently there was like some AI that was able to look at a scan of like a retina or something. Was this something you guys told me about? You look at your retina or something and tell you... It was something ridiculous, it could tell you whether or not you had cancer or something. Kai Hankinson (01:19:10): Gender? Rob Collie (01:19:11): Maybe it was gender. But the point is the human beings looking at it, even though it was deadly accurate, no one could understand what the AI was seeing. Kai Hankinson (01:19:19): Yeah. Rob Collie (01:19:20): The human beings that built it don't know how it's doing it. And this is where Skynet comes from. Right? One day Google becomes self-aware, the AdWords platform, and it starts turning the entire earth into a giant conversion funnel. Then we're all paperclips, all the way down. Brad Miller (01:19:43): It's interesting to note the differences between Google and Facebook. And I don't know if we have time or interest. Rob Collie (01:19:49): Sure. Why not? You do operate on both platforms and they have similarities, but they're of course, very different. Probably more different than similar. What does that end up looking like for you? Brad Miller (01:20:01): It looks like a lot more work on Google. We'd probably spend 95% of our time on Google and 5% of our time on Facebook. Or even less. And that's just based on the design of the two products. They're both so sophisticated and their AI is so advanced, but yet they've implemented in a different way that I would say probably 10 years ago, 5 to 10 years ago, marketing was a very different beast. It was all about, "Okay. I got to figure out who my demographic is. I want to target males 25 to 33, with incomes above X, that live in these zip codes." And whoever could identify their demographics correctly and bid accordingly that was the name of the game that was who won. Now, especially on Facebook, there's none of that anymore. Some of the old school agencies are still doing it that way, just because that's all they know and that's what they've grown up with, but it really doesn't make any sense. Brad Miller (01:21:02): So like at Facebook, what we do now is we advertise, someone clicks on our ads, they come through our flow, we get a conversion, we then send back as much information about that conversion as we can through Facebook. Facebook then goes and figures out who to target using their algorithms and their AI. They then figure out who is most similar to that person that just converted for me. So I have no idea, even, who they're going after and no one at Facebook does either, but based on all the hundreds, thousands, millions of different data points that they analyze, they're able to better figure out who to serve that ad to than I would ever be able to just going after males 25 to 32, when incomes over X. So a lot of what we do at Facebook is just content creation, coming up with new ads, different ad copy, that kind of thing. And then you just kind of throw it in and let Facebook work magic. Kai Hankinson (01:22:08): Can I jump in, there's something about this Facebook thing... Another thing I've learned from you guys that sort of stuck with me, is this notion that if you're going to try something like a new campaign or something like that, and you create 10 different ads that are just 10 different candidates to try to achieve the same thing. You just try 10 different content strategies. And then you play this game with yourself of predicting ahead of time, rank the 10, how well are they going to perform? Brad Miller (01:22:33): And your ability to predict which ones are going to perform or is almost worse than random. Kai Hankinson (01:22:38): Oh yeah. Brad Miller (01:22:38): The end result is closer to the inverse of your guess than it is to your original guess. Kai Hankinson (01:22:45): Yeah. Brad Miller (01:22:45): And just really shows how little we actually know and how little you can know. And the value of experimentation. And, as you said before, the value of letting these massive overlord AIs do what they're good at, which no one understands. Rob Collie (01:23:05): Yeah. The whole notion of AB testing, for example. If Facebook becomes irrelevant because, well, a lot of times where people get tripped up is, let's just say, I'm testing two ads... Well, let me give you a different example. So one of the programs that we advertise for is cosmetology. It's 90% female. And then another one is pick a mechanical engineering, or something like that, it's predominantly male. Facebook, without doing any demographic targeting on our end, Facebook will figure out to serve that cosmetology ad to a female and that mechanical engineering ad to a male within tens of impressions. You'll spend a few dollars and Facebook will already have figured that out. And if, you can imagine, that's kind of an AB test and you were running that you might get a scenario where the cosmetology ad is doing really well. You're showing it to 80% of the people and it's performing well. Rob Collie (01:24:10): The other one is profitable, but it's just not performing as well as the cosmetology ad. So typically, what people would do is they'd call the cosmetology ad a winner and turn off the other one, which is exactly the opposite of what you want to do. Facebook has figured out the subset of people, the minority of people that mechanical engineering ad is best suited for. And that's why you want to leave it running, to show those folks that ad, even though it looks like it lost to the cosmetology ad. Kai Hankinson (01:24:41): Yeah. And this is a very difficult thing to internalize. It's a very difficult thing to build into your intuition. And I can speak to this from my own experience. How many times have I come to you and said, "Hey, this thing we're doing is producing a better return than this thing we're doing. Let's turn off the thing that's doing less well." Brad Miller (01:25:03): Yeah. Kai Hankinson (01:25:04): And pour more money into the thing that's doing better. Brad Miller (01:25:07): Yeah. And the one caveat to that is if you are budget constrained and you can only spend a certain amount of budget, by all means, send it to the better one. But assuming you're perfectly fine spending as long as the spend is profitable and you don't have those budget constraints, then you want to leave both going. Kai Hankinson (01:25:27): Yeah. If you live in California and you work in an internet startup, you're not constrained by budget. The rest of us, out in the middle of the country, little more constrained by reality. Budget constrained but you guys don't worry about budget that's- Brad Miller (01:25:43): Well, I mean, if it's profitable spend, that's the last thing you want to be constrained by. It kills us every time we have a profitable opportunity that we can't throw money at. Kai Hankinson (01:25:54): By the way, folks, you might as well just know Brad never lets me turn anything off. If it's the equivalent of setting a pile of money on fire in the backyard, Brad's like, "Uh-uh, no, keep going. It only gets better from here." Brad Miller (01:26:18): That's how you make a million ad groups. Rob Collie (01:26:18): Yeah. So, that's the Facebook side. So, it's very much "Okay, come up with some new ads." But other than that, it's set it and forget it and let Facebook do its thing. Google, on the other hand, has a million knobs that you could literally spend a lifetime turning. And so, while it's AI is also super powerful, it's just not implemented in the same way that Facebook is because that person is giving you that intent. That person is giving you the signal of saying, "I am looking for cosmetology classes." Whereas you're not getting that signal from the Facebook user. Kai Hankinson (01:26:57): Intent makes it more efficient in a lot of ways. But now, like you say, it's so much more tuneable. Even just the choice of keywords, search strings that should trigger your ad versus not. Which of these actually signals the intent to go back to school as a search string? Probably doesn't mean I want to go to cosmetology school. Probably means what are the back to school sales for my kids and things like that. But how do you know? I don't know. You've got to try it. Brad Miller (01:27:32): Yep. Exactly. Google is not going to get... You're not going to see any statistical significance at Google. that's one of the difficulties, in that if you have 17 million keywords a tiny fraction of those are going to be spending at the levels where you're going to get any kind of statistical significance to say that that is profitable or not profitable, which is the challenge. Rob Collie (01:27:56): Yeah. And so, you've determined that casting a wider net is a good thing. The novice approach to this would be to come in and say, look, "I know, naively, hubristically, I know the five types of searches that people are going to run that I want to advertise on. Going back to school, getting reeducated or something like that. If you think about it naively, you'd think that the number of keywords, essentially, is closer to five than 17 million. I think that's something really, really interesting about the way you operate, that you can even get to 17 million. Brad Miller (01:28:34): Yeah. Rob Collie (01:28:35): But also that there's ROI in doing so. Brad Miller (01:28:38): Yeah. Rob Collie (01:28:38): You guys wouldn't do it, if it wasn't ROI positive, Brad Miller (01:28:41): That's absolutely correct. And also we're forced in doing that because in your example, let's just say round numbers. Let's just say we get paid $30 for every lead that we generate. Some of the best keywords in our industry, things like online colleges. Okay? You can't get at the top of the Google search for less than $50 a click. So if I bought a click for $50 and I'm only getting $30 on a conversion that math is just not going to back out. So the only way to compete is in the long tail. And so, we literally can't compete at a short tail. Rob Collie (01:29:20): And how do you do that? Without giving away any secret sauce, how do you get into that long tail business? Brad Miller (01:29:28): Well, the key at Google is making sure that your ad copy, your search term or keyword, and your landing page are all in alignment. The most important thing for Google is that their users to have a quality experience, they want their ads to be as much like an organic result as possible. So they have what's called a quality score and they will determine your quality score for each of your keywords based on how closely aligned your ad copy, your keyword, and your landing page are. And they will penalize you heavily when those are not aligned. And then that gets fed into you how much you're charged per click. Brad Miller (01:30:13): So it is their way of basically incentivizing that alignment. And so if you're able to take a long tail keyword or search term and come up with finally tuned ad copy and a landing page that aligns perfectly, you're going to get rewarded for that. The trick is how do you do that 17 million times? So it's a combination. And going full circle, taking us back to the poker business, it's a combination of what can I do, programmatically, what can I use the computer for, and what can I use the human for? And so it's figuring out how to draw the line and have the balance between the two where you can get scale, and yet at the same time to get that bump in quality score. Kai Hankinson (01:31:01): Google's investment in the quality of the searcher's experience is enormous to their credit. The quality score varies from one to 10 and they multiply your bid, when it gets inserted into the auction, by that score. So somebody with a quality score of one is getting their bid multiplied by one and someone with a quality score of ten's getting their bid multiplied by 10. Now that means that Google's... Imagine that if we go up against someone that's a one and we're a 10. Let's say we bid a dime a click, or something, and we end up paying a dollar and we win it. And they bid 95 cents and lost. Well, look at what Google just did. Google just sold something for a dime that they could have sold for 95 cents. And that's an extreme example, but that's how wide the quality is score... Kai Hankinson (01:31:53): That's how massive the impact is. And I think that's... Plus quality score is a black box, much like the search engine algorithms because they don't want anyone to be able to game it, but that's why I think it is really important to not take shortcuts and just try to build the entire business and website experience, advertising experience around giving users what they actually want, because that's what Google wants. Rob Collie (01:32:20): It's almost... You listen to it that way, they're willing to give up almost like a 10X revenue opportunity in order to provide a better experience for their users. It almost makes them sound egalitarian, almost like they have our best interests at heart. You could really talk yourself into that, couldn't you? Kai Hankinson (01:32:38): They have their long term best interests at heart. Rob Collie (01:32:43): Honestly, I think that's the best we can hope for, really, from anything is everyone having a long term view. That's probably the closest we're going to get to living at peace with each other is if everyone had a long timeframe. I don't care about what happens to the quality of the air. I'm not going to be around a hundred years from now when it's really bad or whatever. Eventually everything comes around to affect everybody. Hey, I'll take a company, that's got its long term interest and view really clearly at the front of its algorithms over the others. Brad Miller (01:33:21): Yeah. Makes sense. Rob Collie (01:33:22): I just talked myself into Google as a good guy. What do you know? You were here when it happened. To get a little commercial for a moment, because we don't run ads on this podcast so we might as well advertise for ourselves. Rob Collie (01:33:35): So one of the things that we're inching closer towards is combining. We'd be like Hank and Miller, P3 joint offering, bringing the power of our consulting team and our analytics expertise and our ability to span those silos, to provide the kinds of feedback. With you guys, that have basically become like cyborgs that are merged with these advertising platforms, and all the tools that you have behind the scenes as well. Coming soon, whatever soon is. This is one of those things, we're not in any huge hurry. We're going to do this when we're ready, not sooner, but when we're ready. We all believe in it. We're all proud of it. We're going to start offering our joint services to others, help them navigate this world of advertising and the same way that you guys have learned to. Rob Collie (01:34:28): I think that's going to be a really valuable offering for the world. If you look on the Power BI templates gallery, there's an ad words template. It's just so cute. That's not going to solve your problem. It's not even going to come close to solving your problem. It ignores all of your own data on your side, like the things that are happening on your end that the ad platform can't see. It ignores all of this incredibly important tribal knowledge that you've developed over the years like, again, all these counterintuitive things. This ability to go after the long tail, which is like even if you see the opportunity, which most people wouldn't... Even if you see it, it's easier said than done. Kai Hankinson (01:35:11): Yeah. Rob Collie (01:35:11): I'm really excited about it. And, like I said, any opportunity that we have to really hang our friendship out over a cliff, put money in professional fortune stake we got to do that. It's gone so well for so long. Brad Miller (01:35:30): It's time. Rob Collie (01:35:34): Yeah. In closing, when's the next email that's encouraging me to go to Burning Man? I've added a recurring nightmare to my palette, which is that I'm packing for Burning Man and I don't have all my stuff and I got to show up and it is... It takes a lot to add a recurring nightmare to your pantheon. And I've got one now. So how long do I have before we start talking about this again? And I have to start getting terrified and excited at the same time? Brad Miller (01:36:04): Well, Kai and I have been four times now. Our CTO has been, I believe, 17 or 18 times. They canceled it this year, obviously, because of COVID. So if they bring it back next year it is going to be quite the event. It's also my wife's 40th and she loves it more than anything else. So if it does indeed happen next year, we'll definitely be there and- Kai Hankinson (01:36:30): See you at the Thunder Dome at sunset. Brad Miller (01:36:32): Yeah. Rob Collie (01:36:33): We've got some things to settle in the dome, don't we, Kai? Yeah. You're right. Them canceling this year is like the whole Burner community has just been like coiling a spring- Kai Hankinson (01:36:48): Yeah. Rob Collie (01:36:48): For an additional year. Kai Hankinson (01:36:50): Yeah. Rob Collie (01:36:50): Maybe next year is a good one to avoid. I don't know, Brad Miller (01:36:55): Probably, but there's no way I'm getting out of it. So I'll be dragging you guys with me. Rob Collie (01:37:00): Like you say, most guys, when they're the wife goes out of town, that's the time to party. When your wife goes out of town it's when you get a rest. Brad Miller (01:37:07): That's right. Rob Collie (01:37:11): That's a keeper. So, Hey guys, we talked for a long time. I really appreciate you guys devoting so much of your valuable time to this and I hope the listeners do as well. So thanks again. Brad Miller (01:37:26): Thanks for having us. Kai Hankinson (01:37:26): Love you, Rob, take care. Rob Collie (01:37:26): Aw. Announcer (01:37:26): Thanks for listening to the Raw Data By P3 podcast. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Interested in becoming a guest on the show. Email Luke P- L-U-K-E-P@powerpro.com. Have a day to day!

Chris Finlan is a Principal Program Manager on the Microsoft Power BI team, the creator of the popular and incredibly adorable mascot Paginated Report Bear, and a huge Philadelphia sports fan. We try very hard not to hold that last item against him! He's done it all, from the product team to pre-sales, technical training, database administration and sales operations. Tom is off this week, so P3 Senior Principal Consultant Justin Mannhardt steps in to fill the void. The guys cover quite a bit, including: Working in sales vs engineering Power BI, Paginated reports and SSRS; What are the differences? How to make friends after a certain age Seattle area COVID reactions Can someone cheat at Scrabble and still lose? Ignite announcements, Premium Gen 2 and the premium per user license, the differences between Pro and Premium options, and Rob has some fun with Chris about some pricing

Quick, everyone get your act together, we've got a bigwig coming through: Joe Phelan, former CEO of DHL Global Mail and Swissport North America, and now (among other things) Chief Strategy Adviser at P3. We get his valuable C-suite perspective on a number of topics, including: What is the typical "data diet" for a CEO these days? The tension between good summaries and robust drilldown capability Is the C-suite truly more "in" on data today than previously, or is it all just lip service? Business and IT relations in an increasingly democratized data environment Splicing across siloes, and Power BI's ability to circumvent the need for data warehousing How COVID's new normal may very well be here to stay IoT and automation, and how they drive the need for even more BI The parallels between the FP&A mission and the BI mission Robustness versus efficiency in the modern supply chain How "data people" can best contribute value to leadership – and get recognized for it Links: Joe Phelan in Supply And Demand Chain Executive Magazine Episode Transcript: Rob Collie (00:00:00): Hello, data people. For our third episode, we get a glimpse into the inner workings of the C-suite because we brought in a ringer. Joe Phelan has held the titles of CEO, COO and even CMO at some very sizable corporations, organizations with names that you'll recognize, and even if you don't, you've been a customer of them whether you know it or not. So he is not one of those self-proclaimed CEOs on LinkedIn. He's not a CEO in the same sense that I am a CEO. He's a real CEO, the kind of person we think about when someone uses that term. He sits on a number of boards today, and we're fortunate to have him as our chief strategy advisor here at P3. He advises me on growing our business and he helps us with some of our strategic accounts. And of course, he's here to talk about data. Rob Collie (00:00:45): Before semi retiring last year, Joe crossed paths with Power BI and he recognized it as a turning point. As you'll hear, that's just not me hyping it up. It's the truth. And some familiar themes definitely emerge in the conversation. I think you'll find those parts totally validating. And it definitely builds confidence to hear that sort of thing from someone like him. But we also made sure to get his advice on how to do things like how to speak data to leadership, how to provide value, how to get noticed. We also talk about supply chain in particular. We talk about FP&A. We talk about the rise of automation and the internet of things and how that drives the need for even more BI and a bunch of other stuff that I hope you find valuable. So let's get to it and you know how it goes. Let's start with that intro music. Announcer (00:01:33): Ladies and gentlemen, may I have your attention please? Announcer (00:01:36): This is the Raw Data by P3 podcast, with your host Rob Collie and your co-host Thomas LaRock. Find out what the experts at P3 can do for your business, go to powerpivotpro.com. Raw Data by P3 is data with the human element. Rob Collie (00:01:54): All right. Welcome. Really excited to have you here, Joe. Joe Phelan (00:01:58): Yeah, I'm delighted to be here, Rob. Thanks for having me. Rob Collie (00:02:02): As one of the intro episodes of this podcast, you're a ringer. We're bringing in a heavy. We don't mess around here on the Raw Data by P3 podcast. Let's speak to that, your ringerhood to get us started. You've had a pretty interesting career. You've been in some very, very, very interesting places. Why don't you give us a little bit of a rundown of where you're coming from? Joe Phelan (00:02:24): Sure. And I'll start off with you're much too kind, Rob, but to tell you a little bit about what I've done, I've been in business for about 40 years, in transportation, logistics and business services. And 25 years was with American Airlines in various capacities, including global VP of cargo operations and managing director of Europe where I was based in London. I was CEO for DHL GLOBALMAIL with operations in 200 plus countries based in Bonn, Germany. That was a great experience. And CEO for Sunbelt Rentals, the second largest equipment rental company based in the US with over 400 outlets in North America, primarily construction equipment rental, and mom-and-pop do it yourself type rental. Joe Phelan (00:03:21): I retired mid 2019 as the global COO for Swissport International. I'll say I kind of retired because I'm doing some other things now as well. And we provided airport ground handling services for over 800 airlines around the world and airports. And I currently sit on a few boards and I sit as a strategic advisor for PowerPivotPro. So I'm happy to be here with you. Rob Collie (00:03:54): All right. First of all, you and I, I mean, we have the same title. You've been CEO a lot places. I'm CEO here at P3. We're basically the same person, right? It's just that you just happen to have had that title at companies that we all know, we all recognize. I mean, even Swissport, most people probably don't recognize Swissport, but you don't really go through an airport without Swissport being involved behind the scenes, isn't that right? Joe Phelan (00:04:23): That's correct. I mean, a lot of the employees at many of the large airports are Swissport employees. You just don't know it because they're wearing the airlines uniforms. Rob Collie (00:04:34): Oh, yeah. Hadn't even thought about that particular angle. And so you said that at Swissport you served and worked with 800 airlines. Today would that number still be 800? It seems like we might be losing some airlines. Joe Phelan (00:04:46): Yeah. I mean, I'm not as close to the business as I was, but I think we can all be rest assured that the number of airlines has turned down quite a bit. Rob Collie (00:04:54): One of the things that you said at the end there which of course is in some ways my favorite part of your intro is that you're chief strategy advisor here at P3. This is in the industry what we would call you're a really good get. Why was that interesting to you? Why was that an interesting thing? Your background, it's next level. I don't get to rub shoulders with people like you very often. And I did back in the software world. I'm sure that everyone listening to this that's the question on their mind as well. What brings you to having a role with us at all? And this is a data podcast, why we got you? What got you interested in all of this? Joe Phelan (00:05:35): Well, it started quite some time ago. I've always been a big believer of information and the power of information and the value in empowering employees by providing the best possible tools. And I'm a believer that most employees want to do a good job for the company. And when provided a sense of direction and vision and the proper tools, they will excel beyond your imagination. So most people want to do a good job. Information to me is fun and that's why I was intrigued when you and I first talked about the opportunity. Joe Phelan (00:06:21): And I see tremendous benefits in business intelligence and in particularly Power BI because I've been a user in the past and I really enjoyed using the tools. And I think there's a great deal of untapped opportunity in this space as well. Several years ago I challenged a group of executives working for me to deliver real time information that could be accessed at various times throughout the day, giving us a view of performance across various departments, which included sales, operations, finance, and finance included accounts receivable, DSO by customer and customer service. Joe Phelan (00:07:13): And one specific department rose to the challenge and came up with a solution by using Power BI. And they produce some dashboards showing our performance relative to the KPIs that we set as an organization. With believe it or not, hourly updates that could be pushed or pulled to the users and drill down capabilities to satisfy whatever level of appetite you might have for information. And so this was created literally in just a few weeks, which I was amazed and what's even more amazing to me now is we can do it in a few days with the technological improvements that have been made. Joe Phelan (00:08:01): So this was so powerful. We ended up implementing across the company in every department and created a corporate consolidated dashboard. And the return on this invest which I was always interested in whenever we made these types of investments was extraordinary. Literally within a few months we recaptured our entire investment and the business was now spending much more time on process improvements versus gathering data. Rob Collie (00:08:33): Now, as a long time C-suite veteran, this wasn't your first brush with BI. It was just your first brush with Power BI. Can you speak at all to the contrast, what was BI like from your seat before that? Joe Phelan (00:08:50): It was pretty powerful, but not as powerful as Power BI. Power BI I think gave us better access to and more user friendly access to dashboards. And the cool thing about dashboards is we could drill down into key KPIs at virtually every department and get to the root of a problem. And this would allow much more time to be spent coming up with process improvements versus compiling data. So the dashboard would behind the scenes very rapidly access various data sources behind each of the KPIs. So depending on your level in the company and your appetite for quantity of information, you could continue to drill down and access what might be most relevant for you as a user. Rob Collie (00:09:52): So before this Power BI dashboard project that you're talking about, before that, in that same role, what was your daily regime of data consumption? What did it look like by contrast? What were you looking at ahead of time? Were you looking at spreadsheets? Were you looking at reports coming straight out of IT? Joe Phelan (00:10:13): Yeah. Typically the reports I would look at would be on a weekly basis or a monthly basis. They could be Excel spreadsheets or tables and the information or the data wouldn't really lead you to what I would call information, usable information and would require a lot more digging and compiling to get to the root of the issue. So as an example, as the CEO I might want to just see on a dashboard under the new environment a green, yellow, or red traffic light for each KPI. I'm not really interested in all of the information leading to why it was red, green, or yellow. And so if we take a weekly sales report by key account, if it showed red as the CEO in a meeting, the executive committee would ask, "What's going on?" Joe Phelan (00:11:27): And if I wanted to I could click as the CEO and drill down into the information, but more often than not the sales executive having access to that report knew ahead of time that the question would be asked and had already drilled down and had the answer. And so that's the beauty I think of the tool, is it gives visibility to all levels in the organization of the performance, and it helps you anticipate problems or questions and come up a lot more quickly with solutions to those problems, to those issues. Rob Collie (00:12:07): So many fascinating things to me in what you just related. So first of all, let the record state that the C-suite at some of the biggest companies in the world still is being handed spreadsheets on a weekly or monthly basis. And these are organizations with a lot of resources. Is not that they don't have BI. There is a BI department, maybe multiple BI departments, and yet it's still so often a spreadsheet. And not only is it a spreadsheet, which is just funny in a way, right? And it's not uncommon. This is everywhere. Spreadsheets still to this day, they still 100% run the world. And we're just in the earliest, earliest stages of evolving. But you also said that these were weekly or monthly. And so now we get into a real inefficiency here. Because they're spreadsheets they were taking a lot of human effort to compile. Rob Collie (00:13:12): And while you can't burn that amount of effort on an hourly basis, you can really only afford to allocate that kind of labor on a slower, less frequent rhythm. On the other side you're talking about Power BI and how it was running multiple times a day and you were able to... So when you say real time, I mean, it goes from weekly or monthly down to multiple times a day. I mean, that was actually one of my questions for you, was what does real time mean to you? That is a dramatic difference in frequency. Rob Collie (00:13:46): So even if what you were getting, those spreadsheets, even if they were perfect and they told you exactly what you wanted, the fact that you were getting them only weekly or monthly means that you're always looking in the rear view mirror. You're finding out things when it's in the sense almost... The opportunity to improve in many cases has already passed, or it feels like diminishing returns because you're never sure if you're going to be faced exactly with those sorts of conditions again, right? Whereas if you're [inaudible 00:14:16] more in real time you have an opportunity to head things off. Is that what it means to you? In real time, is that- Joe Phelan (00:14:22): Absolutely. The old business intelligence environment, you were looking in the rear view mirror and the information was so old. You might lose customers in the process of gathering information. And the cool thing about Power BI that I liked is we refreshed every hour. So every hour on a daily basis we were getting fresh information about whatever department it was that we were interested in. And what used to be really frustrating to me is to get a call from a customer and the customer tell me what his performance was. I was interested to hear from the customer, but I should know when the customer calls what the performance is. Joe Phelan (00:15:15): So if I've got a message to call a customer back under the Power BI environment, I could pull up real time information about what's going on with that customer and proactively then deal with the issue. So I can start off with an apology, "I see that we delayed your flight or your shipment today, and here's what we're doing about it," rather than hearing from the customer first and then you're on your back foot. You're on the defense. Rob Collie (00:15:47): Yeah. Yeah. I can absolutely see that. Joe Phelan (00:15:50): And I might just add the customer really appreciates that as well, because they feel like you've got a good handle on their business and what it is that you're doing for them, the service you're providing. I think the worst thing is for the customer to have to tell you and you're surprised. Rob Collie (00:16:10): Another thing that we were talking about that you mentioned about the spreadsheets that you were getting was that you were almost drowning in data rather that having actual information. This is a mistake that analysts and people who are good at these tools... It's a mistake that we, I think pretty frequently make, is assuming that there's exactly the same level of curiosity, exactly the same level of excitement at the next level up or multiple levels up the org chart from us. We're expecting everyone to be just like us. And so we'll very often take the data that's at our fingertips or that we can access us and we'll produce something... In a way is almost like about us. It's about my world. Rob Collie (00:17:01): And then on the other hand you said, "Sometimes all I want to see at the top level is a red, yellow, green. Is there even anything for me to drill into? I don't want to drill into everything just to find out red, yellow, green. I just want to know where our problem spots are and then I can focus attention." There's an interesting cynicism that develops here, down amongst the data people when it comes to executives in that there's a joke, in a way it's like, "Oh, no, the executives, all they really want to see is a big meatball of color like red, yellow, or green. That's all they ever want to see." Rob Collie (00:17:36): Or this morphs into another misunderstanding, which is... I guess this is true sometimes, but most of the time it's not, that, "Oh, no, the executives really only want it to look good." I think this is just really just a misunderstanding, a misinterpretation of signals. And it comes back to one of the things we talk about a lot, which is when you're building dashboards, you're building reports of any sort, you're really building software. You're building an application. You're not just publishing data. You're building software. Rob Collie (00:18:10): At Microsoft when we were building software, as much as we could, we tried to understand who was going to be using it, tried to walk in their shoes. And it's hard. It's hard for someone to walk in the shoes of the C-suite, even if we really want to. You have to start with wanting to and understanding what your day looks like. I know we're jumping around a little bit here, but can we talk a little bit about that? What would be ideal for you as an executive? What would be the advice you would give to people who are building dashboards or reports that are going to make it to someone like you? Joe Phelan (00:18:51): What I would want in an organization is my executives to have access each morning to information about yesterday's operation and for them to be able to access in each of the departments information on an hourly basis during the day if they need it, if they have an appetite. But as a minimum, to be able to start their day knowing what yesterday looked like and what is forecasted for today from an operating standpoint. And that could be true for just about any department. So having access to information real time and really understanding what's driving the organization in terms of key KPIs I think is extremely important. Joe Phelan (00:19:53): And what was always important to me was our having transparency in the organization. And so open and honest transparency. And I think that's what information does for you, is it helps make the organization much transparent and provides then the motivation for the team to act in a more proactive way. I don't necessarily want to know as a C-suite member, want to know all the details of a problem. What I'm more interested in is what are you going to do to resolve the issue? So I'm mostly interested in the fact that you've identified the issue and you have a solution that you're ready to employ to solve the problem. Thomas LaRock (00:20:51): I've never been a CEO, but I'm willing to try if some of these wants to take a chance, but when it comes to those dashboards, I've helped provide the data for these dashboards. But what I see often is what I call the dashboard danger or danger of the dashboard in general. You two have talked about a little bit already. You talked about the red, the yellow, the greens, and you talked about addressing the issues and knowing the problems. But here's why I usually push back on, so Joe, your executives show up and you said that dashboard should tell them what things look like yesterday and everything's green. And I would then push to you and say, do you know why it's green? Do you know why you're being successful? Do you know why? Do you have that data as well? Thomas LaRock (00:21:35): Because when I was coaching basketball all those years ago in my other life, when you were having success, you had to know why you were being successful, otherwise you really didn't have that advantage or that edge. So I would push back. When we talk about these dashboards, I don't see that often enough. I see people, they put their metrics in, they have their KPIs, everything's green, but there's also a level of greenness, right? Sales is doing good, so they're hitting their number. They beat it by 1%, but they could have been 10%. You don't really know the because you may never click on that little green light. Joe Phelan (00:22:11): Yeah. That's an excellent point, Tom. And you'd be a good CEO because that- Thomas LaRock (00:22:16): Hold on. I'm updating my LinkedIn right now. Hold on. Joe Phelan (00:22:19): I think just not accepting the data is green is important and I think great organizations want know why it's green, but also want to test and drill down in the green a little bit, because what you'll find within the green is the average has come out to be green, but there are some spots within that particular area you're measuring that aren't green and need some continuous improvement. And so that's the value of drilling down. And I think most CEOs will not just accept that... It's most good CEOs, will not just accept that it's green. Joe Phelan (00:23:04): They will in fact drill down a bit to try to understand where the weak spots are, because as the old saying is, is you're only as strong as your weakest link. You need to be mindful of the weakest links, understand what's driving the good performance and see if that's applicable to other areas of the organization as well so that you create a learning environment and you learn from each other. That's a good point, Tom. Thomas LaRock (00:23:36): Thank you. I'm making note of all the good points I have on this podcast- Joe Phelan (00:23:39): We'll keep score. Thomas LaRock (00:23:40): ... for my review later at the end of the year with Rob. So that's one. Rob Collie (00:23:45): Yeah. Well, Tom, just like Joe Joe said, I mean, your overall indicator can be green, but there can still be a lot of places where you need to work. Thomas LaRock (00:23:53): Oh, yeah. Oh, yeah. I know. I can't fake it with a dashboard for you. I get that. Yeah. Rob Collie (00:24:01): I think you're doing a great job, Tom, just so you know. Thomas LaRock (00:24:03): Again, going on my LinkedIn. Rob Collie (00:24:07): I'd like to continue to role play a little bit on my end. I won't ask you to role play anything. You just get to be you, Joe. [crosstalk 00:24:15]. Continuing this cynicism in a way, so when we started off and you were talking about your sincere interest and your authentic experience with using data and seeing Power BI and turning it into an advantage, turning it into very clear ROI. And sometimes the people in this line of work might not believe that that's true, meaning they don't necessarily get a lot of reinforcement that filters down to them, that the C-suite actually truly sees the value in it. Rob Collie (00:24:53): So it's almost like I want to ask you and so I will, how unusual do you think your stance is? Are there still a lot of the real old fashion CEOs out there that, again, the cynics amongst us are expecting to say, "Ah, I don't need no stinking data. I know how to run a business. I know what's going on"? That is a bit of a caricature that does exist. It doesn't mean that it's correct, but there's reasons why that caricature exists at certain levels of the organization. Joe Phelan (00:25:31): I think, Rob, the C-suite today is much different than it was years ago. It's comprised more of individuals, I think with strong communication skills, strong collaboration skills, the ability to create teams and the mindset for an open environment. It used to be information was power and amongst the C-suite it wouldn't be shared because it gave you as an individual an advantage within the team. But it's very different today. I think the environment is very open. It's more collaborative. I think people value working as a team because they recognize that it brings one plus one equals three, it brings the team working together, allows you to create success a lot more rapidly. Joe Phelan (00:26:28): There's a lot more receptivity today to providing employees with real time, meaning information as a result, is it gets back to that notion that I mentioned earlier, that if equipped with information and employees, frontline employees understand the direction, they'll excel and they'll perform extremely well for you. You just have to have the courage to give them the information that they need. So, yeah, I think we provide solutions for improved business performance throughout the organization by simply making I think information available in a real time basis. Rob Collie (00:27:17): It's good to hear that that's happening increasingly at the highest levels. That's a very encouraging trend. I got to admit, I haven't worked at a large corporation for a full decade now. I know what it's like with our clients, but our company, we only really attach at places where they're thinking the right way. We get a very positively skewed sample by virtue of the lens, the filter that gets applied to our client base. So it's really fascinating ongoing curiosity to me, to what extent... When you hear all the hype, data, data, data, data is the new black, you use data for competitive advantage and you can't help but wonder to what extent is this just a platitude that everyone is supposed to repeat, versus how much they've walked the walk. Joe Phelan (00:28:13): I think the C-suite today, Rob, recognizes that having real time information gives you a competitive advantage in the marketplace. And so the C-suite will wake up very quickly when they realize that they're losing market share or their unit costs are rising faster than a competitor or unit revenue isn't going up as high as a competitor. And so business intelligence I think will continue to be the center for driving business performance in the future. We're much more resilient now than in the past. And if you're not nimble and ready to move at or before the speed of change, you'll simply be left behind in today's business world. Things are moving very, very quickly. And so information powered in the right way and in the hands of the people driving your business will be I think the recipe for success in the future. And I think CEOs in the whole C-suite recognize that much more today that just a few years ago. Rob Collie (00:29:28): Well, that certainly paints a bright future for people who are good at data, but also people who are... As you were hinting at even at the C-suite level, it's not enough to be good at data. You have to be good at it. You have to understand the business and you have to be a good communicator. I'd love to get your reaction to this, but I personally believe that the longstanding separation between business and IT, where watching those in a way collapse into each other in some really important fashions. IT still very, very, very much has a role structurally and in terms of governance, but when it comes to BI, they used to also, in addition to those two, infrastructure or governance, right? They also had the role of the creators of all of the sanctioned reports. Rob Collie (00:30:24): It's that last part that's antiquated. And that there's a rise of this hybrid, a hybrid professional that's actually anchored in the business side of the house that becomes good at these tools such as Power BI and then in collaboration with IT is able to deliver a quality and a pace of information. It was just never seen before. Does that align with your experience and what you've seen? You talk about this set of Power BI dashboards, was that spearheaded primarily by IT or was it more of that collaborative? Were there people on the business side who actually had hands on the keyboard during the creation process? Joe Phelan (00:31:05): No, I think it was more the business side was engaged with Power BI and brought the solutions forward. And that's the beauty I think of Power BI, is most of what an IT organization is doing today in today's world is there's so many mergers and acquisitions. They're putting systems together and trying to rationalize the entire IT infrastructure. And so it was refreshing to me to see the business bring forward, the operations department bring forward a solution to this that didn't really require any IT support. Joe Phelan (00:31:46): It required an interested individual within that department to be the go-to and to quickly become the Power BI expert who could then train and get everybody up to speed on how to use the tools. That's the cool thing I think about and why I like Power BI so much is you don't have to rely on a large IT organization to put it into place when they've got many other big, big strategic projects that they are working on. And if you had to wait, it would simply take too long. Rob Collie (00:32:28): That's why Excel shows up on your desk every day. Joe Phelan (00:32:30): Yeah. Rob Collie (00:32:32): If you're waiting, you're just going to get Excel from the business, right? Joe Phelan (00:32:36): Exactly. Thomas LaRock (00:32:38): Hold on a second. The bottleneck isn't the use of Power BI, Excel, Tableau, any of that. Crystal reports, I don't care. That's not the bottleneck. The bottleneck is the data. The bottleneck is somebody in that organization saying, "Okay, so we got to go build this data warehouse." And these days it's not just a data warehouse. And we've gone beyond data mart. Now I live in a data estate by a data lake. And I shop at the data mart, we store everything in the data warehouse, but it's made by the data factory. And it's just all the data things. So that's always been the bottleneck, is somebody in the organization saying, "I've got a reporting tool, but where am I going to point it at?" Right? So it's great Power BI can come in. It's great you can assign a custodian to be in charge of this and all that, but you're still going to have somebody sitting there going, "I need some data." Joe Phelan (00:33:25): Yeah. I mean, you're so right. We're working with a client right now- Thomas LaRock (00:33:29): Twice. Joe Phelan (00:33:30): Twice you've been right now. That's good, Tom. We're working with a client now that is expressing that exact issue, is saying, "I'm ready to get going, but we're working on our data warehouse and I want to make sure that I have all the data from all the different systems loaded into that warehouse before we start to execute." And that's the beauty of Power BI. Again, is you don't need to wait to have all the data in the warehouse and have a perfect data warehouse. We can go to each of the systems and extract the data and produce what I would call then information in a very timely manner. But that is an education in itself because it's old IT thinking, and you don't know what you don't know. And it's just people not familiar with Power BI and its capabilities. Rob Collie (00:34:30): The software industry absolutely is to blame. People like me in my former job, we're the ones that are to blame for that because we produce an entire generation of tools that made the default assumption. First and foremost, for instance, that all of your data that's interesting is in a database and it's in a single database. So it produced this infrastructure first mindset because the world didn't have a choice. If given a choice, the world probably would've chosen something different, but the software industry said, "No, no, no, no, we're going to do it this way." It's really simple. Power BI just decided, "Hey, we're not going to work that way. We're not going to require that." And the sky didn't fall. In fact things really have taken off, but it's actually two things you were hinting at there though, Tom. Rob Collie (00:35:23): I'm going to call them both though. So it's not just access to data. It's not just this former... It used to be you'd build plumbing and then you'd think about faucets years later. Now we say that you can think about it as faucets first and then figure out what the plumbing needs to look like. And that ends up being a lot more efficient. But the other thing is, is that when IT was responsible for building everything, there was also this tremendous subject and domain expertise from the business side that had to be mind meld telepathically transmitted from the business to IT, and then implemented according to that spec. And that turns out to chew almost as much time as the endless infrastructure investment, because boy, that is... It's really hard. There's a lot. Rob Collie (00:36:12): The tribal knowledge on the business side, if you could dump it out and print out, it would be like an old encyclopedia Britannica, like lining a whole bookshelf and there's just so much nuance there. And so bringing it within the business to execute, to build shortcuts, all of that, all of that requirements transmission and misunderstanding and mistransmission. And then of course my other favorite point is that even when you write a perfect spec for something, and then you implement that spec perfectly, which by the way, has never happened in human history. Rob Collie (00:36:49): We've never had those two things happen in sequence, but even when they do happen, the first thing that happens when the business gets back the report that they just asked for, they go, "Oh, you know what? This doesn't even really answer our question now that we think about it. We need to filter it like this or drill down like that or correct for something, blah, blah, blah." Human beings don't know what they need until they've seen what they've asked for. And so the whole requirements gathering process, which again was inflicted on the industry by the software people, it's just a fools' errand from the start. So it's really, really, really a very different world when you don't have to wait so much on infrastructure and you're unconsciously communicating with yourself armed with all of your own business knowledge as you're executing on something. Rob Collie (00:37:39): So Joe, you said that this was a business led initiative. I've got to ask you and I've never asked you this, so what was IT's reaction to having something so valuable be built on the business side? I'll preface this. I'll even make it multiple choice, because everyone's going to be on the same page about this. On one hand, the reality that you describe, it's never been realistic to expect IT to keep up with BI. They're outnumbered tremendously. By definition they don't know everything about that business, the business knowledge. They can't. They have to do their other job. And on top of that they're being saddled with all kinds of things that only they can do. Rob Collie (00:38:23): I would want them to be grateful and appreciative and collaborative with this sort of development, but they also could be a little bit threatened. They could be a little bit resentful and I'm sure that the real world spectrum of this, it goes from guardrail to guardrail. There's plenty of examples of either endpoint and all points in between. How did that play out? Joe Phelan (00:38:45): Well, I think initially they were suspicious of the solution, but once they found out more about it and how it worked, I think it was quite refreshing for them because they were inundated with projects. Didn't really want to pull away from what they had on their plate. The business had a huge demand and an urgent demand for the information. So for the business to be able to come up with a solution, I think in the end was refreshing for the IT organization and it required very little support. So given the fact that it required very little support, it was welcomed, believe it or not with open arms. Rob Collie (00:39:35): Well, that is the way that it's supposed to go. And that's the way that it goes with our clients. We work very hard on that. It is truly a situation where everyone can win. I can understand it though, in the places where IT still feels a little defensive about it. It's not just something's being taken away from them. It's also this weird psychological trick that as long as it's Excel that the business is using, when Excel goes bad, it's generally not IT that's blamed. That's Excel. It doesn't mean that it should be that way, it just is that that's the perception. And then as soon as you use something like Power BI, "Oh, it's got the letters BI in it." So now when that goes sour, people will turn. And IT has one of those unenviable situations where they only get noticed when things go bad. Rob Collie (00:40:27): I understand their reluctance to increase their surface area in terms of the things that they might be held responsible for even though they didn't do it. Our governance and adoption program, we've got a really good recipe for this and you just can't play defense against something like this. You have to play offense. And if you do play offense and you set things up the right way from the beginning, you don't really need nearly as much defense. You don't need to be afraid of it. But if you sit back and try to control it and just squeeze it, white-knuckle tight prevent the business from unlocking this Pandora's box, they're going to open it anyway. You're never going to fence in that earthworm. So it's good to hear that. Joe Phelan (00:41:09): Yeah. Rob Collie (00:41:10): I do want to talk about supply chain of your specialities, but I'm going to throw one more curve at you before we go there. You and I have also talked about FP&A. We've talked about financial planning and analysis. I shared with you my theory that in a lot of ways FP&A is an organic... The existence of FP&A, that's a discipline, is a natural emergent response by the business side of the house to the underserving nature of IT centric BI. Because when I first got to know what FP&A meant and I started meeting people and talking about it, I'm like, "Oh, you do the BI mission. You do what BI was always supposed to be. That's what your charter is." And so I approached you with that theory saying... This rose color glasses theory, this is the new thing. I really found your response to that very intriguing and compelling. Are you willing to share that with us? Joe Phelan (00:42:17): Sure, if I can remember what it was. But let me give it a shot. I'll just talk from experience with FP&A and the organizations I've been with in the past, typically FP&A is there to support the CFO and provides the CFO with relevant information about the business for each of the departments. And so will typically sit in meetings, ask questions, gather information. And because they have a limited knowledge about each of the departments, the questions can be sometimes a little insulting to some of the departments and overly elementary. And so that I think causes frustration. It causes a waste of time and energy in the meetings. Joe Phelan (00:43:18): And one of the things that intrigued me about Power BI is when it was embraced by FP&A in the organizations I've been in, they would come to the meetings equipped to ask a lot more intelligent questions. And so the time spent in those meetings was a lot more fruitful and they became more team members in the problem solving process and support process for the organization and were respected a great deal more because they were more equipped for those meetings. So I think the overall respect for the FP&A folks was elevated quite a bit. Their overall knowledge of the business and their ability to ask more prudent questions, more meaningful questions in the meetings was much improved. And so overall it was time better spent and allowed us to get to agreed upon actions going forward and delivered a, I think more consistent and more accurate message to the CFO. Joe Phelan (00:44:34): And the CFO was really charged with keeping as well as the CEO, the shareholders informed, the investors informed. And so having good, accurate information about how the business was doing was extremely important to the CFO. So overall I think our efficiency as an organization improved quite significantly. The time used to be spent by FP&A, a lot more time used to be spent gathering information and gathering data and very little time on analysis. So Power BI allowed them to spend a lot more time on analysis and coming up with solutions and helping the organization than they were able to do before, because they just simply only had so much bandwidth. And when you're spending all the time gathering data and information, it doesn't leave you much for anything else. Rob Collie (00:45:32): Well, I think you remembered your answer from before perfectly. Joe Phelan (00:45:35): Is that the way I answered it before? Rob Collie (00:45:36): Yeah. And I remember when you first told me that, that first part especially when you said something... Was it along the lines of FP&A is this separate discipline, this separate team, and they're almost like a federal government type of entity? They're going around and they're trying to understand various departments and not doing a great job of it and therefore losing credibility. I started laughing and thinking to myself, "Well, oh, perfect. Yeah, they absolutely inherited the BI mission. That's just like traditional BI. It's the same failures just recast." And of course their tool set was different, right? The FP&A leans very heavily on Excel. Rob Collie (00:46:18): In some ways that gives them an agility, but it also gives them a tremendous labor intensive and static view, all the things we've been talking about. Whereas the traditional BI, their failures were more like not even being farther away from the business than FP&A was. And having tools that move at a glacial pace, even though they're great, once you get them built, you just actually never get them built. That's the only problem with traditional BI. Joe Phelan (00:46:46): Well, they say the power of being truthful is it's easy to remember what you said last. Rob Collie (00:46:51): That's right. That's right. mark Twain said that, something along that. The thing about always telling the truth is you never have to remember anything. Joe Phelan (00:46:59): Exactly. Rob Collie (00:47:03): All right, supply chain. I am not a supply chain expert. We have some at our company and not just you, but what does even supply chain mean? Does every business have a supply chain? Joe Phelan (00:47:18): I think every business in some form or fashion has a supply chain. Whether it's people, resources, or equipment or technology, just about everything that we do in the world today has some sort of supply chain attached to it. Rob Collie (00:47:35): We talk also a lot about logistics. Are these two terms basically interchangeable to you or do they mean different things? They seem to have a lot of Venn diagram overlap. Joe Phelan (00:47:47): Well, I think supply chain is part of logistics. And so logistics comprised of many different things in which supply chain is a piece of it. And I won't profess to be an expert. I've spent a fair amount of time in the logistics and supply chain business, but I'm not sure I'm a complete expert. Rob Collie (00:48:14): Oh, yeah. All you did was make it to CEO. You're right. We're still waiting for your career to blossom. Joe Phelan (00:48:22): There's always a chance. Rob Collie (00:48:24): Yeah. We're really hoping you pull it together one of these days, Joe. All right. So let's talk about data and the supply chain. And if you want, we can blend COVID in here or we can keep that separate. We can talk about that separately, but data, supply chain, logistics, COVID, go. Joe Phelan (00:48:47): Yeah. So let me talk a little bit about the general trends that I see in the short term in response to COVID and how that relates to the supply chain. I think consumers and businesses have become very resilient and are jointly learning to operate in a way that could add significant utility in the future. So what we've been through has resulted in a great deal of learnings and opportunities, I think, although it's been painful. Providing time savings for consumers is a commodity that I think carries more value today than it ever has. And we've all grown to learn more about the value of time savings as well. We've grown accustom to spending I think more time at home. That time's been with our families and we've had to adjust in many different ways. We've adjusted our purchasing habits. Many of us have learned to appreciate the time savings associated with online purchasing and delivering goods and services. Joe Phelan (00:50:01): Many of us have done that for some time, perhaps not to extreme that we do it today. And some of us haven't done it at all and have been exposed to it, and they've said, "Wow, this is really cool." And so I think the most successful businesses in the future will have to better manage all of the data that they have around consumers purchasing habits and turn that data into information that will increase their share of wallet. And so there's a lot of examples where information will help you increase your share of wallet and will help you understand what's driving consumer habits going forward. I think this data will also bring new discoveries for products and services delivered to your home when you need them at the right time. Joe Phelan (00:50:58): And so you will get the product without having to ask for it because the information leads to the conclusion that now is the time. And so that in itself I think will be pretty powerful. And some may argue, Rob, that this is all temporary adjustment. Dining restaurants will open full again and movie theaters and malls will all open back up and grocery stores will get back to normal. However, I think there's a high probability that many consumers like the change that has occurred. They like the change to their purchasing habits and will opt for these different options in the future, drive up for pickup of your groceries, delivery of your groceries to your door, classrooms will also look very different in the future with online delivery of education at all levels, distance learning, things like medical appointments. I don't know about you, but I'm afraid to go to the doctor's office. Rob Collie (00:52:13): Yeah. Joe Phelan (00:52:13): I probably shouldn't be, but I am. I'm afraid to go see the dentist, but with telehealth now and I've used it, you can dial up a doctor and get some really good consultation and advice and you can see the doctor and she or he can see you and prescribe a remedy. And so the way of doing business I don't think it's temporary. I think we've had some setbacks. We've made some adjustments, but a lot of what has happened as a result I think will stick for some time to come. And so with all the previous spending habits changing, a significant shift has already occurred. And as a result of our getting more comfortable with doing things online these past several months, I think we'll see the new... I hate to say the new normal because it's cliche, but there will be a new normal going forward. Joe Phelan (00:53:18): And some people still like going to the grocery store and some people still like going to the movie theater and that will continue, but I think some habits will change. All of these have changed rapidly the businesses and the new data that's needed to crunch and understand consumers demands, desires, and purchasing habits will become even more prevalent in the future. So the demand I think for information will continue to excel. Supply channel changes, the consumer purchasing changes, consumer demand changes, all of this will thrive off of the need for new and more real time information. I think businesses will need to be more agile with the data and the information. Joe Phelan (00:54:15): Waiting until the end of the quarter or the end of the month or the end of the week will no longer allow you to be as resilient as you need to be as the consumer demands you to keep up daily. And crunching data the old way is too time consuming and too cost prohibitive to be as agile as you need to be to compete in today's world. Today's world is happening today, it's not happening tomorrow and it's not happening yesterday. It's happening today. Businesses will need to access realtime information captured from mounds of data collected throughout the organization across multiple functions to ensure that timely decisions are being made for consumers and automation, I think will play a big part of that going forward. Joe Phelan (00:55:14): So all of these IoT devices that we've grown accustomed to use in the past, we'll continue to use in the future, but they'll be aided with more real time and relevant information as we go forward. So they'll become an even more important part of our lives I think going forward. It's a long answer, but there's a lot going on there I think. Rob Collie (00:55:43): Well, true or false, in my household we have IoT cat feeders. Joe Phelan (00:55:50): [crosstalk 00:55:50]. True. Rob Collie (00:55:54): True. We have IoT cat feeders. And speaking to the logistics and all that kind of stuff, we ordered new dishes for these boutique IoT cat feeders, and the dishes arrived the next day. It's just unbelievable. Joe Phelan (00:56:12): That's called Amazon Prime. Rob Collie (00:56:14): It is. It is. Yeah. When you said, Joe, that you think a lot of these changes are going to stick more than people expect, I think one of the things we almost always implicitly underestimate is the impact of shifts in investment on these sorts of things. So for example, I remember... I guess it was two years ago, maybe it was more, I don't remember really, but OPEC decided to just dump a bunch of oil on the market to drive the price way down. And their goal was to crush the domestic United States shale oil industry. Drive the price of oil down to a point where it wasn't profitable to extract from shale. Rob Collie (00:56:58): And my first blush thought was, "Well, that's just silly," right? Because as soon as they put the price back up to where OPEC wants it to be, it'll be profitable to do shale again, but it'll temporarily wipe out all the investors and the infrastructure and equipment. And maybe it's a very expensive business and it's going to drive the price of investment higher in the future because it's been proven to be risky. And I was like, "Oh." Once I connected those dots I was like, "Oh, this is three dimensional chess." And so there's no OPEC that released COVID on us. It's not a 5G bill gates conspiracy, but it still has some of that same sort of effect. Rob Collie (00:57:44): Even if you're just a small regional bank, you're going to be less inclined to provide a small business loan for a new in-person restaurant going forward than you used to be. You're going to be more likely to fund the carryout joint. And so even if the pandemic passes and we don't get another one immediately on its heels and people's short memories start to... There's still this smart allocation of investment and those people have longer memories because they have to. That's their whole job. And so I think there's a durability to this trend that's going to be enforced by the financial community. And even if the rest of us decide, "No, let's go back to normal," it's still going to be just more expensive to launch a brick and mortar sit down restaurant than it used to be, for example. Joe Phelan (00:58:42): It'll change. I think you won't see a traditional perhaps brick and mortar restaurant like we've seen in the past. There's many restaurants that I've seen that have done fairly well through this crisis because they were able to very rapidly change their model. And so knowing that they were going to be shut down for a while or they would be back open with a lot less capacity, they turned a lot of their business to dine out and very quickly adjusted with their staff and with their consumer base and reaching out to the consumer base and advising them of the new business, "This is how we operate now. We still have the same products, the same good food, but we'll bring it to you or you can come curbside and pick it up." Making those adjustments real time and quickly is extremely important for any business. Rob Collie (00:59:52): Yeah, totally. So let's get back to supply chain because I actually know just enough to be dangerous here. I mean, we have a number of clients that we work with who in response, not to the pandemic, the disease itself, but more in response to the financial fallout, the financial crisis related to it, the collapse in demand in a lot of sectors, we see a lot of inventory reduction projects. Inventory reduction really it's just like a one time savings for a company. If we temporarily stop making new stuff and allow our warehouse to halfway empty, then we're making revenue on that stuff, but we're not spending as much as we normally would. We don't have to replace it. So we allow ourselves to get a little thinner in reserve. And so that's a pretty time honored tradition during downturns. Rob Collie (01:00:45): And it's been really, really gratifying to see how much help we can be in that space. We can really help drive it because sometimes it's hard to change old habits. And so having good dashboards to track all of that, it really makes a huge difference. We even have a case study that I can't talk about yet because it hasn't been finally approved, but a really, really cool case study on exactly this. We also have experience with the other side of this equation, which is the demand shock in the other direction and the supply shock. So we have another client who before COVID put a monstrous order in as they do every year from China for a crucial component of what they manufacture. They always do that because of the Chinese new year. Rob Collie (01:01:30): And they know there's going to be an outage in supply and it turned out there was a huge outage in supply this year when COVID ravaged China. And it just happened to be timed just right so that they were sitting on a good supply. They were stocked up and able to produce a bunch of product when COVID caused their sales to go up tremendously. Actually COVID influenced their sales in the positive direction. And if they hadn't had this lucky chance that they had just stocked up on a bunch of supply, they would've been unable to take advantage of the demand. And if their competitors had been able to step into that void, they might have actually experienced a permanent loss of stature in their market. Rob Collie (01:02:15): So there's these two opposite tensions in a way. There's this insulation against the black swan outage, where you actually need more inventory. And then there's the traditional desire to reduce inventory in response to a financial crisis. And I don't really have a question here to be perfectly honest. I just find this to be a fascinating tension and want to know if you had any thoughts on it. Joe Phelan (01:02:39): A lot of businesses, big businesses have traditionally gravitated to one supplier for a product or a component because it allowed them to get the unit cost down for that product. So ordering larger quantities gets you to a lower unit price. But the cost of logistics when things don't go well can be quite significant. So I think a lot of companies now are moving more towards having geographically more suppliers. And so there may be a supplier in China, there may be one in Latin America and there may be one in Europe that you use and your costs for producing those components might be a little bit higher, but you're able to then leverage the supply chain either during downturns or times when things are extremely good in a more productive way. So I think many businesses are reevaluating their stance on that and their procurement habits and are making different decisions based on what they've witnessed particularly over the last several months. Rob Collie (01:04:10): Yeah. One of the things that I've seen many times over the years now is that whenever the world changes, whether for good or bad, positive or negative, it immediately invalidates all of the tribal roadmaps that have been developed over time in terms of trial and error. And maybe those roadmaps really weren't all that good to begin with. A lot of times when you poke at them you discover that they really weren't as good as everybody thought. So sometimes it's a blessing that everyone comes to this realization that we can't trust that roadmap anymore. And so the need for good analysis, the need for good information, the ability to see through all of the noise, to see the forest for the trees, there's only a higher premium on that. And this is one of the reasons why our business is actually blessed to be doing quite well during otherwise completely historic, awful situation. Rob Collie (01:05:08): But BI has always done this. Even bad BI used to absorb more spending during downturns than it did during "normal good times". And a lot of the things that you've been saying today, in fact without any sort of prompting or pre-planning, I have a blog post and a talk that we're turning into a white paper that's basically 10 things that should be table stakes for your modern data culture. And, Joe, you've just been again, without any leading the witness you've been hitting on so many of the things that we talk about, that like the increased frequency, the ability to drill down. You use the word across a lot. Seeing across different silos of the business. You also talked about mergers and acquisitions and how many modern companies that's how they've been built. Rob Collie (01:06:04): And so even if they're operating in parallel spaces, you're still inherently siloed from the get go and being able to see across is a big strength here. And so another thing we wanted to talk about for sure is your recent interview in the Supply & Demand Chain Executive magazine. You're part of the cover story. And if we were playing Power BI bingo or a Power BI drinking game for every time you managed to get the words Power BI into that interview and into the pages of this publication, I mean, we'd be living pretty happy at the moment. What did you think about that process? Was that enjoyable? Joe Phelan (01:06:39): Yeah, that was fun. I enjoyed the opportunity. It was with a publication called Supply & Demand Chain Executive. It was the September issue. And I think they did a very nice job of discussing how a smarter supply chain is developing as we result of our better using information assisted and various technological solutions. The article is called industrial revolution 4.0, which I thought was a cool title as well. And in that article I emphasized the need to have the right information available at the right place and at the right time. I think the same can apply. Joe Phelan (01:07:25): This is the beauty I think of Power BI is whether you're using various IoT devices, geofencing, AI, robotics, drones, RFID sensors or GPS, real time information will always be the key success factor I think for these types of solutions. And smart devices aided with smart information and smart people I think will provide a smarter supply chain and smarter solutions for the consumer. And that's what I tried to emphasize in the article, but there was a number of industry experts quoted in the article and I thought they did a very nice job. It's a good read. Rob Collie (01:08:11): And one of the things you mentioned in there that I really liked was as you incorporate more and more automation, more and more devices in your operations, somewhat paradoxically your need for monitoring increases because there's fewer and fewer humans involved. And even though humans are of course their own source of error and imperfection, an automated system can run for a very long time in the dark doing something you don't want it to be doing when there's literally no human being watching it. I find that fascinating as well. So it's basically constant reinforcement that I have chosen the right industry, the data industry, we can all just pat ourselves on the back. Joe Phelan (01:08:53): That's called confirmation bias [inaudible 01:08:55]. Rob Collie (01:08:55): Is it? Is it? Well, I'll tell you what? If I started getting different signals I would've changed jobs. Joe Phelan (01:09:03): Well, that makes it legit [crosstalk 01:09:04]. Rob Collie (01:09:05): At least when I talk my own book I am putting my money where my mouth is. So I think in closing here, we talked about this a little bit earlier on, let's make it a real crisp direct question opposed to a sidelight. What advice do you have for this new breed of modern data professional in terms of becoming more relevant, more noticeable even, more valuable to the C-suite or to leadership and executives in general? What are the advice that you would give someone who's at that point in their career right now? Joe Phelan (01:09:38): Wow. When you ask that it brings back memories of my walking by the C-suite early in my career and thinking to myself, "How do I get one of those offices in the future?" The answer I came up with is, "I guess I need to start thinking more like a C-suite person. And if I start thinking and acting more like a C-suite person, maybe I will become one." And so I think the question's relevant from the standpoint that data people should start thinking and acting more like C-suite people. Don't consider yourself just a "data person". Put yourself in their shoes and try to first understand their needs and the needs of the organization, and then use the lingual that they use. What type of lingo does the C-suite use? ROI, return on capital deployed, shareholder value and understand how the solutions you provide will help them accomplish their objectives. Joe Phelan (01:10:56): And I think that's how you get at the C-suites attention, is understand what their objectives are and how you can solve those problems by providing solutions. Because I don't necessarily view us as data people. I view us as problem solvers and process improvement engineers and many other things, but we can aid the business in making significant improvement in performance. It's not about creating just a pretty dashboard or gathering data. It's about providing meaningful real time solutions for the organization. So I think that's an old term, data people and I think we need to all start thinking about ourselves as something a little bit different in the future. Whether it's a process improvement engineer or it's strategic enabler engineer or whatever, we need to look at it a little bit differently going forward. Rob Collie (01:12:10): Do you hear that, Tom? You are no longer ever allowed to use the term data janitor ever again? Thomas LaRock (01:12:17): That was not the takeaway I had from what our guest just said. Rob Collie (01:12:22): He said you shouldn't call yourself data people. You're like, "Yeah, so I'll just call myself data fluky." That'll really set the right mindset. Thomas LaRock (01:12:32): Data professional. I usually say data professionals and that covers a lot. Rob Collie (01:12:37): It does. Thomas LaRock (01:12:37): It's your data, [crosstalk 01:12:38], it's your DBAs, your data engineers, your Power BI, but- Rob Collie (01:12:41): We stopped talking. Joe Phelan (01:12:42): You behaved. Rob Collie (01:12:45): Mostly. Mostly behaved. Yeah. We were talking about data janitor and all of that. And I think we probably got a little bit chopped up there. We're okay. I mean, Tom, was just wrong, so it doesn't really matter. We'll just move on. Joe Phelan (01:13:01): Yeah. Hopefully- Thomas LaRock (01:13:02): Wait a second. Wait a second. Nobody said I was wrong. I'm two for two on the day. Rob Collie (01:13:09): I said you were wrong. You're right, my vote doesn't carry the same weight. You're right. I understand. Thomas LaRock (01:13:16): How did we go from some great advice to all of a sudden Tom being wrong? Rob Collie (01:13:21): I don't know how that happens. Thomas LaRock (01:13:23): Oh, as the DBA though, I accept this result. Rob Collie (01:13:26): I did want to magnify what Joe was saying, which is... I actually yesterday just started a document. All it's got right now is a title. The working title is the I is for improvement. Sort of like an attempt to rebrand BI, intelligence was the wrong goal. Thomas LaRock (01:13:44): I like that. Rob Collie (01:13:45): Being informed. Well, those spreadsheets that you're talking about, Joe, that people send to you, that's checking the check box of informed as far as they're concerned. And when you talk about thinking like the C-suite, you've got to work backwards from improvement. And it's really, really, really interesting. You'd think on the surface that if you work forwards from the data and think about what you can do with the data and what you can present from the data and then put that out there versus if you work backwards from... Rob Collie (01:14:18): Don't even think about the data first. Work backwards from asking what can the organization do to improve and then go looking for data and building a solution that's completely the inverse of what "data people"... Their first instinct is to go the other way. To go data forward rather than improvement backwards. And the Microsoft stack has really planned in advance for this. They call it the Power Platform now because it's not just Power BI, it's also Power Automate and Power Apps. And the idea is that you should create positive action loops in your business that drive improvement. Rob Collie (01:14:58): I got to give them credit. Microsoft has already positioned themselves for the next fight. I think that within the next few years people like Gartner will be evaluating BI vendors not just on their ability to build the right kinds of reports and things like that, which has been most of the criteria. They will start adding the integration of action like why do I have to leave my dashboard? If I've got a conclusion about a change that I need to make, why do I need to leave the dashboard to do that? Why is it so disconnected? Why can't there be a button right there next to that store? We need to increase their share of product X or something like that. Why can't I at least contextually have a jumping off point there to go and take that action? Rob Collie (01:15:45): And having that mindset of embedding those kinds of action, taking verbs in the reports also helps really keep you honest when you're building reports, when you're building dashboards with this improvement in mind that they're like, "Oh, if that's part of the dashboard, I can't cognitively let it out of my head. It has to be there. It's just as important as the charts." And I think we're going to see this is the new thing, this is the new mindset and Microsoft has already quietly built the platform that can do this. Even though it's not the thing yet, they're ahead of the market. And I was just, "Oh, yeah, I'm really excited about it." Joe Phelan (01:16:22): I think our consultants are really performance improvement consultants. Rob Collie (01:16:27): I agree. Joe Phelan (01:16:27): So they can provide the tools and information for your business to allow you to improve. Rob Collie (01:16:35): Yeah. It's a different breed. Just like we're talking about the future is the hybrid. Our team, it very much reflects that. It's not an accident. This not your father's old mobile. This is not your father's BI consulting team. Joe Phelan (01:16:52): Yeah. Agreed. Rob Collie (01:16:53): I'm glad that you see that. That's incredibly validating for me and so I appreciate it. Joe Phelan (01:16:58): But somehow we need to get, I think that messaging across. That might help with business engagement in the future, is to position... We're getting off topic now, but position the business more as performance improvement consultants than- Rob Collie (01:17:14): We also just need to get people to start searching for performance improvement consultants. Joe Phelan (01:17:18): Yes. True. Thomas LaRock (01:17:19): I got news for you. If people search for performance improvement you guys will never be front page of Google. I'm just telling you. Joe Phelan (01:17:26): Oh, I see. I see. Yeah. It's going to be some sort of herbal supplement. Thomas LaRock (01:17:32): You will not win that SEO race. Rob Collie (01:17:36): You're talking about sports, am I right? Thomas LaRock (01:17:40): I'm talking about things that do require endurance from time to time. Yeah. Joe Phelan (01:17:46): Great place to stop, right? Rob Collie (01:17:48): Oh, it's a fantastic place to stop. Joe, this here has been a real pleasure. Thanks again very much. Joe Phelan (01:17:53): You're very welcome and thank you. I really enjoyed it and love the opportunity and hope that we can do it again sometime in the future. But thanks for having me. Appreciate it. Announcer (01:18:03): Thanks for listening to the Raw Data by P3 podcast. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Interested in becoming a guest on the show? Email lukep, L-U-K-E-P@powerpivotpro.com. Have a data day!

This episode features journalist Michael Salfino; who writes for The Wall Street Journal, FiveThirtyEight, The Athletic and other top shelf publications. In a spirited and wide-ranging discussion, we cover all of this (and yes, we use the word "multiverse" a few times): The statistic that changed Mike's life The wisdom of Einstein and Twain The theme of the Hybrid professional resurfaces, this time for Analytics and Communication Why your career is a decathlon, and 90th percentile can actually be 99th How it's often better to be LESS informed in your own domain Why the courage to improve must be fostered top-down rather than bottom-up How analytics might be "ruining" sports (and sports might be bad for compassion) Why theory is "over" facts, and facts sometimes should be ignored Why Rob thinks predicting political winners is VERY different from predicting sports winners Here's some handy dandy links for you: Michael Salfino's Twitter Michael's article on Data In Politics Michael's article on Facial Recognition and Race Michael's article on Movie Ratings Episode Transcript: Rob Collie (00:00:00): Welcome friends for our second episode and our first non co-host guest. We're very fortunate to have the one and only Michael Salfino. Now this guy writes for some really prestigious outlets, including the Wall Street Journal and FiveThirtyEight. And he's primarily known as a sports writer, but I think you'll see that he's a bit of a Renaissance man. And of course he's into data, otherwise we'd just called the podcast raw, wouldn't we? I've known Mike seemingly forever, even though we've actually never met in person. And I've been following his work since probably the mid two thousands. He and I even collaborated on a research project back in the day when I was still at Microsoft and I was working on the original, great football project. And that was the first time I aimed these BI tools at football data, which was my hobby at the time, but that was years ago. Rob Collie (00:00:46): And we recorded this episode. We recorded this at the end of week three of the NFL season. And so you're going to hear some dated references in there from a few weeks ago. And don't worry if you're not into sports or football. The key thing to pay attention to is really how his mind works. He blends data with his subjective experience in a really, really, I think professional and smooth way. One of the things I really like about him is how curious he is. And of course, he's pretty funny. We had a really, really, really good time with Mike and I think we're definitely going to have to have him back on the show regularly. So without further ado, let's get to that intro music. Announcer (00:01:20): Ladies and gentlemen, may I have your attention please? Announcer (00:01:27): This is the Raw Data by P3 podcast, with your host, Rob Collie, and your co-host Thomas LaRock. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Raw data by P3 is data with the human element. Rob Collie (00:01:45): Welcome, Michael Salfino. I've really been looking forward to this for a long time. I've been reading and listening to you for well over 10 years now, probably 13 years. You write for and contribute to a number of different projects and outlets. Why don't you just kind of give us a rundown of that? Michael Salfino (00:02:02): Sure. I guess the best way to say it is I'm more of a reporter and journalist. So I'm just trying to use some of the more advanced analytics tools to communicate the players and the teams that are actually good and likely to remain good for the foreseeable future, as opposed to some of the methodology usually employed by journalists, which is more descriptive rather than predictive in terms of forecasting. Michael Salfino (00:02:31): So I don't consider myself in any way some sort of like data guru or even statistician. But I think I have a good sense, from my background that originally began in the corporate world, of how to actually translate some of these higher minded concepts and to ferret them out so that readers are able to, I think, glean from them a better understanding of the game. So that's all I'm trying to do. Michael Salfino (00:02:55): Like I want to give my readers their eureka moment that I had when I was a kid, when I discovered the yards per pass attempt statistic in a book about professional football. And despite following football so intensely from grade school through high school, I had never heard of this statistic. I had never heard of it on the radio shows that I listened to or the television broadcasts or in any of the articles. So I was extremely annoyed that there was a statistic that correlated to winning, they said 75% of the time, irrespective of any other statistic. Michael Salfino (00:03:29): And it was an efficiency stat, which I sort of intuitively knew at that time. So it was much better than a total yardage stat. And I felt that I was cheated, as a person who is passionate about football, to not know something so important and not to have been told this through all of those years, thus far in my life. So I'm trying to actually provide some of that information to the people who read my stuff so that they can maybe better understand the game that they're so passionate about. Rob Collie (00:04:00): At our company we have a saying, which is your metrics that you're using, half of them should have words in the name of the metric, like the word per. Michael Salfino (00:04:10): Yeah. Rob Collie (00:04:11): Or adjusted or indexed. If you're not doing it that way, if you're just sort of using raw metrics, there's no denominator, there's no correction in it. You're probably breathing your own exhaust in a number of ways. But before we even get into that, I want to give you a chance to name drop a bit. I'm just going to cue this softball up there, right down the plate. What are some of the outlets you write for, Mike? Michael Salfino (00:04:33): Well, I started a syndicating nationally in newspapers for fantasy sports. That was after I wrote for, then it was Baseball Weekly, but it became Sports Weekly. So they wanted somebody to cover fantasy football. Then after the fantasy football, I started working for some of the regional sports networks on the content side. And from there, I was discovered by a Wall Street Journal editor who liked article that I did on evolution and pitching mechanics, as it related to a Yankee pitcher at the time, Joba Chamberlain. He was criticized for suffering an injury and losing effectiveness because he was unable to precisely repeat his throwing motion when pitching. Michael Salfino (00:05:17): But the biomechanics on the evolutionary side said that that is something that's just natural in athletics because nobody was able to hunt the mammoth the same exact way. You know what I mean? And so since sports is more akin to something like that, the ability to precisely repeat movement is not something that comes naturally to athletes. And just because you look at it when somebody gets hurt and you see that they're different, we're not looking at it at all the times that somebody doesn't get hurt or lose effectiveness. So there's no way to actually know whether this ability to precisely repeat motion, with slow motion video evidence is actually predictive of any kind of success, or even if it describes success. Michael Salfino (00:06:02): So the Journal liked that article and they wanted somebody. They wanted, what they said was somebody who they said was like a real nerd to write analytics driven pieces on the sports side. So I did that for about 10 years. I still do it occasionally for them, but now I mostly write for The Athletic on the fantasy side of things, and for FiveThirtyEight on the analytics side of things. When an editor from the Wall Street Journal went over to FiveThirtyEight, I kind of went over there with him. Rob Collie (00:06:33): Just kind of minor publication, you know. Wall Street Journal, FiveThirtyEight. Michael Salfino (00:06:36): I'm very fortunate. Yeah. I don't know like how this happened, but I think I found a niche because I was able to produce reader friendly content because of my journalism experience. And I always had a passion for data, just coincidentally. So it was just a perfect fit to be at a publication like the Journal where you had to convey some of these higher minded concepts in a more sort of earthy and lay person kind of way. So it was just a fortunate fit really on my part. I've been lucky. Rob Collie (00:07:11): I think that's actually really the thing that was compelling. I've been much more into fantasy football 13 years ago than I am today, but I'm still doing it. Michael Salfino (00:07:19): Yeah. Rob Collie (00:07:20): But I mean, I used to read basically like the entire internet of fantasy football content. Michael Salfino (00:07:25): Right. Rob Collie (00:07:25): And so I had exposure to plenty of different people. And I think the thing that drew me to you and your oftentimes partner, Scott, is the blend, sort of, of the nerdery, of the technical stuff with also human element, which is, by the way, the tagline of this podcast, data with the human element. So that blend and the ability to explain it. One of Einstein's famous quotes is, "If you can't explain something simply you don't actually understand it well enough." Michael Salfino (00:07:56): Right. Rob Collie (00:07:56): It's not just a communication problem. It's even a symptom of not having a good model in your head. So I think it's more than just a communication skill. I think it's communicating those things simply to yourself is the first part, before you can communicate it to others. Michael Salfino (00:08:12): Exactly. Rob Collie (00:08:13): I've always enjoyed that. I understand from your perspective that it would seem like fortunate accident and all of that. But as an observer, as an outsider, I think it makes total sense. I mean, you're able to take important concepts and make them broadly accessible, and also not scare everybody off with it. It's not elitist. Michael Salfino (00:08:33): Yeah. Maybe when I say that I'm fortunate, I think I'm fortunate in really not having both feet in either world. Like having one foot in each world may make me somebody who, in one of those sides of the sports world would seem less than super qualified to be talking about whatever it is that I'm talking about. But I think the fact that I tow both of them enables me to kind of find a sweet spot in communicating to the reader. You know? If that's clear. Rob Collie (00:09:03): Yeah. Michael Salfino (00:09:03): So the people on the analytics side, the people who go to like the, the Sloan meeting and stuff, those people would maybe interpret my understanding of some of the statistical concepts to be lacking. But I'm still able to report on the stuff that they're doing, I think, to the lay reader, in a way that's clearer and concise and compelling, which sometimes they lack with their non journalistic background. Rob Collie (00:09:32): Yeah. What I've discovered, and I didn't expect this. I've sort of been on a similar journey in some ways. You can be 90th percentile in a few different fields and feel relatively inferior because you only tend to hang out and sort of consume the work of the 99th percentile, and things like that. But then when you realize that your particular profession or life isn't a single event sport, it's more of a decathlon. And if you're 90th percentile in a bunch of different things, the overall decathlon sport, you might be 99th percentile when it's time to blend them. Michael Salfino (00:10:06): It's really interesting that you say it that way, because one of the things that I've always done, and it's unusual, I think, in my profession, is that I don't really consume a lot of sports information. I voraciously consume all kinds of news and articles and analysis, but it hardly ever relates to sports. And I was talking to my friend, Cade Massey, who I've worked with in the past and who is a professor at Wharton. And we were at lunch. I told him how I didn't read sports very much anymore. I mean, obviously I did prior to writing about it. And he said that his mentor, Richard Thaler, from the University of Chicago, always told his students do not read about economics, read about everything else. And then you will learn the things from all those other fields that you can then and apply to economics. Michael Salfino (00:11:00): And you reduce, I guess, being so sort of cloistered in whatever it is that you're reading that you feel that you can only pursue those paths. So in other words, if you're reading only about your field of expertise or what you're actually writing about, you may feel that that field of play is limited to only those ideas that people have kind of presented. And you may feel that you don't really have the reign, creatively, to maybe focus on things that you would have naturally if you didn't feel that you're kind of stuck with what everybody else is talking about. If that makes any sense. Thomas LaRock (00:11:36): I wanted to mention something real quick, because Rob had just talked about the 90th percentile and that struck me. It's like describing somebody who's consistent, slow, steady, not flashy or over the top and excelling in any one area. And when I saw your name come up as our guest, so I did a little research of course, and I'm finding you in all these great places, such as FiveThirtyEight. Just what came out today here, the little transcript of the conversation you're having, this resonated with me. You made this comment just today. It just says the NFC East is now 2-9-1, by the way. Michael Salfino (00:12:14): Yeah. Thomas LaRock (00:12:14): And I love it because you're just stating a fact as a journalist, a sports writer. And a lot of times I think these sports writers kind of have hidden agendas to some degree, or they want to be a little more flashy because they need that, the ratings, the hits. Michael Salfino (00:12:28): Yeah. Thomas LaRock (00:12:29): They want to be at 95 percentile. So they would just say, the AFC is 2-9-1, and then they would go on and on and on about how horrible everything is. You didn't do that. You just stated the fact. You say, by the way, draw your own conclusion. And I love that. And I've been reading more and more of your stuff today. And I really like the way you communicate. Like you said, you don't want the reader to be cheated. But you're also not going to bring them to a place where you want them necessarily to go. You're going to let them get there through your words. Michael Salfino (00:12:58): Exactly. Thomas LaRock (00:12:59): And, and I think that's just great. Michael Salfino (00:13:00): It's kind of like when I've taught in the past. The university that I attended when I was a kid wanted me to come back and teach a couple of journalism classes, and I did it for two semesters. But, similarly, that was like sort of the approach, right. Where you're just trying to make sure when you're editing, like somebody's journalism paper, I feel compelled almost to just fix it. But what you have to do is you have to kind of tell them what they're doing wrong so that they understand it and that they can fix it. And that's the same thing that I kind of do, especially with the fantasy analysis and the player and the team forecasting, is that I try to show what I'm doing so that they can then take those things and apply it to whatever player it is that they're looking to analyze. You know? Michael Salfino (00:13:50): So it's kind of the old saw about teaching somebody to fish, rather than just giving them fish. But I really think Twitter is probably ... with what you were saying, as far as the concision of some analysis, Twitter is just such a great tool for really sharpening your ability to say things in a very condensed way, which I think really heightens the ability of a reader to understand what it is that you're saying. Rob Collie (00:14:19): Rob Scribbles in his notebook, must start using Twitter more. Because I'm famously long-winded. There's a corner of hell where I will be forced for all eternity to write conference abstracts, like the abstract of a talk. It must be 150 words or less. Writing those for all eternity would be pretty much my worst case scenario. Michael Salfino (00:14:42): Well it's weird because, one of the things that people do in my field is they'll promote something that they've just written on Twitter by the word count. And a greater word count is supposed to equal a greater level of sophisticated analysis. And to me, it's almost like a confession. Like if you just said, I just finished 2000 words on Kenyan Drake and the Arizona Cardinals running game, it's like, why the hell do I want to read 2000 words on that? You know? And what I always say, and this is, again, I think I have been fortunate because at the Wall Street Journal, I was on the print side. So I had to say these very complicated things in about 275 words or less. And sometimes when the subject matter would be extremely involved when I was talking to like Stanford biomechanics people about thoroughbred racing and why Secretariat was sort of a unicorn and why horses haven't gotten faster, like human athletes have. It's like, how am I going to do this in 275 words? Michael Salfino (00:15:45): But I did it. It was hard, but I did it. And I always tell people who complain about a short word count that Lincoln saved the Republic in 275 words. So if Lincoln can do that in 275 words, you should be able to tell somebody why they should either own or not own Kenyan Drake in 275 words or less. Rob Collie (00:16:06): I feel so indicted, you know? Yeah. But you know, it's another saying, right? The mark Twain quote, "I apologize that this is so long. I did not have time to make it brief." Michael Salfino (00:16:20): Exactly right. Thomas LaRock (00:16:21): Yes. Michael Salfino (00:16:21): That is exactly right. It's so much harder. And you know, what's weird is a lot of times writers get paid by the word. And it's like, guys, it's harder to write shorter than it is to write longer. You know? So that's just such a stupid way of compensating writers. Rob Collie (00:16:37): Yeah. And there's actually a parallel to that in the old Revenge of the Nerds documentary with Steve Jobs and Ballmer and Gates were their talking about how IBM was measuring their output by KLOCs, thousands of lines of code. And if you go back and look at sort of like the origins of Microsoft, it's these competitions between Bill Gates and Paul Allen to see who could write a certain routine in the least code possible. Michael Salfino (00:17:04): Right. Rob Collie (00:17:05): Like viciously. Like a single byte was enough to make a difference for these guys. IBM was measuring it like you were lifting bales of hay or something. It was just- Michael Salfino (00:17:15): Exactly. Yeah, that's interesting, how the same thing could apply to fields that on the surface appear to be completely different, the same objectives. Rob Collie (00:17:26): Yeah. We see that everywhere. Clients or perspective clients are very often asking us, "Well, do you have any experience in our industry?" And the answer's always yes. I mean, we have experience in every industry. Michael Salfino (00:17:36): Right. Rob Collie (00:17:37): But the harder thing to appreciate is that it turns out that our experience in other industries is going to be just as important. I'll give you an example. Many, many years ago, I did a project for one of the big four accounting firms on, I forget if it was FIFO or LIFO, the first in first out or last in, first out accounting. So we were doing these accounting financial models, basically to assess tax liability. And then years later was talking to someone who worked in a food warehousing business about their spoilage model, when food would go bad in the warehouse. And it turned out it was exactly the same problem. Michael Salfino (00:18:15): Right. Rob Collie (00:18:16): It used exactly the same formula, the same techniques. And we just like copy pasted that whole pattern. And the industry isn't actually the thing. There's almost like these forms of problems that you do start to see. Michael Salfino (00:18:29): Right. Rob Collie (00:18:30): And it's fascinating. Michael Salfino (00:18:31): In my field, the less I know about a subject, usually the less likely it is that I will make a mistake in that article. So in other words, if I'm writing about something that I'm intimately familiar with, I may rely a little bit on my memory and just flag something very easy, or just not explain it well enough, or skip over something. But if I don't understand. So when I write about hockey, which I really don't have a passionate interest in, those articles tend to be the clearest examples of my writing because I almost have to explain it to myself before I could explain it to the reader. And that's what I try to do you anyway, that's my process. But it's so much easier to do when you literally do not know much about the subject matter than it is when you are so well versed in it. Rob Collie (00:19:20): Well, I'm feeling a little bit less indicted now. Because I have been, for example, critical and took a lot of heat, within our little community anyway, for a number of years about being so critical about the way our industry operates, which is, at least traditionally, you would always first go and build a data warehouse before you'd ever put a dot on a chart. And I've been calling BS on that for a very, very long time. And I had enough of sort of bonafides from my background at Microsoft that I sort of felt okay. Rob Collie (00:19:49): But there's like this 1,200 page bible written by Kimball. And you must always say his name in hushed tones. It's like a priesthood- Thomas LaRock (00:19:59): Wait, you're a Kimball guy, not an Inmon? Rob Collie (00:20:02): Exactly, right. I have no idea. I consider the fact that I've never read this 1,200 page book to be a qualification her than the disqualification. And that goes back to some things you were saying earlier, Mike, which is that if you consume too much of your own industry, you become much more vulnerable to groupthink. Michael Salfino (00:20:22): Yes. Rob Collie (00:20:22): And groupthink isn't always bad, but it almost always is. So I like that. I like that you don't have a problem differing from the consensus. Every year going into NFL season, there is a consensus, one through 10 of who the top players are. Michael Salfino (00:20:40): Right. Rob Collie (00:20:41): That the industry just happens to just magically arrive at as if it's the right thing. But it's really just like the kids walking in Dead Poet's Society, how they just automatically fall into a marching pattern with each other, right? Michael Salfino (00:20:52): Well you'll notice, once you're sort of in the industry and like completely OCD about fantasy sports, that it's the drafts that take place at the very early part of the season, before ADP, which is average draft position, is even remotely cemented where you have such a freedom. And a lot of people are intimidated by it and a little bit afraid of it because they don't want to go on record over-drafting or under-drafting a player, especially over-drafting a player, like taking him too high based on where he eventually settles in his ADP. But I find it liberating to just be able to draft the players based on the value that I think they have rather than what that sort of consensus groupthink is. Rob Collie (00:21:36): Groupthink runs business, largely. Thomas LaRock (00:21:38): Absolutely. Michael Salfino (00:21:39): Yeah. Rob Collie (00:21:39): Escaping that is a little bit scary. It's kind of like the coaches that used to not, and a lot of them still don't, the coaches in football who, or really in sport, who didn't follow analytics. We used to always think why don't they go for it on fourth down, the percentages are great. It's just that their goal isn't necessarily to win the most games. Their goal is to not get fired. Michael Salfino (00:22:02): Oh yeah. Scott and I ... Scott Pianowski of Yahoo, who's my partner on the Breakfast Table, where you came into contact, I think, with our work. We've always written about that, about how coaches ... and this happens in business as well. They'll go for the quiet loss rather than take the chance, increase their win probability. But in the process, in defying the conventional wisdom, create a scenario where they will be the subject of much more intense criticism if they fail to win. Which, in either case is going to be the extreme likelihood anyway. Rob Collie (00:22:39): Mm-hmm (affirmative). Yeah. Michael Salfino (00:22:40): But they'd rather just go down in a way where the reporters aren't going to ... I always say it's like when you're playing a game. Say you're playing Strat-O-Matic, or even Madden or a video game or whatever it is, with sports. You want to win. You really want to win. But you're going to do things that just maximize your win probability, intuitively. It's not that complicated to know that if it's fourth and two, you should just go for it. Right? Michael Salfino (00:23:05): But teams ... We don't have to worry when we're playing a game that somebody is going to say that we should be fired and never be allowed to play the game again. So I think that that is so constraining on ... and I wouldn't even call it creativity or courage, but it's almost like a willful ignorance of maximizing win probability just because you are afraid of losing your job, which maybe is rational. Rob Collie (00:23:32): Yeah. It is. It is. The lesson here isn't for the individual. In business, for instance, if we were going to take a lesson away from this. It's not for the individual. We're not saying, "Hey, you should be more courageous." This message is actually more for the managers, that they need to create an environment in which the right outcome is optimized for. Michael Salfino (00:23:51): Right. Rob Collie (00:23:52): It's much more about the environment, because the environment does create different incentives. It creates incentives that are different. Don't stand out, don't get fired. Right. I think people are intuitively doing the math in their heads and coming to the right conclusion, in general. Don't stand out, you know? Michael Salfino (00:24:08): Yeah. Thomas LaRock (00:24:08): There's a reason Norv Turner held onto his job for as long as he did. Rob Collie (00:24:15): We got to give Mike a chance to react to that. You know? Michael Salfino (00:24:18): Well, I think with the birth of analytics in sports that maybe the coaches are going to become, especially in the other sports, maybe not necessarily in football, are going to become more like middle managers, right. And the general managers and the analytics department are going to be more of the key decision makers. And I think that's a double-edged sword. I think the teams will generally be better. But the problem with analytics as it relates to a field like sports, is that it fosters an environment where really teams start playing the same way. So some of the variability that existed previously, especially like in a sport like baseball or basketball, where teams had stylistic ways of playing the game that was inherently interesting to watch. And now analytics is kind of smoothed that out so that teams generally play the same way. Michael Salfino (00:25:11): They may not play the same way as well. And that's obviously the difference between success and failure in the individual sports. But everybody is basically playing the same game in the NBA. And they're definitely playing the same game in Major League Baseball now. And for a while that was the situation in hockey with the Devils where everybody just dumped and chased the puck. And football, I think we're being saved right now by the Ravens, who have managed to find a high level of success in playing a fundamentally different brand of football, mostly because of a quarterback that has unique skills. Thomas LaRock (00:25:46): So re you in one way saying analytics is making sports boring? Michael Salfino (00:25:50): Yeah. And I talk about this with my colleagues all the time. I think baseball has a natural ability to defy the analytics on a team level and maybe find, ironically, better analytics and being better able to exploit market inefficiencies because there's the variability of the ballpark geometry. And so you can theoretically create a ballpark that is going to hurt every other team that is playing the same analytics way while just helping you. You could have 470 feet to center field, like the old Polo Grounds, 360 feet down the line, 390 feet in the power alleys. And it's like, just try to hit home runs here, guys. It's not happening. Michael Salfino (00:26:36): And then you build a team based on the things that are no longer valued in Major League Baseball, or as valued, which would be defense and base running and stealing. Strikeouts would obviously still be valuable, but you could even have more of a contact staff, especially a fly ball staff with all that expanse in the outfield. But nobody really wants to do that. The Mets were sort of situated like that with with Citi Field when it first opened. And then the hitters complained and they just shortened the fences. Rob Collie (00:27:09): You need a ballpark that's got those movable walls, you know? So every single game you're adjusting the outfield fence. You're like, "Oh, this team's got a lot of lefties." Michael Salfino (00:27:19): Yeah Exactly right. Right. Yeah. Rob Collie (00:27:23): A variable geometry ballpark. Michael Salfino (00:27:24): Which is the way ballparks used to be, just because of the fact that they were in neighborhoods. And they were limited as far as the geometry is concerned to whatever the surrounding areas were. But it made for, I think, a more interesting game. Then we got into the cookie cutter thing with the ballparks in the 70s, which were all astro turf and cut the same way for dual purpose football and baseball. And now we're at a point really where the home run is king. And so all these parks are much easier to hit home runs in, generally speaking, than they used to be. Rob Collie (00:27:54): Let's go back a little bit. We started off talking about yards per pass attempt. Not everyone here is into football, but I still think, again, going back to that thing we were talking about, the cross domain experience is oftentimes even more valuable than in-domain experience. For about 20 years, at least 20 years, as long as I've been paying attention, in the football world there's been this ongoing argument. And I think it's largely been resolved now, but it took forever. And it used to go like this. So there's basically two kinds of plays you can run in football. You can hand the ball to someone and have them run, and try to block for them. Or you can drop back and you can throw the ball through the air. So there's passing plays, which is throwing. And then there's rushing plays, which is just running. Rob Collie (00:28:40): Obviously you can make bigger chunks of yards. You can throw the ball a lot farther. And a really good pass is going to generate a lot more ... you're going to go a lot further down the field than a really good run, on average. And yet conventional wisdom forever was that a powerful running game was the number one thing you could have in football. It was the most valued thing, by a lot of people anyway. And if you went and you looked at the statistics, you would see that the number of yards you accumulate rushing per game did correlate very strongly with how often you won. But there was a problem with that, right? What was the problem? Michael Salfino (00:29:21): Well, first of all, there were a couple problems with that. But the main problem that is just maddening to me as a football fan in 2020, that you get professional analysts and coaches still talking about when Emmett Smith or now I guess Ezekiel Elliot gets 20 carries or more i a game, the Cowboys never lose. Or their record is incredibly like a .900 winning percentage. But that's just a product of already having won the game, usually with your passing game. And so you're just increasing the rushing attempts. And it's just so obvious. I swear I knew this when I was 11 years old that well you're winning, so that's why you're running. But the interesting thing is if you look back in the history of football and there's this guy, Bud Goody who's I guess sort of like the Bill James of football, he passed away probably within the last decade or so. But he worked until like his nineties, working with coaches and with teams. And starting in the mid-60s he developed the yards per pass attempt stat and found its correlation to winning. Michael Salfino (00:30:29): And if you look back, even on the Vince Lombardi Packers in the super bowl era, and that's the epitome of like running to win, is Vince Lombardi. He's like the Supreme example of that. But they actually dominated, if you look at the statistics, at an historic rate in yards per pass attempt for, minus yards per pass attempt allowed, which is sort of that net YPA stat. They were about four yards better gaining than what they allowed. Which if you're like two yards better, you're one of the 98th percentile teams in NFL history. So that's just a ridiculous margin to have. So even going back to the Dawn of the Super Bowl era, it really was a pass dominated league, but nobody really knew it. But now there's just no excuse not to know it. Michael Salfino (00:31:24): So one of the things that I did with the Journal, we had access to a stat service. So I wanted to see, in the intervening period, which was many decades, whether this 75% correlation of winning yards per pass attempt in a game, and we call it net yards per pass attempt because it includes sack yard in passing yards, and sacks as pass plays. I wanted to see if that 75% threshold was still true. And so they went all the way back to the merger and it was like 74.6%. So if you don't know anything else ... and that's way better than turnover differential, which is another stat that's cited by people who want to find correlations to winning. Because turnover differential is, as you know, so much more extreme than any differential. And not even relevant to every game, because you could have the same number of turnovers, where net yards per pass attempt is always going to be won, even by the smallest margin, by one of the teams. Michael Salfino (00:32:23): So this includes every game, and no matter what the margin, 75% win probability for the team that wins that stat in each game. So obviously teams, if they were rational, would be building teams to win the battle of the passing game. They would be focusing on receivers, pass protection, and quarterback on offense. And they would be focusing on pass rush and cornerbacks and defensive backs on defense. And they wouldn't really care about anything else, especially in a salary cap situation where your resources are limited by the rules of the game. Thomas LaRock (00:32:57): There are so many factors, and especially in football. I remember Wayne Winston talking about how if you could ever figure out what 11 guys to put on the field for any one particular point in time, you'd make a lot of money because nobody knows how to do that. But it's also the game situation. I think you talked a little bit about that. But just the net passing yards itself, there's so much that go into it. the down, the distance, the clock, everything about it. And unless you factor that in, you could talk about 75% of this, whoever wins this stat wins the game. but I'm like, there's so many points in time for you to get to that. There's more layers than just looking at any one stat and correlating it with a win, in my opinion. Michael Salfino (00:33:39): Sure. But there is going to be probably one stat that's more important than all the other stats. So even though there are many exceptions and there are ways where maybe you can distort that statistic. Garbage time, by the way, is not one of them, which is something I learned in working with Rob's data back in his Microsoft days, which I found very interesting at the time, which has never changed where it's actually harder for teams to generate yards per pass attempt in garbage time than it is in regular time. So that's why the efficiency stats are the stats that you should be looking at. Rob Collie (00:34:13): We're kind of microwaving this meal, and I want to slow cook it for the listeners. So let's slow it down just a little bit. Right? So we started off by saying that, Hey, in the dark ages 20 years ago, and it does persist to this day, which is still maddening, I agree. But 20 years ago, except for a few outliers, everyone believed that a powerful running game was the key to winning. And if you looked at the surface of it, like hey the teams with the most rushing yards do tend to win more. Rob Collie (00:34:44): But as you said, I want to make sure that the non-football crowd knows this. In a game, once you've got a sufficient lead on the other team, you start running the ball a lot more often than you start passing the ball, simply because of this dynamic in football where the clock keeps running at the end of a running play. But a lot of passing plays result in the clock being stopped. So the way to run out the clock is to run the ball. Michael Salfino (00:35:10): Right? Rob Collie (00:35:10): And so you tend to ... every rushing play tends to be positive. So if you're running the ball more, you're just going to amass more rushing yards. Michael Salfino (00:35:19): The way it's said is, and I ascribe to this, is that you pass to beat the opponent and you run to beat the clock. Rob Collie (00:35:28): That's right. That's right. Okay. So this is an example of a metric that if you just looked at it ... and keep in mind, this is an industry that's been around for a very, very long time, with a tremendous amount of money on the line. Michael Salfino (00:35:40): And scrutiny. Rob Collie (00:35:42): You would think that "the market" would've figured this out a lot faster than it has. And as you point out, there are still some stalwarts who get on there and parrot the old line, the old line about the running game. Okay. So people started to slowly figure out, okay maybe it's not the running game, maybe it's the passing game. And so then you would naturally go, okay, it wasn't rushing yards. Let's look at passing yards. But this was also a little bit misleading because what happens when you fall behind in a game, if you're trailing, you tend to throw the ball a lot more, so you generate more passing yards that way. Rob Collie (00:36:22): So this yards per pass attempt, it starts to factor in. And I think this is the thing, Tom, that you were saying, yards per pass attempt is actually multiple metrics. You're integrating a lot of information. Because the number of attempts that you throw is indicative, especially late in the game, is indicative of the game situation and the time that remaining. A Lot of things just sort of get accounted for. But then there's also the yards per pass attempt that your defense allows. And so now you've got the defensive side of the ball. And so if you have your offense yards per pass attempt on the positive side, your defense is sort of negative, right? And you balance those out. And if your net yards per pass attempt, defense versus offense, is positive. That thing that Mike talked about, the old Vince Lombardi teams had a differential between their offense and defense yards per attempt was four, four yards per attempt. That is a massive number. That is just ridiculous. Michael Salfino (00:37:24): And the thing about it is, that's great about the stat is that it's a power stat. You maybe aren't utilizing that power, but that power always exists for you to use. So if you are generally winning the battle of the passing game or if you have great team passing efficiency, even if you don't pass a lot, like the '72 Dolphins famously were a running team, but they passed at an extremely efficient rate. But they controlled whether they felt like doing that or not. You know what I mean? You couldn't stop them really from passing. It's just whether they chose to. And that's still the case in football. Michael Salfino (00:38:00): Russell Wilson used to be a great example of this, not necessarily anymore, because he's having what's on pace to be an historic season. But here's a guy who was always extremely efficient as a passer, but the Seahawks just wouldn't pass that much. But whenever they needed him to pass late in games, when they were trailing, they just unleashed Russell Wilson and would win. Now you could say, why not unleash Russell Wilson early in the games and just blow teams out of the water. So now they're being forced to, by a subpar defense. But there was nothing stopping them from just beating teams into oblivion with Russell Wilson early in the games in the past, except for just poor decision making. Thomas LaRock (00:38:38): So I think another layer here that is especially true for football, it's really your opponents matter a lot because the schedule is balanced only by division. So you know, this year my Patriots get to play the NFC West, and we have to go on the road to Seattle and somebody else. So that's not the same. It's not even the same for everybody in my division because the Seahawks will play at a couple of AFC East teams. So it gets close. You get to play each other. And maybe it'll be more equal without a crowd. But the opponents matter so much when you're looking at a stat like this, there's just no question. Thomas LaRock (00:39:12): But you mentioned the dolphins and I would just say that's a great example of a team that was so good at running. Every other team, all their opponents would try to focus on stopping the thing that would kill them the most. And you would gamble that you wouldn't get beat with the pass, but they did. And it's like that in every sport. I watch my Celtics get beat. I say, you know what if Iggy makes four, three pointers and we get beat, I'm willing to give up that chance because I've got to guard these other guys. And you know what the damn guy sticks those three pointers and we're going home. And that's a huge thing because at some point you have to take a chance. Michael Salfino (00:39:46): Because you're maximizing the probability. Thomas LaRock (00:39:48): Exactly. Michael Salfino (00:39:48): Yeah. So it's rational. Thomas LaRock (00:39:49): But it leads to this ridiculous stat where people try to tell me how great the '72 Dolphins were at passing. And I'm like, but they weren't. Michael Salfino (00:39:56): Well, I think they maybe could have been if they did it more. It's very interesting, the whole chicken and the egg thing. A theory that I subscribe to was that your ability to run well did not increase your ability to pass well, that the two areas were, despite conventional wisdom and how easy it is intuitively to accept the fact that, well, if you're running well teams will jump your running game and then you could throw behind them more easily and you could trick them. But when you really think about it, if a defense is stuffing your running game, there's nothing they want to do more on earth than to keep stuffing your running game. So like why wouldn't a play action work, even if they're having a high level of success and limiting your running game. There's blood in the water, man. Those guys just want to crush you. Michael Salfino (00:40:41): And it turned out, with the more sophisticated data that we have now and utilizing my friends Cade Massey and Rufus Peabody and their excellent Massey-Peabody site, that there really is no correlation between run success and the ability to throw. There's really no correlation, which is extremely counterintuitive, between running the ball well in a game and the success of your play action passing game, where you're faking a run. Rob Collie (00:41:07): Yeah. So I remember following you for a little while when you were talking about this, this wasn't recently- Michael Salfino (00:41:11): Oh, I've been talking about this since like 2000. Rob Collie (00:41:14): Yeah. I'm going to harvest a memory from a while ago and this just might no longer be state of the art for you. But it's something that I remember you talking about a while ago, which is that it was really just your willingness to try running it, that set it up. Like if you were willing to run every now and then just to kind of keep them honest. Michael Salfino (00:41:31): Yes. Rob Collie (00:41:31): Even if you were terrible at it, as long as they were guessing as to what the play call was going to be, and they were off balance, it didn't really matter whether you were getting three yards every time you ran versus getting four and a half. Michael Salfino (00:41:44): And a lot of it was the quality of the fake. Rob Collie (00:41:46): Yeah. Yeah. Michael Salfino (00:41:47): You know, because it's a game of deception. Rob Collie (00:41:50): Here's a really nerdy thing that I love. And it's a bit of a reach, but I have to bring it up. So if you have position ... We're talking about derivatives for a moment like physics, calculus derivatives. If you take the first derivative of position over time is speed. The change in where you are in are, in a unit of time, that's speed. Okay. The second derivative of position, with respect to time, is acceleration. The first one's how fast is your position changing? The second one's how fast is your speed changing? Now the third derivative of this, how fast is your acceleration changing? The only word we've got for it is jerk. If you're sitting somewhere and suddenly the acceleration of your body changes rapidly, that's when you tend to lose your balance. It's an unexpected change in acceleration. Rob Collie (00:42:36): So only living things with brains and nervous systems care about jerk. Jerk has no physical impact on anything. Buildings aren't more likely to fall over from a high jerk. They only care about acceleration. So it's just this interesting thing that even in physics, when you start taking more derivatives, you get to a point where the living thing's ability to anticipate becomes crucial. It just like the ability to anticipate comes out, even in basic physics, basic calculus, when you're running derivatives like this. And it just seems like to me ... Again, I'm a 2:00 in the morning in the dorm room kind of nerd, those kinds of conversations. And to me that's the same thing, right? If you can't anticipate what's coming, that's all that it takes. That's the real value of a running game in football. Michael Salfino (00:43:26): And the same thing in baseball with the changeup and the fastball. Or you could see now and you could see it even with the pitchers of a generation ago, when the arm slot matches perfectly and the release point is the same and they do the overlap of the two pitches you could see, from a hitter's perspective, how pitches that end up at wildly different places in the strike zone appear to be exactly the same for 7/10 of the journey to the plate. Rob Collie (00:43:56): Yeah. Information processing. Michael Salfino (00:43:58): Yeah. Rob Collie (00:43:58): It's the same thing why in the history of warfare weapon has always beaten armor. There's never been armor that came out that defeated the weapons of the age because the weapon gets to choose where it hits, and the armor doesn't. The weapon has an information advantage, right? Michael Salfino (00:44:16): Yeah. It's kind of like what we were talking about before with passing. Rob Collie (00:44:18): Yeah. Michael Salfino (00:44:19): Yeah. Rob Collie (00:44:19): So I got one other football thing that I thought would be neat to talk about. And again, it's one of these counterintuitive things. The most devastating single thing that a quarterback can do, even this I might be uncertain of, but if you're a quarterback and you throw a lot of interceptions, you're constantly throwing the ball to the wrong team. Michael Salfino (00:44:39): Yeah. Rob Collie (00:44:40): That is really going to hurt you. Michael Salfino (00:44:42): Not all turnovers are the same. Interceptions have a much greater ... and that's one of the things that I talked about early on in the Breakfast Table, which nobody really knew, could articulate why. But interceptions are much more costly than fumbles. And it's got nothing to do with return yards. Rob Collie (00:44:58): Right. Interceptions are a much better predictor of failure than fumbles. Michael Salfino (00:45:03): Now I have a theory about that, but you never know if your theory is right. I remember I went to a symposium at the Museum of Natural History. And this was said by the physicist who was giving the lecture. But his whole point was that facts are basically meaningless. They're so low in the pecking order. They're just things that are out there. And we tend to think that facts are more important than theory, right? Because people will say, "Oh, you have the theory of evolution, but it's not a fact." And it's like, no, the theory is over the fact. facts need a theory, otherwise they're just things that are just floating out there. So unless you have a coalescing theory as to why those facts exist, the facts don't even really matter. Thomas LaRock (00:45:49): Wow. That's deep. Rob Collie (00:45:50): It is. Yeah. So it's not the dots, it's the lines. It's the lines you draw through the dots that are the important thing. Yeah. I like that. I'm going to be going around saying that, with my fingers steepled, for the next 18 months. That's great. Michael Salfino (00:46:12): There's certain things that happen in sports. For a while, Ben Roethlisberger was horrible on the road. And it's like, well, okay. Drew Brees being horrible on the road can be explained by the fact that he plays indoors. Why would Ben Roethlisberger be horrible on the road? But it went on for like two years. And then it ended in a sudden flash of great road performances in 2018. But I never really pushed it, like so many of my colleagues did, because they were obsessed with the fact of how Roethlisberger was playing on the road, where I could not find a rational theory to explain why he was playing poorly on the road. So I just ignored it. I just figured it was noise. Michael Salfino (00:46:54): But I was always open to the idea that somebody would come up with something where I was like, "Yeah, that makes a lot of sense. I could buy that." Rob Collie (00:47:00): Oh, interesting. You were open to the existence of a theory. Michael Salfino (00:47:03): Yeah. Rob Collie (00:47:04): But because we didn't have one- Michael Salfino (00:47:06): I just ignored the data. Rob Collie (00:47:07): You just ignored it. I don't know that I'd be that strong. I do get fooled by randomness a little bit. Plus you oftentimes just don't get time to wait for statistical significance. Michael Salfino (00:47:18): As far as the randomness, maybe it was in The Drunkard's Walk or whatever, the book on randomness, I don't know, wherever I read this. But it's basically that if something is random, it's always going to have the appearance at some point of not being random. You just have to have the discipline to just ignore it, unless you could figure out a reason why something is predictive and not merely descriptive. So if you're flipping a coin, you know you're going to have a stretch where it comes up heads or tails like 80 out of a hundred times, if you flip it long enough, right. Or eight out of 10, let's call it. Maybe that's more likely. But it won't mean anything, because we know it shouldn't mean anything. Michael Salfino (00:48:00): In the real world we don't know if those things are random. So if a CEO has been a CEO of one company and they have a great product, was he just randomly in the right place at the right time? Or was the company successful because of his or her leadership? It's kind of like the Pat Mahomes situation, right? Is Pat Mahomes a 95th percentile hall of fame quarterback, no matter where he would've been drafted? Or is he a hall of fame 95th percentile quarterback because he was put into a 95th percentile situation for a young quarterback? Thomas LaRock (00:48:39): That's also deep. And I'm hoping that he is 95% because I have him starting tonight. Michael Salfino (00:48:44): Well that's the thing. The environment is still 95th percentile, right? Like he's got great players all around him and a great offensive coach. And it's harder to fail in a situation like that. See, like in football, there's no true skill level, I maintain. Or maybe, obviously, Pat Mahomes is highly skilled. So I shouldn't say that there isn't any. But what I'm saying is it's not transferable, like it is say in baseball .I could put Mike Trout on any team and he is Mike Trout. I put Pat Mahomes on the jets, he may not be Pat Mahomes. Rob Collie (00:49:16): That's true. Michael Salfino (00:49:17): He probably won't be. We actually can say definitively, he would not be Pat Mahomes. We just don't know the extent to which he wouldn't be Pat Mahomes. I mean, would he be Sam Darnold or would he be something much better? Thomas LaRock (00:49:33): Well, can we just use LeVeon Bell as the test case for this? Michael Salfino (00:49:36): Sure. Yeah. But a lot of people would say he took a year off, he's older. But I think you're ... I would agree with you, that the environment fundamentally changed. There was really no explanation otherwise as to why his performance would decline as precipitously as it did. Thomas LaRock (00:49:52): So can I ask one more football question? Michael Salfino (00:49:54): Sure. Thomas LaRock (00:49:55): Because I'm going to challenge you on something, because you mentioned earlier how garbage time doesn't matter. Michael Salfino (00:49:59): Sure. Thomas LaRock (00:50:00): I was going to bring this up as well, because in basketball it, was a Bill James? They have, the lead is safe. Is this lead safe? You're up by so many points. There's so much clock and so many possessions. Michael Salfino (00:50:11): Yes. Thomas LaRock (00:50:11): Football doesn't have that equivalent, is this lead safe, that I've found. But let's just say that existed. Michael Salfino (00:50:19): Well, they do have win probability. Thomas LaRock (00:50:20): But let's just say this existed, that there's a way where you'd say this lead is safe. I'm up by two plus scores with two minutes to play and- Rob Collie (00:50:29): Guys, we have to make a Falcons joke here. Any assessment of when a lead is safe has to start with the question, yes/no, are you the Falcons? Michael Salfino (00:50:36): Here's a stat. Here's a stat that's amazing. The Falcons are the first team in NFL history to lose two games where they had a 17 point lead in the second half, or at any point in a game, in the same season. And they did it in consecutive weeks. Consecutive weeks. Thomas LaRock (00:50:51): That's awesome. Michael Salfino (00:50:52): Is that random? Thomas LaRock (00:50:56): No, no. I think it's a trend. So here's my counter example for you. And it goes back to the early 2000s. So maybe you might remember this. But there was a quarterback in the league at that time. And I was the commissioner for our fancy football league, and so I was in tune with all the games and I would do writeups for the games. And this quarterback excelled in garbage time. Michael Salfino (00:51:20): Mark Bulger? Thomas LaRock (00:51:21): No. Michael Salfino (00:51:22): Okay. Because that was the one that we studied back around that time, with Rob. Thomas LaRock (00:51:27): This one that I picked up on was Aaron Brooks. Michael Salfino (00:51:31): Oh yeah. Okay. Aaron Brooks. Thomas LaRock (00:51:32): He played for the saints. He would have horrible stats for three quarters, but in the fourth quarter he was the king of the fourth quarter. He'd pick up 14, 15 points. It was all garbage time points. By the end of the game, he'd have 20, 22 points. And as you know, you compare quarterbacks, how did my quarterback do against the other quarterback. I had 22, he had 25. So it was even. Aaron Brooks always came out, at the end of the day, fairly competitive with whoever he was lining up with. But his NFL team was horrible. Everything about it was horrible. The only way he score points was just in complete garbage time. So when you mentioned that garbage time doesn't matter, I'm like, "I don't know." Michael Salfino (00:52:08): But you see how it could matter in fantasy. I'm never going to say it doesn't matter in fantasy. Thomas LaRock (00:52:14): Okay. Michael Salfino (00:52:15): Because you want pass attempts. And we always say the greatest thing that could happen to your fantasy quarterback, unless you have like an insane tax for it in terms of negative points, like I do in one of my leagues, is the pick six, because he comes right back on the field and now he's got to throw even more. Thomas LaRock (00:52:31): Yeah. Michael Salfino (00:52:31): So it's like manana, right? But we're talking about different things. You're talking about the scoring of fantasy points. I will not deny that garbage time can significantly increase a quarterback's fantasy scoring. Like Matt Ryan on the other end, when he's trailing, is a perfect example of that. He's the king of that right now. But what I'm saying is that prevent defense's work. It's harder to throw. So your efficiency stats will be worse, not your counting stats. Thomas LaRock (00:53:02): Right. So yeah, two different things. Gotcha. Rob Collie (00:53:04): Something you were saying earlier, Mike, that I really wanted to dig into. The only thing, and I mean this, the single thing that I learned in college that I considered to be useful, one thing. It was in a psychology class and it's this thing called attribution bias. And I think it might be called different things. But basically it's like when we look at someone's performance, whether in life or in relationships or in sports, we tend to human beings, we tend to overwhelmingly allocate their success or failure to attributes of the per person. Michael Salfino (00:53:45): Right. Rob Collie (00:53:46): We under count, dramatically, the input and the impact of their situation. Michael Salfino (00:53:54): Right. So like with Mahomes, what we were talking about. Perfect example. Yeah. Rob Collie (00:53:58): That's exactly right. It goes way beyond sports. It's everywhere. Michael Salfino (00:54:02): Well, like the CEOs that I was ... the hypothetical CEO that I was talking about. Rob Collie (00:54:06): Exactly. Right. And so this actually comes back to also the fooled by randomness thing as well. Right? How do you separate? And I agree with you that football is a wonderful example of this. Pat Mahomes can only be Pat Mahomes in a situation that is going to be conducive to him. With Jimmy Johnson of the Cowboys' dominant era, would Jimmy Johnson have allowed Pat Mahomes to be Pat Mahomes? Probably not. You know? Rob Collie (00:54:35): And also, here's one that Tom will appreciate. Tom Brady, who, as far as I know, is not known for his levels of graciousness. I don't think that's something he's known for, has said that Aaron Rodgers would be amazing if he got to play for Bill Belichick. Michael Salfino (00:54:54): Yeah. Rob Collie (00:54:55): Better than me, way better than me. You have no idea how many titles Aaron Rodgers would win playing for Bill Belichick. Michael Salfino (00:55:01): Which is why Mike McCarthy should be indicted for his head coaching. And he's doing the same thing in Dallas you got Dak Prescott throwing for like 7,000 yards a game. Thomas LaRock (00:55:09): Hey, just relax. We're okay with the job he's doing for Dallas. Michael Salfino (00:55:14): Because they should be 0-3. Think of how much happier the world would be. There would be a much higher level of happiness worldwide if the Cowboys, who are so close to 0-3, were actually 0-3 right now. 1-2 is good, but we would be dancing in the streets if the Cowboys were 0-3, like 95% of the world. Thomas LaRock (00:55:34): That's an example of the schedule though. They were fortunate to play Atlanta. That's all. Rob Collie (00:55:39): Yeah. Two forces conflicted. Atlanta must always lose. Michael Salfino (00:55:43): That's one of the 2-9-1 wins. The 2-9-1 NFC East, that's one of those wins. And the other one was NFC East versus NFC East, is the Redskins of all teams. What if I told you the Redskins would have the other win? Rob Collie (00:55:57): Wait, I don't- Michael Salfino (00:55:57): Well actually the Washington Football Team. Rob Collie (00:55:58): That's right. Michael Salfino (00:55:59): I did this in the chat too. I am so sorry because I am so ... I totally believe in the politics of the Redskins name change. And I actually like ... I made the joke on the FiveThirtyEight chat two weeks ago that it's like The Band. They should just call it the Washington Football Team. Thomas LaRock (00:56:16): Yeah. I agree. Michael Salfino (00:56:16): It's a great name. That is so cool. So I hate when I slip, but I can't ... there's a ghost in the machine, man. It's been too many years. I can't let that go as easily as I want to, morally. Rob Collie (00:56:29): Mike, you've also written, at least in the past, I think you've written for some psychology magazines. Isn't that right? Michael Salfino (00:56:34): I did. I wrote an article about facial recognition and why witness testimony me is completely invalid, because we're face blind to different races. And I wrote an article about the movie rating system and how basically everybody was gravitating towards PG-13, and violence was not something that they used to assess the suitability of a movie for an age group. So they were just jamming in all the violence that they wanted to. And the only thing they had to worry about was using the F word in a sexual context. Rob Collie (00:57:16): Or more than once. Michael Salfino (00:57:17): Or more once. Rob Collie (00:57:18): Because you're allowed one non-sexual F bomb. Michael Salfino (00:57:21): You are. You are. Exactly. Thomas LaRock (00:57:24): So, in the show notes, can we link to these articles? Can we remember to do that? Michael Salfino (00:57:29): Those are old ones. Thomas LaRock (00:57:30): I don't care. They sound fabulous. Michael Salfino (00:57:32): I think the face blindness was related to the Duke ... It was pegged to the Duke rape scandal where somebody was criticized of a different race for not being able to recognize the faces of the accused, white assailers. But that's just a natural product of the face blindness with race that I cited, based on other people's research. Rob Collie (00:57:58): So you also write for FiveThirtyEight. Do you stick to sports on FiveThirtyEight? Michael Salfino (00:58:02): I do. I would love to write about politics, but I could not be trusted. It would be like writing about the Jets. I'm too partisan to write about politics. Rob Collie (00:58:13): Being partisan, it's so rare in this day and age. I mean, you're such an outlier. Thomas LaRock (00:58:23): Most of us are just middle the road. Rob Collie (00:58:24): It's actually almost hard to trust someone that isn't partisan these days. Right? Michael Salfino (00:58:28): Nowadays, yeah. You know, it's so weird. I won't say really like what side I'm on. Although I guess you could kind of figure that out. Rob Collie (00:58:38): You could probably infer it. Michael Salfino (00:58:39): And this may be a male, female thing. I am far from a social butterfly. My wife is very socially outgoing. She had a big problem in this hyper partisan environment with having a lot of friends who were on the other side, and it was very upsetting. I had no problem. Every single person ... and it's not that many. There must have been some sort of test that had nothing to do with politics that I used to ferret people out, so that there was no one I knew who was on the other side. And you know, this is many generations of friends and probably 20, 25 people in total, at least, who I've stayed in pretty close contact with. To be 25 for 25 is kind of crazy. Rob Collie (00:59:28): That's awesome. Michael Salfino (00:59:29): But there's got to be something. I don't know if it's a male, female thing. I don't know if it's a social, anti-social thing. But maybe it's being judgmental. Maybe it's a bad quality in most cases that actually accrue to my benefit in this circumstance. Thomas LaRock (00:59:47): Doesn't that lead to confirmation bias? If you're surrounded by- Rob Collie (00:59:52): Yeah, it sounds like groupthink to me. Michael Salfino (00:59:52): Probably. Probably. Yeah. Rob Collie (00:59:56): They're all taking McCaffery at 1.1. All right. So politics though, right? Without the content of the politics, let's talk about sort of the analytics side of it. Michael Salfino (01:00:09): Yeah, just the numbers. Yeah. I love the numbers, man. This is at the sports level for me. So, go. Rob Collie (01:00:16): All right. So I've given you a little hint of this in the past, but I have a bit of a controversial ... I don't think it should be controversial, but its proven to be every time I bring it up with smart people. I mean, I get attacked. I get labeled as anti-intellectual and written off really quickly before they even really have a chance, I think, to hear what I'm really trying to say. Let me present the thesis in the least controversial way possible by saying that analytics, when applied to presidential elections, it's just bullshit. It just doesn't work. You know, it's like that old saying, so when you slap a probability on an event, like a 65% chance of this person winning. I think it's like The Naked Gun joke of he's got a 65% chance of winning, but only a 40% chance of that. Rob Collie (01:01:07): I think in situations where we have the benefit of many trials and many sort of similar circumstances, I think we have a much better chance at developing models that put a percentage chance on an outcome that is about as good as it can be. I don't think that any of our percentage predictions about something as big and as uncommon as a presidential election, I don't think it's that kind of game. I just don't think it is. I think the assumptions that the human beings building the models, assumptions that they're forced to make in order to build the models, are themselves the vectors of so much noise. And then combine that with the fact that on the day of the election, we'll say something like there's a 65% chance, before the first votes are cast, right? There's a 65% chance of this person winning. Rob Collie (01:01:54): It's not the same as a sports game. A Sports game, you don't know whether that ball is going to be ... when it's spinning and the guy tips it, is he going to hit the strings of the laces of the ball or not. That's going to change the trajectory of the tip. And the whole game can turn on something as small as that and completely random. Whereas, on the morning of an election, who's going to win, I think it does not hang on anything remotely like that. Who's going to win is already sort of known to the universe. It's just not known to us yet. Michael Salfino (01:02:28): I see what you're saying. But don't you think that the margin of the outcome can be random or can be attributed to randomness? So in other words, Trump won by 77,000 votes across the three states. If we had that election again, without anybody saying, "Oh my God, what did I do? I didn't even think Trump was going to win. I would've definitely not voted for a third party candidate," or any stuff like that. If we just had it again, without any fore knowledge, do you think that Trump would've won those three states by that combined margin again? Do you think that was just set in stone for perpetuity? Rob Collie (01:03:07): Well, I mean, it's sort of like the law of large numbers, et cetera, right? The handful of small, random events that lead to someone voting or not voting or changing their mind- Michael Salfino (01:03:18): Maybe there were 100,000 people who wanted to vote, but just got into an argument with a spouse or- Rob Collie (01:03:26): Right. Michael Salfino (01:03:26): Or the kid broke his leg. Who knows what could have happened, got sick. Rob Collie (01:03:30): That's right. I believe those things happen. But I believe that they happen in a way that tends to not sway the outcome very much because Democrats or Republicans more likely to get arguments with their spouses that morning. It's just like ... it's hard to have a lot of those random events pull in the same direction. Michael Salfino (01:03:46): Definitely Republicans. Rob Collie (01:03:47): Okay. Yeah. Michael Salfino (01:03:48): I don't want to be partisan. Rob Collie (01:03:48): No. Don't. You know. Michael Salfino (01:03:48): No. I have no idea. Rob Collie (01:03:50): I mean, you know. I think that the range of initial conditions in which the randomness that can happen in a single day would sway the outcome from one to the other, I think that's incredibly tiny. I just think that that happens like never. We never find ourselves in a situation where the amount of randomness that can happen in a single day will change the outcome from blue to red or vice versa. I just don't think we find ourselves in those initial conditions. It's just not that kind of thing. Michael Salfino (01:04:18): I thought the problem with the viability of the predictions for the last election were extrapolating national margins and not really focusing on the state polls, because historically the likelihood of the national polls consensus winner correlated so well to electoral success. But, I think even though that had happened previously with Bush and Gore, I think it happening again has changed the nature of the polling for this cycle. Michael Salfino (01:04:52): So even though the national margin that everybody relied on was more or less in line with the forecast, I don't think you're going to have that problem again this time. Rob Collie (01:05:04): Right. Michael Salfino (01:05:04): But the thing that interests me the most about what you said is whether a sporting event can be analogous to a percentile or a percentage prediction of a political outcome. And I guess that just has to do not necessarily with being able to assess how the people are going to vote at an individual level, but just creating some sort of uncertainty regarding the polling metrics that were used. Would that be a fair way to say what ... and plus, we only have a single trial outcome, right? Rob Collie (01:05:40): Right. Michael Salfino (01:05:41): So if there was, like some people said, a 95% chance or whatever it was of Hillary Clinton to win the last election, maybe in the multiverse she won 95% of the times, but we just happened to be in one of those 5% cases. But you rejected that when we spoke the out it previously. Rob Collie (01:05:58): First of all, I respect any method by which we get the word multiverse onto the podcast. This notion that quantum randomness actually results in forking universes from every point in time, so you get the multiverse. I don't believe that. I believe that even though there is a lot of quantum randomness that happens every day, that leads to unpredictability, you just can't predict the future. That over the large volume of voters, it just gets all kind of netted out to zero over the course of the day. So, no, I think in 2016, I bet there were almost like 99.999% of all multiverses, trump won that election. The only one's- Michael Salfino (01:06:38): You're depressing me. Rob Collie (01:06:40): Again, we don't know which side you're on. Michael Salfino (01:06:42): No, no, no. Hypothetically. Rob Collie (01:06:45): Yeah. Hypothetically, that might depress you if you had that ... The ones where he lost are ones where he just randomly had a heart attack that day and died, right? That's an example of one where he would lose, he's just not around to win. And eating that diet also doesn't doesn't help his chances. Rob Collie (01:07:04): I just think that it's what you said earlier about we made some mistakes in our modeling last time that we're not going to repeat. That, to me, is the other big part of it? It's really two key points. One is that yeah, we do. We have assumptions that we make, and this election we're likely to discover other methodology mistakes. There's going to be a new set of methodology mistakes. It's kind of like if they only played a football game in the entire world, they only played it one game every four years, the amount of change that would happen in the players and strategies and coaches and everything, it'd be so hard to produce a good model there. Michael Salfino (01:07:38): But don't you think of model was good in the sense that it did pretty accurately forecast the national margin? And now there's the theories or the models that Nate has developed over at FiveThirtyEight, which extrapolate the margin of polling differential into an electoral win probability. And so anything below 4% gives Trump a chance. But then once you get up to 6%, 7%, 8%, it's a very small chance that that can be overcome. Rob Collie (01:08:10): Well, this starts to remind me of the CEO example, right? You have a product that succeeds and then you have a product that doesn't succeed. You just don't get enough trials really to evaluate whether a certain methodology is effective. Like this is the highest stakes, small sample size, low number of trials. I think the number of trials in this science is essentially almost always one. Michael Salfino (01:08:32): But what about ... can't you take ... Just from of polling, couldn't you take all the polling results and sort of grade the pollers, like FiveThirtyEight does? And come up with a model that was reliable. In other words, it's not just the presidential cycles, but polling is polling, right? As far as the election, all of those elections wouldn't those be reasonable tests of polling accuracy? Rob Collie (01:08:56): Maybe. But two things. Number one, I think I've dragged this out too long. Probably boring everybody to death. It's only interesting to me. And two, I don't know. I mean a presidential election is a presidential election. It's got different things going on. And we're going to find out, right? We're going to find out, especially in this election, we really don't know qualitatively what all the extra factors are going to be like. We're even wondering ... There is even articles right now wondering whether or not the loser of the election accepts that they lose. Thomas LaRock (01:09:27): We're not worried about the loser accepting the results. We're worried about one person accepting the results. Michael Salfino (01:09:33): But that person doesn't ... We don't need that person to accept the results. Where is it written that the loser has to accept losing? Thomas LaRock (01:09:39): It's true. Michael Salfino (01:09:39): Can the Jets actually just not accept losing? Would their record be better? Rob Collie (01:09:44): I've often wondered that. If a team just said- Michael Salfino (01:09:47): "We won this game. No." Rob Collie (01:09:48): "You ruled it a turnover, but we're not going to give you the ball." I've always wondered how long it would go before they'd actually have to pay up. Thomas LaRock (01:09:56): Fake interception. It just didn't happen. Mulligan. I get to do that one over. Rob Collie (01:10:01): Atlanta will come back and say, "Listen, you see what just happened? That's not actually possible. We all agree that that's not possible." Michael Salfino (01:10:07): Or Atlanta could just say you should take the probability of each game and turn that into a winning percentage, and that's our record. Thomas LaRock (01:10:13): Sure. Michael Salfino (01:10:14): Totally rational argument. Thomas LaRock (01:10:15): They'd still be 0-3, I think. Rob Collie (01:10:17): All right. Well, one thing I want to make sure we do is I want to give the people listening ... I want to give them a chance to track you down. You're Michael Salfino. Michael Salfino (01:10:24): MichaelSalfino on Twitter. I'm going to be doing a analytics site for gambling and also for player forecasting in terms of prop bets. That site is called Bet Prep. But it's just a tool for people to sort of mine data, as opposed to ... It doesn't give you the picks. It just avails the database and all of the historical games and statistics to you for your strategies. And I write for the Athletic, occasionally for the Wall Street Journal. And at FiveThirtyEight I do the chats every Monday on the NFL. Rob Collie (01:11:02): And everyone that plays fantasy football with me, you can just tune out right now. Just turn off your ears. It's cool. The podcast is over. But for those of you who stuck around, you also do the Breakfast Table podcast. Michael Salfino (01:11:16): Yes. Oh, I should have said that. Yeah. Rob Collie (01:11:18): Yeah. Michael Salfino (01:11:19): Okay. Members only. Rob Collie (01:11:21): Yeah, members only. I'm in the big spender bucket. I'm in the $7 a month. Michael Salfino (01:11:25): Yeah. $7 a month. You get everything. Rob Collie (01:11:28): And I even leave it active during baseball season. I just feel like that's how it ends up being a fair price for it. I don't turn it off during baseball season. I never tune in. Michael Salfino (01:11:38): We made more money when we just let people give us whatever they wanted, because some people would just like give us $100 or even more. Rob Collie (01:11:46): Right. Yeah. Michael Salfino (01:11:48): But I didn't know. I have no idea the way to do it. We're grateful that in a world where there are so many fantasy football podcasts, that people think that this is a worthy product that they want to pay for is flattering. Rob Collie (01:12:01): I know I'm trying to wrap this up, but it reminds me of the ...I think this was in Freakonomics where the daycare was tired of parents picking up their kids late. So they decided to impose a $20 fine. Thomas LaRock (01:12:13): Right. Michael Salfino (01:12:13): Yeah. Rob Collie (01:12:13): And instead, late pickups exploded. Michael Salfino (01:12:16): Yes, because it was like, "20 bucks. I don't even have feel guilty." Rob Collie (01:12:19): Yeah. It's like the $20 guilt tax. Michael Salfino (01:12:21): Yeah. Thomas LaRock (01:12:24): Wait, you'll watch my kid for another hour for 20 bucks? Michael Salfino (01:12:27): Right. It's way more of a motivation to not have to feel guilty than it is to just feel like you could pay 20 bucks and get the service. Rob Collie (01:12:34): Yeah. Thomas LaRock (01:12:34): That's awesome. Rob Collie (01:12:35): Yeah. Michael Salfino (01:12:35): As a former parent with kids in those situations, trust me, I would've been spending that $20 liberally. Rob Collie (01:12:43): I've really enjoyed this. I know Tom has as well. I told Tom ahead of time that he was going to love you. And he said, "Well, I'm married, but you know, maybe." Michael Salfino (01:12:52): Look, if Jets and Patriot fans can get together. Thomas LaRock (01:12:55): We can. Absolutely. I have no problem. Of course, I've been successful for two decades. Michael Salfino (01:12:59): Exactly. Rob Collie (01:13:00): So Tom still associates himself with the team. Earlier he said like we have to fly out to the ... he's on the plane, you know? Thomas LaRock (01:13:07): Absolutely. Rob Collie (01:13:07): He sits next to Belichick, going over film. Michael Salfino (01:13:10): Really the Patriot ... How could the Patriots have had this success without Tom being a fan? Rob Collie (01:13:14): They say the fans important. They're really talking about Tom. Thomas LaRock (01:13:17): Oh no, no, no. I'm sorry, but I've been a fan through all the lean years too. The Rod Rust era. Michael Salfino (01:13:24): Oh, stop with your lean. I'm so tired. The Boston sports victimhood is gone, forever. Rob Collie (01:13:31): Yeah. Thomas LaRock (01:13:31): I'm just saying, I survived the Rod Rust era. I was fan when they were 1-15 and the only win was against the Colts. I remember that season. So I'm not a bandwagon. That's why I guess I'm trying say. Michael Salfino (01:13:44): Yeah. And I totally respect. Let's put it this way. I have no respect for bandwagon fans. You have to stick with it. People say, "Why are you still a jet ..." Like my wife all the time. "Why are you a Jet's fan? You don't have to be." It's like, "You don't understand." I don't even mind. I'll wear my Jets hat when the Jets are like, now. I'm not going to do it when they're playing well. You know what I mean? Because I don't want to seem like I'm on the bandwagon with the Jets. Now, this is when the true fans come out, now. Thomas LaRock (01:14:15): Agreed. Rob Collie (01:14:16): Yeah. This is when the true irrationality comes out, where we associate ourselves with something that is totally beyond our control. Michael Salfino (01:14:24): Name something else that people could have loved since they were like seven years old, the same way they love right now, the same way they love those things right now. There's really nothing else other than a sports team. Thomas LaRock (01:14:34): Peanut butter. Michael Salfino (01:14:34): You know what? I agree with that. Rob Collie (01:14:37): Yeah. Michael Salfino (01:14:37): I agree with peanut butter. Rob Collie (01:14:40): I stipulate peanut butter. Michael Salfino (01:14:43): Non food. Rob Collie (01:14:46): I love sports, obviously. At the same time though, I just don't think that we need any mechanisms growing up that teaches tribalism. I really- Michael Salfino (01:14:57): But we're tribal. So why not have something really safe, like being tribal for a sports team than something that's actually dangerous? Rob Collie (01:15:04): I Agree. Michael Salfino (01:15:04): Excluding Eagles fans, of course. Because that's dangerous. They're dangerous anyway. Rob Collie (01:15:08): No one ever gives us the lesson as well, which is ... and don't extend this to other parts of your life. This is the one place in your life that you get to be like this. But when you put this down, don't be this way. I would love that. Michael Salfino (01:15:22): But it's not like I'm a face painter, you know. There's tribal and there's tribal. I think you could be a fan without being completely ... I'm like Groucho Marx, man. I'm not going to be in any club that would have me as a member. There's no way I'm going to go over the moon like some people do. I have to keep a somewhat ... I have to keep a safe distance for appearances. But inwardly, trust me, I have my face painted. Thomas LaRock (01:15:50): I'll give you a quick story. It was years ago. I'm watching the game. You remember Jerry Glanville? Michael Salfino (01:15:55): Yeah. Thomas LaRock (01:15:55): He had stopped coaching and he was announcing. And this defender did everything right. Everything right. Just perfect defense. And the guy still made the catch. And Jerry said, "Well, when he gets the sideline, what you say is the offense gets paid too." Michael Salfino (01:16:11): Yeah. Thomas LaRock (01:16:12): And the thing is, Jerry, he wouldn't be screaming at the guy. He would just say, "You know what, we got beat by a good play." You shake somebody's hand. And we're never taught that compassion, that empathy for somebody else and just congratulate, "Hey, you know what? You beat me." Instead. it's yelling and screaming. "You suck." Michael Salfino (01:16:28): "You cheated. This is rigged." Thomas LaRock (01:16:30): Like, "I can't believe I lost at this." And that is what's really missing. The tribalism isn't really the issue. It's having compassion for the other person. Rob Collie (01:16:38): Yes. Michael Salfino (01:16:39): Yeah. So see, you Patriot fans should be more compassionate to us Jet fans. Thomas LaRock (01:16:44): I do my best. Michael Salfino (01:16:46): Sure you do. Thomas LaRock (01:16:48): I have no issue with the ... I really don't. There are very few teams in sports I hate. I don't hate the Jets. Why would I hate the Jets? You're not good. There's no reason to hate you. Michael Salfino (01:16:57): And that's why we hate the Patriots. You just summed it up. Rob Collie (01:17:01): It's that condescending ... you know, that pity. Thomas LaRock (01:17:02): But no, it's true. I don't. There's very few teams I hate, and the Jets are not on that list. Michael Salfino (01:17:11): Well, hopefully one day we will be worthy of your hatred. Rob Collie (01:17:16): We aspire. Yeah. Michael Salfino (01:17:18): Definitely. Rob Collie (01:17:19): Mike, we've enjoyed the hell out this. Michael Salfino (01:17:21): It's been a lot of fun. I hope it's good for you guys. I was just winging it, so sorry if I- Thomas LaRock (01:17:27): No, you're good. Michael Salfino (01:17:28): Sorry if I talk too much. Rob Collie (01:17:29): Are you kidding? Michael Salfino (01:17:31): Okay. Rob Collie (01:17:31): Do you know where you came to? Michael Salfino (01:17:33): All right. Rob Collie (01:17:34): So Mike, we've got an election coming up. We'll definitely want to have you back on after that. Michael Salfino (01:17:39): Oh my God. I may be ... you may have to contact me in Canada or Italy. I found out I could move to Italy. They treat me as a citizen. I could just go to Italy. Rob Collie (01:17:48): Really? Michael Salfino (01:17:48): Yeah. If you have any Italian blood, you can move to Italy, no strings attached. Go there and they take you. My wife, get the hell out. She's got to figure it out. Thomas LaRock (01:18:02): I have this. I can go? Michael Salfino (01:18:03): All you have to do is prove any Italian blood and you're Italian in the eyes of Italy. It's like the mob. Thomas LaRock (01:18:09): I didn't know this. Rob Collie (01:18:10): I wonder if Luke has any Italian blood with the last name of Pierazoli. Thomas LaRock (01:18:18): My great-grandparents came off the boat. I've got the blood, I've got the documents to prove it. Michael Salfino (01:18:22): I'll see you there, November 5th. Rob Collie (01:18:26): As long as we're all not out in the streets fighting the second American civil war. Michael Salfino (01:18:32): Oh my God. Rob Collie (01:18:33): We'll have you back on. How does that sound? Michael Salfino (01:18:35): Yeah, that'd be great. I appreciate it, guys. Rob Collie (01:18:37): Awesome. Michael Salfino (01:18:37): Okay. Rob Collie (01:18:37): Awesome. Looking forward to it. Michael Salfino (01:18:39): Thanks a lot. Announcer (01:18:40): Thanks for listening to the Raw Data by P3 podcast. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Interested in becoming a guest on the show? Email. Lukep, L-U-K-E-P@powerpivotpro.com. Have a day today!

In our inaugural episode, Rob and Tom cover all this and more: The slow decline of the "storage only" professional Tom has some unkind words for a rock and roll superstar Why SQL Batman had to retire The rise of hybrid IT/Business professionals Why are database administrators (DBA's) such a miserable crew? Rob shares a great story about a former colleague who pissed off Steve Ballmer Why storage has gone "curly" but analysis remains "rectangular" How discovering Power BI feels like you're the first person to discover fire Episode Transcript: Rob Collie (00:00): Okay, welcome. So this is our inaugural episode and our first guest is also going to be our ongoing 75% co-host; 75% only because his schedule won't always allow him to join us, but he's Tom, or Thomas LaRock. Now, I've known Tom for over a decade and he's just a fantastic human. Rob Collie (00:22): I hope you're going to find that to be a theme here on Raw Data with our guests and various participants. But data-wise, the thing I've always found so compelling about Tom, his crossover status. Now, here's a guy who branded himself publicly as SQL rockstar for years, and he kind of still does. And you'd think that pretty much cements him as a storage professional. Rob Collie (00:44): But basically the whole time I've known him, he's been trumpeting the idea that analytics are the real show. Now, in my experience, that sort of crossover is atypical and it's super valuable and it speaks to why I'm thrilled to have him as my fractionally available co-host and as our first guest. So how do we get this started? Luke, do you think we could do one of those like fancy produced intros with music and stuff? Luke (01:11): Yes. The budget did allow for a fancy produced intro. Rob Collie (01:14): Oh yeah? Luke (01:15): Yep. Rob Collie (01:16): Well, let's do it then. Announcer (01:18): Ladies, may I have your attention please? Announcer (01:22): This is the Raw Data by P3 podcast, with your host, Rob Collie, and your co-host, Thomas LaRock. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Raw Data by P3 is data with the human element. Rob Collie (01:40): Welcome to Raw Data. I'm your host, Rob Collie, CEO and founder of P3, powerpivotpro.com. And Tom. Thomas LaRock (01:50): Hi, Thomas LaRock here. I am a head geek at SolarWinds. Rob Collie (01:56): Thomas. He's Thomas LaRock. Thomas LaRock (01:58): Thomas. Rob Collie (01:59): Yeah. And who else do we have here with us? Luke (02:01): My name is Luke. I'm talk radio guy in South Florida. And I'm the guy that knows nothing about data. Rob Collie (02:09): So welcome to the first ever edition of Raw Data. I'm really excited by the crew we've got here. So let's jump right in. Tom, we've... Or should I call you Thomas? Thomas LaRock (02:23): You call me Tom. I usually introduce myself as Thomas just that people don't think it's like John or Dom or something like that. Rob Collie (02:30): I see. Thomas is easier to parse. I get it. But I just wonder if you were one of those guys that changed his name at some point, like you grew up. Because I've always known you as Tom. But Thomas. So we've known each other now for... We had our 10 year anniversary recently, didn't we? Thomas LaRock (02:49): Yes, we did. It was 10 years ago, this past June. Rob Collie (02:53): Yeah. It's crazy. It was like it was just yesterday. So we have interesting complimentary backgrounds, you and I, and that's why I think I was really excited to do this with you. What's your handle on Twitter? Thomas LaRock (03:07): SQL Rockstar. Rob Collie (03:08): SQL rockstar. Now, Luke here, in his day job, he actually interviews real rock stars. He just had Sammy Hagar. Thomas LaRock (03:21): Oh, I thought you said rock stars. Rob Collie (03:22): Oh, cold. All right. Well, he had Jason Newsted. How's that? Thomas LaRock (03:30): I don't even know who that is. Rob Collie (03:34): I'm starting to regret my selection of cohost. He was the basis for Metallica for a while. Thomas LaRock (03:42): Oh, yeah. Okay. Rob Collie (03:44): Not anymore. Luke is really kind of slumming it with us. He's gone from real rock stars to SQL rock stars. Thomas LaRock (03:53): Wow. Rob Collie (03:54): I know, I know. I mean, it's not like I'm any better. I'm not Sam Hagar. Anyway, in a previous conversation, I said, hey, well, I come from sort of like the analytics and business intelligence side of the world. And then I said that your background originated more on the storage side and you kind of recoiled just a little bit. It didn't seem like it was right to you. And then you thought about it. So what does storage mean to you? Is it the right word? Am I using the right word? Thomas LaRock (04:32): Well, I think you are using the right word. I just never thought of it in that manner. To me, when I hear storage, I think of the guys in charge of the SANs or racking servers and things of that nature. The storage admin, there's doing storage. I was a database administrator. I never really thought about it. Thomas LaRock (04:53): But in a way it is storage because it's the storing of the data. So the data has to go into an engine and then to disk and then from disk back through the engine and back to the client. So yeah, you could think of it as storage. I usually think of it or I usually tell people the focus was on the internals of the database engine itself. In this case it would be Microsoft SQL server. Rob Collie (05:18): Yeah. So a lot of the history of the analytics industry and the business intelligence industry is we're still, I think, in the middle of a multi decade hangover of the influence of the storage industry on the way that a lot of analytics were, even from a software industry perspective. By necessity, there have been so many storage professionals. Storage and retrieval. Rob Collie (05:51): If you can't store data and recall it, you can't run a business. You can't even execute a transaction. Whether you're doing any reporting or analytics or not, storage is table stakes. And so when we met, I had recently joined Twitter within the last six months before we met, we met in what? It was like May of 2010, somewhere in there anyway. We just celebrated our 10 anniversary, I should know this. But I forget. I'm that guy. Thomas LaRock (06:23): You forgot our anniversary? Luke (06:26): Shame. Rob Collie (06:27): Yeah, I know. I know. And so I had joined like the data channels on Twitter. I had done the right thing. I had gone and I joined the data channels and I quickly discovered that almost everyone on those channels were storage professionals. They were primarily database administrators, DBAs. So when I walked up to you and I got introduced at that live tweet event that was, I don't know, it was kind of a funny thing that people used to do. Thomas LaRock (06:53): Remember we used to meet people and go places? Rob Collie (06:57): Yeah. That was radical stuff. But do you remember the first question I had for you? Thomas LaRock (07:02): I do. I do. It was essentially the clean version is, why are DBAs so miserable? Rob Collie (07:13): It's quite an opener. Thomas LaRock (07:15): It is quite an opener. I was stunned because I remember looking at and going, first of all, who the hell is this guy? And secondly, how does he know us so well? Rob Collie (07:27): The power of Twitter, man. Thomas LaRock (07:30): Yeah, you've been stalking us. And clearly I had no defense. I wasn't about to sit there and tell you, "Oh no, we're the happiest bunch of people. What are you talking about?" I sort of looked at you and I'm like, "I don't know why we're so miserable. I have some ideas as to why we're so miserable." And I think we talked through some of those ideas. Thomas LaRock (07:52): And if I recall, towards the end of that initial conversation, your comment to me was basically we were just like the Excel community. We had a lot of the same traits. Part of our misery was rooted in working, not just with data, but with users of data. Thomas LaRock (08:17): And it was interesting to find the parallel between what I thought was a unique group of individuals, this database administration community, and all of a sudden the Excel community. I'm like, what do we have in common? We actually have so much more in common than I had ever realized. So you're question really opened up a brand new perspective for me at that moment in time and going forward. Rob Collie (08:47): Yeah. The common thread there, I think, is a community. Although really the Excel people don't really have a community. They're a demographic, if you will, but they don't really have a community in the same way that the DBAs do. They make the world go round. The world runs because of... And I know there's lots of people that make the world run, but DBAs and Excel people, people who are good at Excel, these are incredibly essential roles for the world that no one really sees or appreciates what really goes into it. Rob Collie (09:25): And so when they interface with the rest of the business world, they tend to be taken for granted, even though what they do is some pretty arcane skills that are developed there. But yeah, the Excel people don't have a water cooler in the same way. I have been more recently monitoring things like the accounting sub Reddit. Okay. Now here we go. Here's where the grumpiness is. Yeah. There's some Excel grumpiness. It's the same kind of blowing off steam outlet as what I was seeing on Twitter back 10 years ago. Thomas LaRock (10:03): Yeah. I've often referred to just the internet in general as a cesspool of misery. But then you get into that dark corner called Reddit and you're going to just find I think a lot of people more... Maybe it's the anonymity. You don't have to really use your real name and you can just sort of vent. Thomas LaRock (10:24): And I think for some people, Reddit is just a place where they can vent. But for an outsider like me, I'm not really active in Reddit. I can go there and I'm like, "Wow, these people are really miserable." No. Actually, they just need to vent. Rob Collie (10:37): Maybe. Although honestly I find Reddit, and again, I curate what I consume from Reddit rather than just like taking the default feed, but I find it to be the most intelligent and civil corner of the internet. But again, it's probably because I've tuned it. Thomas LaRock (10:55): Oh yeah. Absolutely. There are corners of Reddit where people are civil. Absolutely. And then there are some horrible, horrible places. But for all that, nothing's as bad as YouTube comments. Rob Collie (11:11): That's what Joe Rogan says. Thomas LaRock (11:12): That is the worst, worst thing. Rob Collie (11:16): Yeah. I don't think I've ever really gone down that rabbit hole, so I'm going to probably stay away. Circling back, where you and I sort of found I think almost immediate common ground was in the notion that I had come originally from the Excel community. That's what I worked on at Microsoft for a very long time, was worked on Excel before I got involved in the business intelligence side of software. Rob Collie (11:44): And now for the last 10 years been running a company in that space, a consulting company. Where these two worlds meet is where an extra kind of value is created from data. So there's the primary usage of data, which is running the transaction. Someone buys something, for example, you've got to record the transaction. You've got to process it. You've got to make sure that they paid, all that kind of stuff. Rob Collie (12:11): Obviously that's primary usage of data in business is to make the actual transactions operate. But then there's this secondary value of data and the secondary value is mining it, if you will, for insights about your business to improve and optimize. And that's where I've been. For a while out there, I was a SQL server MVP. Microsoft had knighted me as a SQL server MVP and I don't know SQL at all because the BI stuff was looped in with it. Rob Collie (12:44): But one of the things I found super, super, super compelling about you over the years, Tom, is that you don't view where you came from as the only thing. You're evolving. And you've been very, very open and enthusiastic about the world of analytics, BI, whatever you want to call it. Whereas not everyone in your original community, your community of origin, where I met you, not everyone in your community that you came from is like that. And you are exceptional in this regard. You're not the only one. Thomas LaRock (13:18): Right. I don't think I'm exceptional, but you are absolutely right. There are lots of people that would have you understand that let's say you were at an event, a large three-day conference, and it was SQL server focus event. Then everything should be core engine, deep dive, 500 level sessions. Thomas LaRock (13:39): And wait, what's this thing? Business intelligence? We don't need any of those sessions here. Go somewhere else. And those people absolutely exist. They still exist today. There's fewer I would say today, but they're out there. They used to be a lot more. Rob Collie (13:54): Where'd they go? If they're not there anymore, where did they go? Thomas LaRock (13:59): I think they've started just disengaging with the community as a whole. Because the middle ground, the middle class, they have kind of embraced a little bit more of the analytics space because it's everywhere now. It's prevalent all over. You can't get away from it. I think some of those more extreme people just don't find comfort in being around or say they just don't feel that's the group for them anymore. Thomas LaRock (14:25): They want to be with people that are really just core engine. That's it. So we certainly had that, like when you and I first met, those people were out there. I wouldn't say I was one of them, but I would say I was kind of stuck in my own little silo. I had my own blinders on for my own reasons. It was just stuff that I was working with and I wasn't really exposed to as much. But over time I was exposed to it. Thomas LaRock (14:51): I have a background in mathematics. And so for a lot of it, it was kind of interesting and familiar to me. And of course, a few years after that, that's when data science started becoming an actual term, which was interesting again. I see a lot of these people saying, "Oh, well..." I'd also worked on my Six Sigma certifications at the time. I think I got green belt at the time. And that was a lot of stats. Thomas LaRock (15:22): And these people were just mesmerized by being able to understand what a standard deviation was and how to apply it and how to use it. It was the application of these tools in order to get insights from your data. And I liked it. So I continue to kind of try to absorb a little bit of that. Meanwhile, I still have one leg in, hey, what's happening inside the engine? Somebody's going to come to me and say, "My query is slow. I need to make it faster." And I want to be able to help them too. Thomas LaRock (15:54): But I also want people to come to me and say, "Hey, sales are down. What can we do?" "Oh, well, what data do we have?" "I don't know." "Let me help you sort this out and figure out if I can find any value for you." So yeah, it was an interesting time, I think, around 2010. And I think that's when BI really started getting more mainstream. There was a lot of work by Microsoft for reporting services. Thomas LaRock (16:21): Of course, when Power BI came out, I want to say it was about five years ago now. So there's a lot of work by Microsoft to help turn the corner. I mean, they even changed the name of, I'm no longer a SQL server MVP. I'm now a data platform MVP. So even the wording and everything about it has kind of changed and been a little bit more welcoming is what I would say. So these days I think most people are very comfortable with the idea that they might have to have one foot in the analytics space as well inside the database engine. Rob Collie (16:55): Yeah. It seems like such obvious, low hanging fruit, if you're already up to your eyeballs in the data platform in various ways. Why not even just from a career standpoint go and pick up that secondary value? You're like nine tenths of the away there maybe. The challenge though, of course, is always the human element. The more someone identifies as an IT professional, typically the less business interested they happen to be. And this is a very, very broad brush. Rob Collie (17:28): So people are listening to this right now going, "Wait, I'm in IT and I'm obsessed with business." Well, yes. And that's great. And that's actually a rising trend, this idea of the hybrid. I'm running into all kinds of people these days with job titles that so clearly scream one foot in IT and one foot in the business. And that's the way of the future, I think. Rob Collie (17:51): But there's still a pretty strong center of mass there for if you think of yourself as IT, you're focused on the tech and not necessarily even as interested in the business problem. And that's the human component of it. I think that if you view BI and analytics as just another part of the stack, just another part of the technology toolkit, well, that's where we've come from. The entire BI industry has always been like that. Rob Collie (18:22): And spoiler alert, it's never worked. That mindset has never once worked. It's this hybrid mindset and the tools that enable it that are really changing things right now. Boy, I've really buried the lead here. Here's the question I have for you, and it's a two parter. Thomas LaRock (18:41): Oh, you didn't say there'd be a quiz. Rob Collie (18:43): Yeah. You can't be wrong about this. I'm going to ask you both parts at the same time so you have an opportunity to contemplate both answers simultaneously. So you're SQL Rockstar on Twitter. That's your brand. That's in many ways synonymous with you. And you know how rebranding is. Rebranding is very difficult. So when did you first get into storage? When did you first get into SQL server? Thomas LaRock (19:08): Oh, early two thousands. Let's just put a stick and say 2003 ish. I was programmer/developer before that and using Sybase and Oracle and SQL server. But around 2003 was when I started doing more the database administration role. Rob Collie (19:27): Okay. So let's set 2003 as just semi arbitrary milestone and say you could go back to 2003 and tell your 2003 self, "Hey, self, when you get around to branding yourself, here's what you should call yourself." Would it still be SQL Rockstar? That's question number one. Thomas LaRock (19:51): Oh, I'm sorry. Do you want me to wait? Okay. I'll wait. Rob Collie (19:54): Well, I was hoping for a little bit more than a one word answer as well, but I'm sure. You're as long-winded as I am, so we're a good pairing. The second question is if you could instantaneously rebrand your Twitter, for example, today, you could pick another handle today and have it be retroactively what you always had been, would it be the same as your answer for 2003? If you could just pivot today with no switching cost. Thomas LaRock (20:21): Here's something that I guess you didn't know about me. I've already changed my Twitter handle once. Rob Collie (20:26): Oh, I did remember that. Yeah. You were just Thomas the Rock for a while, weren't you? Thomas LaRock (20:30): No. Rob Collie (20:30): What? Thomas LaRock (20:31): I was SQL Batman. Rob Collie (20:33): No, you weren't. Thomas LaRock (20:34): I was SQL Batman. And I was SQL Batman because in my role, that's basically what you are. You're Batman. Something goes wrong, they call you and you come in and you're superhero and you have to fix it. You're just like, I'm Batman. And I had that for maybe almost a full year of being on Twitter at first. I had stickers that said SQL Batman with the bat logo. I had all this stuff going on. That's what I was originally doing. Thomas LaRock (21:05): And I ran into some issues with licensing, if you can imagine. If I wanted to get stickers printed up and use a service, they'd be like, "We're not doing this. You can't use that." And I'm like, "Come on. I'm harmless." But no, there's a whole thing about protecting trademark and copyrights. So I got tired of trying to utilize the SQL Batman. I even had sqlbatman.com. Thomas LaRock (21:38): I just made change and I said, you know what? I'm going to change my blog to be thomaslarock.com. I'll just use my name for that. But for Twitter, I'll use the handle SQL Rockstar. And the reason I chose rockstar, my last name is LaRock. I've had the nickname the Moniker Rockstar since I was about 16. My friends in high school, like LaRock, rockstar. It's just the way it was. Thomas LaRock (22:03): So I decided I would just call myself rockstar. But at the time the rockstar movie was out and that was yet another problem. So I just said put SQL in front of it and then it's what I own and I'll move forward with things. And if I could go back, I'd probably tell myself, "No, do data rockstar instead." But honestly, these days I look and I go, "Just use your name." Thomas LaRock (22:25): There's no reason to have the thing at the time. It was what pretty much most of the cool kids were doing on Twitter at the time. If you were inside of the database community, you were using SQL in front of everything and being cute. And I did it and I've never bothered to change it. So yeah, if I could go back in time, I would tell myself have more of a different focus. If I wanted to use a cute Moniker, it'd probably be Data Rockstar, Data Pro, or something like that. Thomas LaRock (22:58): If you notice, actually for a while, I didn't use my real name. It was at SQL Rockstar and the name was also SQL Rockstar. I changed a few years back to put my real name in there. So you can see Thomas LaRock and at SQL rockstar. That was kind of my compromise for myself instead of just changing my handle, which I don't think I could do, because once I got verified, you can't touch things. Otherwise they take away your check mark. You don't want to lose the check mark. The check mark makes me legit. Rob Collie (23:25): Oh yeah. That's right. That's why I invited you. It's the check mark. We needed more check mark. Infinite percentage increase in check marks on the show. So yeah, data rockstar. That would've been good. I could get behind that. Here's a hypothesis of mine. That's more than a hypothesis. This is an opinion of mine. It's been really interesting. I've watched this evolve over probably almost a full 20 years I've been watching this story and it's basically that storage changes all the time, but analysis is... Rob Collie (24:00): Actually, there's all kinds of technological improvements that allow us to do things fast or do things better, lower friction, et cetera. There's been a lot of big changes on that front. That's really like the reason why my company even exists is because of how much change is happening and already has happened in that space. But at Microsoft's in the early two thousands, about the same time that you were getting into SQL server for the first time, Microsoft was really struggling with this, starting to wake up to the idea that data might not be just stored in tables. Rob Collie (24:39): That data might not always be table shaped. And this was really causing almost like an existential crisis in the data world at Microsoft. And it's really funny. I got to read white papers written by like the architects that even at the time were being paid like $3 million a year. These really high [inaudible 00:25:00] flu white papers that sounded really smart. And from what I remember them now, they have no whole bearing on where the world actually went. Rob Collie (25:11): They were just completely off. Basically the answer was, oh, well, we'll just make it so that we can also store XML blobs in SQL server and that'll take care of it. I mean, there are all kinds of funny things. There are so many funny things to reflect on. But while the storage half of Microsoft was freaking out about this and while often in the shadows things like Hadoop were being born that was the real answer to this crisis. Not XML of blobs in SQL. Rob Collie (25:49): The analysis world was also panicking about it at Microsoft. Something really fundamental about the way that we work was potentially at risk. And there was another architect at Microsoft who had decided to kind of crash the Excel team for about a year. He just needed a place to land. This guy had been around forever. He came to the Excel team to tell us that the Excel grid, rows and columns, that was old fashioned, that was outdated, and that was going to go away. We needed all of that sort of jagged, heterogeneous content, where like, how do you store a webpage? Rob Collie (26:32): That the content of a webpage doesn't fit into a row oriented storage that well, does it? And each web page is different. Every different piece of information might have different columns if you were going to try to store it in a column oriented way. And he was convinced that that same phenomenon was coming for Excel. That every row of data in Excel might have a different column set than the previous row and Excels whole formula language, everything needed to be redone to accommodate this. Rob Collie (27:11): Now, of course, we now have the benefit of hindsight, 15, 17 years later, this hasn't happened and no one's dying for it either. But remember, this is like a main man at Microsoft. This is someone that gates himself. He was almost like a lieutenant of him. And at that point in my career, I'd already finally learned enough to know when someone was just like totally off the rails. And he was so much more senior than I. The only card I had to play against him was just to repeatedly say like Tom Hanks in big, to say, "I don't get it," just over and over and over again. Rob Collie (27:55): Rows and columns. We've had rows and columns in Excel forever. Everyone's played battleship. They know how to line up a row and a column and give it a coordinate. Your thing. I don't get it. Thomas LaRock (28:07): It's a great strategy. Rob Collie (28:11): I also knew that it would work because my superiors had made it very clear to me that we weren't going to do stupid, like crazy computer science things with the Excel product. We had more responsible things to do. But none of them really wanted to go toe to toe with this guy. So they kind of put me out there. But I knew who was writing my review. Rob Collie (28:34): And this guy would go around and tell everybody behind my back that, "Whenever I bring this up with Rob, that guy, that guy, he's just done. He's just done." He was so obnoxious. He's basically telling everyone that I was too stupid to understand what he was going for. But again, even him now, all these years later, he would probably admit that analysis is rows. Rob Collie (29:00): The only things that you analyze are the things that are in common between like if you've got like 15 rows of data or 5 million rows of data, and some of them don't have certain columns, well, you wouldn't be including those in your analysis unless you had common attributes in each row of data that was interesting to analyze. Otherwise that row wouldn't even be involved. Rob Collie (29:29): So the way I've been boiling this down for people lately is that... And this architect, he described this odd heterogeneous storage, he described it as curly data. And I liked that. I like that idea, curly data. And like you need to go store the internet for a search engine, damn straight that's curly data. That is not nice, clean tables. Rob Collie (29:55): But when it's analysis time, analysis you're always pulling rectangles. You're always extracting rectangular table shaped row sets in order to perform the analysis. That's separate from the storage. So the query engines that have been built over time that allow you to retrieve data from things like Hadoop. Well, how many sequel-like interfaces have now been built to pull regular shapes out of those sources? Rob Collie (30:32): So my world of analysis has, at least until now, been very well insulated from the storage revolution in terms of, what do you want to call it? Curly data. The curly data storage revolution. And the same way that analysis wasn't disrupted terribly much by the transition from tape reels to hard drives, the fundamentals of what you were doing, the technology was different, but the fundamentals of what you were doing were not rewritten just because we started storing things differently. That was a monologue. That's one of my things. What's your reaction? I've never told you that story before. I don't think so. Thomas LaRock (31:14): No, I don't. And my first reaction is, is that person still at Microsoft? Rob Collie (31:19): No. Thomas LaRock (31:19): And I need to know right now. I was going to say later you're going to tell me who that is. Rob Collie (31:26): Before we move on then, I won't tell you who it is, but A, he left Microsoft in a huff shortly after that, after he did not get his way on that. Right. I kind of get to almost like paint a silhouette of him on my airplane. And he famously when he told Ballmer he was leaving, Ballmer threw a chair across the room. So now you know everything you need to know to look up who this was. Thomas LaRock (31:53): Threw a chair across the room, because Ballmer wanted him to stay. Huh? Rob Collie (31:58): Yeah. And he went to Google and all of us on the Excel team were just sitting chuckling like, "Eat it up. Yeah, you should take him." Thomas LaRock (32:11): Well, that explains a lot about Google Sheets. Okay. So here's the thing, when you were just describing to me about the curly data and you're talking about the analytics and you got to the point that was in the back of my mind as you were speaking, which is that you say row, I'll say it's... What's the fancy word? Observation. That row, the observation of a data event, you may not have information for all those columns or attributes. Thomas LaRock (32:43): And that's totally normal to me right now. I'm like, yeah, I get it. One of the things I say is nobody goes to school to become a data janitor. Hmm. I didn't. There was no course. And I think your response to that was here we are. This is what we do. We are the data janitors of the world, whether you're Excel or you're a DBA, this was the common ground we had. Thomas LaRock (33:12): I didn't know it 10 years ago. It took me a while to get up to it. But why are we miserable? Because we're data janitors all day. This is what we do. And why don't we have the observations for all this? Are you kidding me? I don't know. A sensor went down. Oh, okay. Or we just didn't think to ask that question. And so it's not included in 10,000 survey results. We didn't think that question was worthwhile. It's like, but there was data, and I had this whole model built and it needed that. Thomas LaRock (33:42): Now what am I supposed to do with these 10,000, 20,000 records? It can be very frustrating. I can see what the man was trying to describe, but he really wasn't able to articulate what he thought was coming. And more importantly, he didn't really understand the tool. He thought the tool had to change, but the reality is the tool itself didn't have to change. Thomas LaRock (34:12): It was the application of the tool was going to become different. And he couldn't see that even at the time. I mean, Python was a thing. You could have done so much more. There's a bigger world out there than just the Microsoft data platform, as great as it is. And I love it. Rob Collie (34:30): Come on, come on. Thomas LaRock (34:30): It's true. But there's still stuff out there, stuff out there. But yeah, that was kind of my thought was this guy was not a data janitor and he wanted the tool to do this specific thing. So you guys were going to have to go and reinvent it, which would've been a huge waste of time. Whoever was really in charge there for you guys, thank God they knew not to try to shift gears. Rob Collie (34:55): Yeah. I completely agree. The thing that he was missing, and it's not like I knew it then either, if I'd known it, I would've told him this storage revolution that they saw coming is decoupled from analysis. Again, up until this point, you never know what's around the corner. But up until this point, analysis has insulated. It's kind of like I just need to know... Rob Collie (35:25): Another way to say it is that I don't actually know truly deep down how a SQL database is structured. I don't need to. I think of the table that I pull from it, which is oftentimes a view written by a friendly person, such as yourself. That view is reconstituting my rectangle of data that I need from all kinds of other tables that I don't necessarily see. Rob Collie (35:53): I don't care. It's beautiful. I don't need to. And so if the view, the rectangle that I'm getting happens to be stored out there on many different hard drives and in a hive farm or something and in curly format, but I get a rectangle back, my job doesn't really change. Take the analysis hat off for a moment. What are your observations of this? Rob Collie (36:22): When I call it like a revolution in storage, is it really? How much has the curly storage model, Data Lakes, Hadoop, all that kind of stuff. How much is that... I don't know. I was going to use the word invaded to sound dramatic. How much has that stuff kind of invaded your world? Thomas LaRock (36:43): Well, in terms of say the Microsoft data platform, it was years ago when they introduced the concept of PolyBase. So PolyBase is just a simpler way for you to link to almost any other data structure and to pull the data into SQL server. And they're trying to make it very easy to connect basically from their data platform and extend into any other platform in order to get the data into one place and then build your rectangle for you. Thomas LaRock (37:17): So it's there and it can comes up every now and then and somebody says I've built this. It's not working as well or things of that nature. So it's definitely part of the ecosystem these days. And the latest one, what is it called? Big data clusters that Microsoft just rolled out. They're making efforts to build into their ecosystem something that is equivalent in other ecosystems. Thomas LaRock (37:46): So if you are a Microsoft customer and you need certain functionality inside that data platform, it actually exists somewhere. It's a framework, it's a tinker set. All the pieces are there. You might have to build something more or less than other things. But a lot of that functionality is really there, especially in Azure. There's just so much these days. Rob Collie (38:09): Azure. This is not SQL server. Now, there is SQL Azure. Thomas LaRock (38:19): No. There's Azure SQL database. Microsoft marketing would not want to hear you say SQL Azure. Rob Collie (38:24): Well, if they're listening, I'll call it a win. If we reach the point where we can upset people with the way we describe things by using the... I'll start calling Power Pivot, I'll start calling it Power BI in Excel. It's just really the only rational name for it. Thomas LaRock (38:44): It used to be called SQL Azure and I love that name. Rob Collie (38:49): Look how dated I am. It makes sense why they call it data platform, because there's just so many things in there now. And so many of them, as you were hinting at, are clones in a way, improved clones in many cases of things that we see on the Linux platform. If you go look at AWS, so much of AWS, the services available there, it's what I call the Linux cool kids stack. Rob Collie (39:21): If you're launching a startup in Silicon Valley, you're issued your MacBook and here's your AWS subscription. These are like the starter kit. Microsoft licensed, what is it HDInsight? That's basically like a Linux distribution. And so there's a lot of literal Linux services available on Azure. Rob Collie (39:46): And at the same time, you also see these more windows based services in the Azure platform and you start almost like lining them up. You start saying, "Oh, this one's kind of like that one from over in the Linux stack," but it's Microsoft taking a look at it going, "Oh, we can do better." Rob Collie (40:02): And so it's a really interesting ecosystem going on over there. Let me put you on the spot here. Have you done any technical hands-on work with, I wonder what we call it, modern storage, the curly storage, or have you've been in sort of the chief geek role long enough that you haven't gotten your hands dirty with that? Thomas LaRock (40:23): So it's head geek. Rob Collie (40:24): Head geek. I'm so sorry. Thomas LaRock (40:27): I do joke that I haven't had a real job in a long time. I'm very far removed from my production DBA days. However, in my role as head geek, I get my hands on the things, but not for production purposes. It's more for I've got to learn to understand what these things are doing, how certain things work, because I need to be able to explain some stuff to others. Thomas LaRock (40:49): But what I have done, and it's been a few years, Microsoft partnered with edX and they put together some certification programs. So you would take like 10 classes online through edX and they would align with a certification. I got a certification in, let me think now, well, one was in big data, one was in machine learning, I think, and another one in artificial intelligence. So have I put my hands on the curly data? I'd say yes. Thomas LaRock (41:24): But those being Microsoft focused programs, it was touching a lot of areas of Azure. So did I have to go into Azure data factory, consume some data, transform it, write some use SQL to pull some insights out of it? Yeah, I had to do all those things. It's been a while. If I had to do it again, I could probably go back and figure it all out again. Thomas LaRock (41:47): But once I did it for the program, there was really no need for me to touch it again. Lately what I have been doing is I've been spending a lot of time learning Python. Sometimes people say, "What should I learn, Python or R?" And I kind of view it as two different things. I think R is very much focused on being a tool for data scientists. And I think if you're a data scientists, you want to use it. That's great. Thomas LaRock (42:12): I think Python is a little more extendable. It can do all the same data science things that R can do, but it can also do some other things. So that's why I chose to dive into Python and I've been spending a lot of time on it. And then there's this little website called Kaggle. Have you ever heard of Kaggle? Rob Collie (42:27): I have. Thomas LaRock (42:28): Yes. So I've started doing some learning and competitions in Kaggle. And again, focused on using Python, but I can also go use other things. If I need to drag some data into Excel to be a data janitor for a little bit, then, yeah, I can do that. So there's a bit of an ecosystem than a say a toolkit that I built up for myself now. And that's where I've kind of been spending some time and getting my hands on that curly data. Rob Collie (42:58): I'm all kinds of angry now. Thomas LaRock (42:59): Why is that? Rob Collie (43:00): I've got a couple of things to straighten out. First of all, in the answer to the question of, should I learn R or should I learn Python? The answer to that question is nine times out of 10, DAX. Thomas LaRock (43:14): All right. You're wrong, but that's okay. Rob Collie (43:19): Come on. There's a lot of trendiness in it. Now, there's still a tremendous usage of it. I'm not saying that learning Python is a bad thing. I think it's actually a really good thing. It's so often people's actual needs are better served by something that might not have that same kind of cool kids edginess to it. Thomas LaRock (43:39): Yeah. I wouldn't want to do a lot of... A lot of times I see Python being used as all these examples. Some of the ways they're manipulating data, to me, I'm not sure I would really want to do it that way. I would want to use a different type of tool like Excel or Power BI or something, because I'm a little more comfortable with that than what these lines of code are doing. Thomas LaRock (44:01): But if I want to build a model in machine learning, I could use Azure ML Studio. But under the hood, it's kind of just running the same code I could just do for myself. So I don't know. It's either/or, but I just feel that at the end of the day, Python just has a little bit more. Rob Collie (44:18): Yeah. I mean, it's just so often a lot of Python will be written to draw a chart. Thomas LaRock (44:26): Yeah, exactly. Oh no, you're right. Rob Collie (44:29): Or to do a very fundamental aggregation that would've been so much more powerful and flexible if you built a DAX data model around it. I even go to developer conferences on occasion now and the whole goal is to say, "Hey, look, you know so many things that I don't know, you're so much more technical than I, and yet I'm going to do some things up here on stage that you can't do, really important everyday things that you can't do and I want you to be upset about it." Rob Collie (45:06): Because I'm really just not that technical. I'm the least technical person at the company. Everyone we hire is so much better even at the things that I am good at. They're so much better at those than I am. A lot of things we're talking about like in the Azure platform, for instance, we have people who are very good at those things. I've never seen them. I haven't gotten the certification that I could even forget, like what you were talking about. Rob Collie (45:34): And then you said, "If I need to drag some data into Excel and be a data janitor." Come on now. Modern Excel that has the DAX engine and the power query engine in it. We've escaped that. We've escaped the janitor hood as long as we work in an organization that understands what we can do, which, again, that human factor. Most companies are very, very, very slow to wake up to the fact that their resident Excel guru has now become a completely new species. Rob Collie (46:03): The person who discovers what I call modern Excel, which is really the Power BI engines, the under-the-hood engines baked into Excel, when they discover that or they discover Power BI itself, they feel like they're the first person to discover fire. They sit back at their desk and go, "Oh my God." And they say things like the equivalent of, "Did you see that?" And everyone looks at them like, "No, we didn't see anything. In fact, maybe you should get back to work." It's a very unsatisfying. Rob Collie (46:31): And then some period of time later, those people end up working for us. That's where our employees are made, is in those trenches. I was mostly just joking. You just equated Excel and data janitor hood so glibly that I had to circle back. I had to say something. Thomas LaRock (46:52): I think it Excel is the tool of choice for most data janitors. We should make a commercial. Rob Collie (46:58): That's true. That's true. We've experimented with some advertising like on Facebook. It's not running at the moment. The ad says, "Are you running a spreadsheet sweatshop?" Thomas LaRock (47:11): Yes. I think I've seen that. Rob Collie (47:13): Yeah. We have these people sitting in what looks like a bombed out factory, but there's all these spreadsheets on these monitors and everything. The reason I don't like the janitor term is because it sticks to the person more than it sticks to the org. That's why the spreadsheet sweat shop, I prefer that nomenclature. That's not the preferred nomenclature, dude. Thomas LaRock (47:40): So I totally get how you have that apprehension about using the data janitor term. But I want you to know that in those courses I was doing, to earn that certification, one, or actually more than one, was taught by a friend of yours, Wayne Winston. Rob Collie (47:56): Oh, The Wayne. Thomas LaRock (47:57): And Wayne opened my eyes to how to use Excel in so many wonderful ways with descriptive statistics. And that's the type of stuff I'm talking about. I'm talking about, hey, I have these columns. How many are missing values? How many are no? Stuff of that nature that a lot of people would use Python for, but for me, I might just use Excel for that from time to time. Rob Collie (48:19): Yeah. Good old rectangles. Thomas LaRock (48:21): Wayne was so good. Such great courses. Rob Collie (48:24): Was it live? Thomas LaRock (48:26): No, it wasn't live. It was recorded. Rob Collie (48:30): I tell you, a live course with Wayne would be another experience all together. He is such a character. I bet they had to edit him down to 20% of what... He used to visit Microsoft and believe it or not, teach classes to Microsoft's finance departments. Thomas LaRock (48:54): I believe that. Rob Collie (48:55): But then he'd come hang out with the Excel team in the evening and just like hold court. Oh man. It was like drinking from a fire hose. It was awesome. He lives near me. I mean, I'm in central Indiana now. I'm in Indianapolis and he's in Bloomington. I've been here for five years and we still haven't gotten together. That's on me. I'm probably not going to see him... Thomas LaRock (49:19): Can't get together now either. Rob Collie (49:20): Can't get together now either. Yeah. Thomas LaRock (49:23): So it's not all on you, like the last three months. Rob Collie (49:25): Oh yeah. I mean, I've got a good three or four month excuse now. I mean, I did reach out. We were going to get together, but then you know following up, that's the trick, isn't it? I think that's probably a pretty good place to wrap episode one. What do you think? Thomas LaRock (49:38): I think so. I think we battled long enough about nothing. Raw Data is really a podcast about nothing. Rob Collie (49:44): Is that what we're going to do? It's a podcast without substance. I look forward to doing more of these. We have not come close to talking about everything. We've got lots of ground to cover. Announcer (49:59): Thanks for listening to the Raw Data by P3 podcast. Find out what the experts at P3 can do for your business. Go to powerpivotpro.com. Interested in becoming a guest on the show? Email lukep@powerpivotpro.com. Have a data day!