Raw Data By P3 Adaptive For Those About to DAX, w/ Microsoft's Greg Beaumont
We didn't know what to expect when we sat down with Greg Beaumont, Senior Business Intelligence Specialist at Microsoft specializing in serving Microsoft's Healthcare space customers' technical Power BI issues. What we got was an insightful, delightful, and impactful conversation with a really cool and smart human!
References in this Episode:
The Future Will Be Decentralized-Charles Hoskinson
Episode Timeline:
- 3:10 - The magic of discovery with the Power Platform, It's all about the customers(and Greg has a LOT of customers!), and Greg's Data Origin Story
- 21:10 - The IT/Business Gap, Getting good BI and keeping data security is a tricky thing, The COVID Challenge hits Healthcare
- 43:00 - Power BI-Not just a data visualization tool, a very cool discussion on Genomics and using data to save lives, the importance of Data Modelling
- 59:10 - The Bitcoin Analogy, The VertiPaq Engine and when is Direct Query the answer
- 1:08:30 - We get a little personal with Greg, Azure/Power BI integration and Machine Learning, Cognitive Services and Sentiment Analysis
Episode Transcript:
Rob Collie (00:00:00): Hello, friends. Today's guest is Greg Beaumont from Microsoft. Like one of our previous guests, hopefully, Greg has one of those interface jobs. The place where the broader Microsoft Corporation meets its customers at a very detailed and on the ground level. On one hand, it's one of those impossible jobs. More than 100 customers in the healthcare space look to Greg as their primary point of contact for all things technical, around Power BI. That's a tall order, folks. And at the same time, it's one of those awesome jobs. It's not that dissimilar, really, from our job here at P3.
Rob Collie (00:00:45): In a role that, first of all, you get broad exposure to a tremendous number of organizations and their problems, you learn a lot super, super quickly. When you're doing it right, your work day is just nonstop magic. The power platform is magic and not really because of the technology, but instead because of its impact on the people who use it, who interact with it, who benefit from it, whose lives are changed by it. And again, I can't stress this enough, software usually doesn't do this. And as we talked with him, Krissy and I just couldn't stop nodding, because we could hear it, he lives it, just like we do. And I hope that just leaps out of the audio for you like it did for us.
Rob Collie (00:01:32): No surprises here, Greg didn't start his life as a data professional. He's our second guest on this show, whose original training was in biology. And so, some familiar themes come back again, that good data professionals come from a wide variety of backgrounds, that the hybrid tweeners between IT and business are really where the value is at today. And I love this about Greg, that we made a point of talking about how much easier it is today to break into the data profession than it's ever been and what an amazing thing that is to celebrate.
Rob Collie (00:02:06): We talked about COVID and specifically its impacts on the industry. How that has served as a catalyst for many organizations to rethink their analytic strategy, the implications of remote work, data privacy and security. And of course, it wouldn't be an episode of Raw Data, if we didn't nerd out about at least one thing. So, we get a little bit into genomics and the idea of DNA and RNA as forms of biological computer code. And as you'd expect, and want, Greg is far from a one dimensional data professional, just such an interesting person, authentically human, a real pleasure to speak with, so let's get into it.
Announcer (00:02:47): Ladies and gentlemen, could I have your attention, please.
Rob Collie (00:02:51): This is the Raw Data by P3 adaptive podcast with your host, Rob Collie. Find out what the experts at P3 Adaptive can do for your business. Just go to p3adaptive.com. Raw Data by P3 Adaptive is data with the human element.
Rob Collie (00:03:13): Welcome to the show, Greg Beaumont. How are you?
Greg Beaumont (00:03:17): I'm doing well. How are you all?
Rob Collie (00:03:19): I think we're doing pretty well.
Greg Beaumont (00:03:19): Awesome.
Rob Collie (00:03:20): Business is booming. Data has turned out to be relatively hot field, but I think it's probably got some legs to it. And the Microsoft platform also, well, it's just kind of kicking ass, isn't it? So, business wise, we couldn't be better. I think personally, we're doing well, too. We won't go into all that. What are you up to these days? What's your job title and what's an average day look for you?
Greg Beaumont (00:03:39): So, I'm working in Microsoft and my title is Technical Specialist. And I'm a Business Intelligence Technical Specialist, so I focus almost exclusively on Power BI and where it integrates with other products within the Microsoft stack. Now, I'm in the Microsoft field, which is different from a number of guests you've had, who work at corporate and we're working on the product groups, which is that I'm there to help the customers.
Greg Beaumont (00:04:01): And you hear a lot of different acronyms with these titles. So, my role is often called the TS. In the past, it was called a TSP. It's just a change in the title. Sometimes you might hear the title, CSA, Cloud Solution Architect. It's very similar to what I do, but a little bit different. But effectively from an overarching standpoint, our goal in the field as Technical Specialists is to engage with customers, so that they understand how and where to use our products, and to ensure that they have a good experience when they succeed.
Rob Collie (00:04:29): Your job is literally where the Microsoft organism meets the customers.
Greg Beaumont (00:04:34): Yep.
Rob Collie (00:04:35): That's not the role I had. I was definitely on the corporate side, back in my days at Microsoft. I think the interaction between the field and corporate has gotten a lot stronger over the years. I think it's a bit more organic, that interplay, that it used to feel like crossing a chasm sort of thing. And I don't think that's really true anymore.
Greg Beaumont (00:04:54): At a green, I think that's by design, too. So, with the more frequent release schedules and also kind of how things have changed under Satya, customer feedback drives the roadmap. So when these monthly updates come out, a lot of it is based off of customer demand and what customers are encountering and what they need. So, we're able to pivot and meet the needs of those customers much more quickly.
Rob Collie (00:05:15): Yeah, you mentioned the changing acronyms, right? I mean like yes. My gosh, a thousand times yes. It's almost like a deliberate obfuscation strategy. It's like who's what? Why did we need to take the P off of TSP? I mean, I'm sure it was really important in some meeting somewhere, but it's just like, "Oh, yeah, it's really hard to keep track of." It's just a perpetually moving target. But at the same time, so many fundamentals don't change, right? The things that customers need and the things that Microsoft needs to provide. The fundamentals, of course, evolving, but they don't move nearly as fast as the acronym game.
Greg Beaumont (00:05:52): Right. I think that acronym game is part of what makes it difficult your first year here, because people have a conversation and you don't know what they're talking about. Right?
Rob Collie (00:06:00): Yeah, yeah, yeah.
Greg Beaumont (00:06:00): And if they just spelled it out, it would make a lot more sense.
Rob Collie (00:06:03): Krissy was talking to me today about, "Am I understanding what Foo means?" There's an internal Microsoft dialect, right? Krissy was like, "Is Foo like X? Is it like a placeholder for variable?" I'm like, "Yes, yes." She's like, "Okay. That's what I thought, but I just want to make sure."
Krissy Dyess (00:06:18): That's why there's context clues in grade school really come into play when you're working with Microsoft organization, because you really got to take in all the information and kind of decipher it a bit. And those context clues help out. Greg, how long have you been in that particular role? Has it been your whole time at Microsoft or are have you been in different roles?
Greg Beaumont (00:06:36): So, I should add, too, that I'm specifically in the healthcare org, and even within healthcare, we've now subspecialized into sub-verticals within healthcare. So, I work exclusively with healthcare providers, so people who are providing care to patients in a patient care setting. I do help out on a few other accounts, too, but that's my primary area of responsibility.
Greg Beaumont (00:06:55): So, I started with Microsoft in 2016. I was actually hired into a regional office as what's called the traditional TSP role and it was data platform TSP. So, it was what used to be the SQL Server TS role. A few months later, the annual realign happened, I got moved over to Modern Workplace because they wanted to have an increased focus on Power BI, and I had some experience in that area. Plus, I was the new guy, so they put me into the experimental role. A year later, that's when they added the industry verticals and that's when I moved into what is kind of the final iteration of my current role. And the titles have changed a few times, but I've effectively been in this role working with healthcare customers for over four years now.
Rob Collie (00:07:35): And so, like a double vertical specialization?
Greg Beaumont (00:07:37): Yeah.
Rob Collie (00:07:37): Healthcare providers, where there's a hierarchy here?
Greg Beaumont (00:07:40): Yeah, yeah.
Rob Collie (00:07:41): Those are the jaw dropping things for me is sometimes people in roles like yours, even after all that specialization, you end up with a jillion customers that you're theoretically responsible for. Double digits, triple digits, single digits in terms of how many customers you have to cover?
Greg Beaumont (00:07:58): I'm triple digits. And that is one of the key differences from that CSA role that you'll see on the Azure team is they tend to be more focused on just a couple of customers and they get more engaged in kind of projects. And I will do that with customers, but it's just, it's a lot more to manage.
Rob Collie (00:08:14): Yeah. What a challenging job. If you think about it, the minimum triple digit number is 100, right? So, let's just say, it's 100 for a moment. Well, you've got 52 weeks a year plus PTO, right? So, you're just like, "Okay." It is very, very difficult to juggle. That's a professional skill that is uncommon. I would say that's probably harder than the acronym game.
Greg Beaumont (00:08:37): Yeah, there's been times I was on a vacation day and I got a call. I didn't recognize the number. I'm like, "Okay, I'm going to have to route this to somebody because I'm off today." And they're like, "Well, I'm the VP of so and so and we need to do this." And I'm like, "Okay, I got to go back inside and work now, because this is an important call." So, you have to be flexible and you're correct, that it makes it a challenge to have that work-life balance also, but the work is very rewarding, so it's worth it.
Rob Collie (00:09:01): Yeah. It's something that vaguely I have a sense of this. I mean, transitioning from corporate Microsoft to, I mean, you can think of my role now as field. I'm much, much closer to the customers than I ever was at corporate. And yes, Brian Jones and I talked about it a little bit. And this is a bit of an artifact of the old release model that it was like every few years, you'd release a product, which isn't the case anymore. But that satisfying feeling of helping people, like even if you build something amazing back at Microsoft in the days that I was there, you were never really around for that victory lap. You would never get that feedback. It even never make it to you.
Rob Collie (00:09:37): It was years later muted whereas one of the beautiful things about working closely with customers and our clients with Power BI, and actually the Microsoft platform as a whole, is just how quickly you can deliver these amazingly transformational like light up moments that go beyond just the professional. You can get this emotional, really strong validating emotional feeling of having helped. And that is difficult to get, I think even today, probably, even with their monthly release cycles, et cetera. By definition, you're just further removed from the "Wow" that happens out where the people are.
Greg Beaumont (00:10:15): Yep. And I'm sure you all see that, too, with your business is that a lot of work often goes into figuring out what needs to be in these solutions and reports, but when you actually put it in the hands of leaders, and they realize the power of what it can provide for their business, in my case for their patients, for their doctors, for their nurses, it becomes real. They see it's actually possible and it's not just a PowerPoint deck.
Rob Collie (00:10:38): And that sense of possibility, that sense of almost child-like wonder that comes back at those moments, you just wouldn't expect from the outside. I had a family member one time say, "Oh, Rob, I could never do what you do." Basically, it was just saying "How boring it must be, right?" It's so boring working with software, working with..." I'm like, "Are you kidding me? This is one of the places in life where you get to create and just an amazingly magical." It's really the only word that comes close to capturing it. You just wouldn't expect that, right? Again, from the outside like, "Oh, you work in data all day. Boring."
Greg Beaumont (00:11:17): I'd add to that, that I'd compare it to maybe the satisfaction people get out of when they beat a game or a video game. That when you figure out how to do a solution and it works and you put in that time and that effort and that thought, there's that emotional reward, you get that I built something that that actually did what they wanted it to do.
Rob Collie (00:11:35): Yeah. And after you beat the video game, not only did that happen, but other people's lives get better as a result of you beating this game. It's just like it's got all those dynamics, and then some. All these follow on effects.
Greg Beaumont (00:11:46): It's like being an athlete and enjoying the sport that you compete in.
Rob Collie (00:11:50): Yeah. We're never going to retire. We're going to be the athletes that hang on way too long.
Greg Beaumont (00:11:56): Yep.
Rob Collie (00:11:58): So, unfortunately, I think our careers can go longer than a professional athletes, so there's that. I can't even really walk up and down stairs anymore without pain, so. So what about before Microsoft? What were you up to beforehand and how did you end up in this line of work in the first place?
Greg Beaumont (00:12:15): Sure. And I think that's actually something where listeners can get some value, because the way I got into this line of work, I think today, there's much more opportunity for people all over the world from different socioeconomic backgrounds to be able to break into this field without having to kind of go through the rites of passage that people used to. So, I was actually a Biology major from a small school. Came from a military family. I didn't have corporate contacts or great guidance counseling or anything like that. My first job right out of school was I said, "Oh, I got a Biology major. I got a job at a research institution." They're like, "Okay, you're going to be cleaning out the mouse cages." And it was sort of $10.50 an hour.
Greg Beaumont (00:12:53): So, at that point, I said, "Okay, I got to start thinking about a different line of work here." So, I kind of bounced around a little bit. I wanted to get into IT, but if you wanted to learn something like SQL Server, you couldn't do it unless you had a job in IT. As an average person, you couldn't just go buy a SQL Server and put it in your home unless you had the amount of money that you needed to do that. Side projects with Access and Excel. Small businesses did things probably making less than minimum wage and side gigs, in addition to what I was doing for full-time work to pay the bills. Eventually caught on with a hospital where I was doing some interesting projects with data using Access and Excel. They wouldn't even give me access to Crystal Reports when we wanted to do some reporting. That was really where I kind of said , "Data is where I want to focus."
Greg Beaumont (00:13:41): We did some projects around things like Radon Awareness, so people who would build a new house now, they're like, "Oh, I have to pay $1500 for that Radon machine down in the basement." But when you talk to a thoracic surgeon and their nursing team and you hear stories about people who are nonsmokers, perfectly healthy, who come in with tumors all over their lungs, you realize the value there and by looking at the data of where there's pockets of radon in the country reaching out to those people has value, right? I think it's that human element where you're actually doing something that makes a difference. So, that kind of opened my eyes.
Greg Beaumont (00:14:14): I then after that job, I got on with a small consulting company. I was a Project Manager. It was my first exposure to Microsoft BI. It was actually ProClarity over SQL Server 2005 and we were working with data around HEDIS and Joint Commission healthcare performance measures for one of the VA offices. So, I was the PM and the Data Architect was building the SSIS packages, built out kind of skeleton of an analysis services cube. He asked me to lean in on the dashboarding side, and that's also where I started learning MDX because we were writing some MDX expressions to start doing some calculations that we were then exposing in ProClarity. And at that point, it was like, "This is magic."
Greg Beaumont (00:14:57): From a used case perspective, what they were doing traditionally doing was they'd send somebody in from some auditing agency, who would look at, I think it was 30 to 60 patient records, for each metric and then they take a look at where all of the criteria hit for that metric, yes or no. And it would be pass/fail, how good is this institution doing of meeting this particular expectation. So, it would be things like, "Does a patient receive aspirin within a certain amount of time that they've been admitted if they have heart problems?" Something like that. With looking at it from a data perspective, you can look at the whole patient population, and then you could start slicing and dicing it by department, by time of day that they were admitted, by all of these different things.
Greg Beaumont (00:15:38): And that's when I kind of said, "This is really cool, really interesting. I think there's a big future here." And I kind of decided to take that route. And from there, I got on with a Microsoft partner, where I stayed for about six years. And that's kind of where I was exposed to a lot of very smart, very gifted people. And I was able to kind of learn from them and then that led to eventually getting a job at Microsoft. But to make a long story short, today, you could go online and get Power BI Desktop for free. There's training resources all over the place, and you could skill up and get started and get a great job. I'd like to tell people take the amount of time you spend every night playing video games and watching television, take half that time and devote it to learning Power BI and you'll be amazed at how far you get in six to 12 months.
Rob Collie (00:16:24): That's such good advice. I'm not really allowed to play a lot of video games, so I might need more time than that. But I had my time to do that years ago, learning DAX and everything. A couple of things really jumped out at me there. First of all, you're right, it was almost like a priesthood before. It was so hard to get your foot in the door. Look, you had to climb incrementally, multiple steps in that story to just get to the point where you were sitting next to the thing that was SSIS and MDX which, again, neither of those things had a particularly humane learning curve. Even when you got there, which was a climb, you get to that point and then they're like, "And here's your cliff. Your smooth cliff that you have to scale. If you wanted a piece of this technology," right?
Rob Collie (00:17:11): You wanted to learn MDX, you had to get your hands on an SSAS server. The license for it. And then you had to have a machine you could install it on that was beefy enough to handle it. It's just, there's so many barriers to entry. And the data gene, I like to talk about, it does. It cuts across every demographic, as far as I can tell, damn near equally everywhere. Let's call it one in 20. It's probably a little less frequent than that. Let's call it 5% of the population is carrying the data gene and you've got to get exposure. And that's a lot easier to get that exposure today than it was even 10 years ago.
Greg Beaumont (00:17:50): I'd completely agree with that. The people in this field tend to be the type of people who likes solving puzzles, who like building things that are complex and have different pieces, but who also enjoy the reward of getting it to work at the end. You've had several guests that have come on the show that come from nontraditional backgrounds. But I'm convinced that 20 years ago, there were a lot of people who would have been great data people, who just never got the opportunity to make it happen.
Greg Beaumont (00:18:14): Whereas today, the opportunity is there and I think Microsoft has done a great job with their strategy of letting you learn and try Power BI. You can go download the dashboard in a day content for free and the PDF is pretty self-explanatory and if you've used excel in the past, you can walk through it and teach yourself the tool. I think the power of that from both the perspective of giving people opportunity and also building up a workforce for this field of work is amazing.
Rob Collie (00:18:42): Yeah. I mean, all those people that were sort of in a sense like kind of left behind, years ago, they weren't given an avenue. A large number of them did get soaked up by Excel. If they're professionally still active today, there's this tremendous population of Excel people if they were joining the story today, they might be jumping into Power BI almost from the beginning, potentially. And of course, if they were doing that, they'd still be doing Excel. But there's still this huge reservoir of people who are still tomorrow, think about the number of people tomorrow, just tomorrow. Today, they're good at Excel and tomorrow, they will sort of, they'll have their first discovery moment with Power BI. The first moment of DAX or M or whatever, that's a large number of people tomorrow who are about to experience. It's almost like did you see the movie The Game?
Greg Beaumont (00:19:36): I have not.
Rob Collie (00:19:37): There's this moment early in the movie where Michael Douglas has just found out that his brother or something has bought them a pass to the game. And no one will tell him what it is. He meets this guy at a bar who says, "Oh, I'm so envious that you get to play for the first time." Also, this is really silly, but it's also like the ACDC song For Those About To Rock, We Salute You. For those about to DAX, we salute you, because that's going to happen tomorrow, right? Such a population every day that's lighting up and what an exciting thing to think about. Do you ever get down for any reason, just stop and think, "Oh, what about the 5000 people today who are discovering this stuff for the first time." That is a happy thing.
Greg Beaumont (00:20:16): Yeah, I actually had a customer where one of their analysts who turned out to be just a Power BI Rockstar, he said, "I'd been spending 20 years of my life writing V-lookups, and creating giant Excel files. And now, everything I was trying to do is at my fingertips," right? And then within a year, he went from being a lifelong Excel expert to creating these amazing reports that got visibility within the organization and provided a ton of value.
Rob Collie (00:20:42): And that same person you're talking about is also incredibly steeped in business decision-making. They've been getting a business training their whole career at the same time. And it's like suddenly, you have this amazingly capable business tech hybrid, that literally, it just like moved mountains. It's crazy. We've talked about that a lot on the show, obviously, the hybrids, just amazing. And a lot of these people have come to work for us.
Rob Collie (00:21:09): That's the most common origin story for our consultants. It's not the only one. I mean, we do have some people who came from more traditional IT backgrounds, but they're also hybrids. They understand business incredibly well. And so, they never really quite fit in on the pure IT side, either. It's really kind of interesting.
Greg Beaumont (00:21:26): Yeah, I think there's still a gap there between IT and business, even in kind of the way solutions get architected in the field. It's understanding what the business really wants out of the tool is often very different from how IT understands to build it. And I think that's where people like that provide that bridge, to make things that actually work and then provide the value that's needed.
Rob Collie (00:21:47): There's such valuable ambassadors. It's just so obvious when IT is going to interact with a business unit to help them achieve some goal. It's so obvious that, of course, who you need to engage with IT. IT thinks, "We need to engage with the leaders of this business unit." They've got the secret weapon, these hybrid people that came up through the ranks with Excel. The word shadow IT is perfect. These people within the business, like they've been Excel people for their entire careers, they have an IT style job.
Rob Collie (00:22:22): Almost all the challenges that IT complains about with working with business, you take these Excel people and sort of put them in a room where they feel safe. They'll tell you the same things. They're like, "I had exactly the same problems with my 'users,' the people that I build things for." And yeah, there's such a good translator. And if the communication flows between IT and business sort of through that portal, things go so much better. That's a habit. We're still in the process of developing as a world.
Greg Beaumont (00:22:51): Yeah. And in healthcare that actually also provides some unique challenges. With regulation and personal health information, these Excel files have sensitive data in them, and you have to make sure it's protected and that the right people can see it. And how do you give them the power to use their skills to improve your organization, while also making sure that you keep everything safe. So, I think that's a hot topic these days.
Rob Collie (00:23:15): Yeah. I mean, it's one of those like a requirement, even of the Hello World equivalent of anything is that you right off the bat have to have things like row level security and object level security in place and sometimes obfuscation. What are some of the... we don't want to get to shop talky, but it is a really fascinating topic, what are the handful of go-to techniques for managing sensitive healthcare information? How do you get good BI, while at the same time protecting identity and sensitivity. So often, you still need to be able to uniquely identify patients to tie them across different systems, can identify them as people. It's really, really, really tricky stuff.
Greg Beaumont (00:24:02): And I think just to kind of stress the importance of this, you can actually go search for look up HIPAA wall of shame or HIPAA violation list. When this information gets shared with the wrong people, there's consequences and can result in financial fees and fines. And in addition to that, you lose the trust of people whose personal information may have been violated. So, I think a combination of you said things row level security and object level security as a start, you can also do data masking, but then there's issues of people export to Excel. What do they do with that data afterwards?
Greg Beaumont (00:24:37): And then there's going to be tools like Microsoft Information Protection, where when you export sensitive information to Excel, it attaches an encrypted component. I'm not an MIT expert. I know how it works. I don't know the actual technology behind it. But it attaches an encrypted component where only people who are allowed to see that information can then open that file. So, you're protecting the information at the source and in transit, but you're still giving people the flexibility to go build a report or to potentially use data from different sources, but then have it be protected every step of the way.
Greg Beaumont (00:25:11): So like you said, without getting too techie, there's ways to do it, but it's not just out of the box easy. There's steps you have to go through, talk to experts, get advice. Whether it's workshops or proof of concepts, there's different ways that customers can figure that out.
Rob Collie (00:25:28): Yeah. So because of that sort of mandatory minimum level of sensitivity handling and information security, I would expect, now that we're talking about it, that IT sort of has to be a lot more involved by default in the healthcare space with the solutions than IT would necessarily be in other industries. Another way to say it, it's harder for the business to be 100% in charge of data modeling in healthcare than it is in other industries.
Greg Beaumont (00:26:02): Yep. But you can have a hybrid model, which is where the business provides data that's already been vetted and protected and there might be other data that doesn't have any sensitive data in it, where it's game on, supply chain or something like that. But having these layers in between, the old way of doing things was just nobody gets access to it. Then there was kind of canned reporting where everybody gets burst in the reports that contain what they're allowed to see. But now, you can do things in transit, so that the end users can still use filters and build a new report and maybe even share it with other people. And know that whoever they're sharing with will only be able to see what they're allowed to see. It gets pretty complex, but it's definitely doable and the customers that are doing it are finding a lot of value in those capabilities.
Rob Collie (00:26:48): That's fundamentally one of the advantages of having a data model. I was listening to a podcast with Jeffrey Wang from Microsoft and he was talking about it. And I thought this was a really crisp and concise summary, which is that the Microsoft Stack Power BI has a model-centric approach to the world whereas basically, all the competitors are report centric. And what does that mean? Why does that even make a difference? Well, when you build a model, you've essentially built all the reports in a way. You've enabled all of the reports. You can build many, many, many, many, many like an infinite number of different reports based on emerging and evolving business needs without having to go back to square one.
Rob Collie (00:27:28): In a report-centric model, which is basically what the industry has almost always had, almost everywhere, outside of a few notable examples, Power BI being one of them. When a report centric model, every single change, I remember there being a statistic that was just jaw dropping. I forget what the actual numbers were, but it was something like the average number of business days it took to add a single column to a single existing report. It was like nine business days, when it should just be a click. And that's the difference. And so, preserving that benefit of this model centric approach, while at the same time, still making sure that everyone's playing within the right sandbox that you can't jump the fence and end up with something that's inappropriate. Very challenging, but doable.
Greg Beaumont (00:28:15): Yep. That reminded me of an old joke we used to tell in consulting and this was back in the SharePoint Performance Point with Analysis Services days is there be a budget for a project, there'd be change requests along the ways, they discover issues with the data. And at the very end of the project, they rushed the visualization to market. And they're like after six months, with 10 people dedicated on this project, "Here's your line chart."
Rob Collie (00:28:39): Yeah. I had a director of IT at a large insurance company one time, looking me in the eye and just brutally confess. Yeah, my team, we spent three months to put a dot on a chart. And that's not what you want.
Greg Beaumont (00:28:59): Right, right.
Rob Collie (00:29:01): That was unspoken. This was bad. To the extent that you're able to tell, what are some of the interesting things that you've seen in the healthcare space with this platform recently? Anything that we can talk about?
Greg Beaumont (00:29:15): Yeah, so I think I'd start with how everything changed with COVID. Just because I think people would be interested in that topic and kind of how it changed everything. I actually had a customer yesterday at a large provider who said, "COVID was the catalyst for us to reconsider our investment in analytics, and that it spurred interest from even an executive level to put more money into analytics because of the things that happened." So obviously, when it hit everybody was, "What in the world is going on here?" Right? "Are we even going to have jobs? Is the whole world going to collapse or is this just going to be kind of fake news that comes and goes?" Everybody was unsure what was going on.
Greg Beaumont (00:29:50): At the same time, the healthcare providers, a lot of them were moving people to work from home and these were organizations where they had very strict working conditions because of these data privacy and data security considerations, and all of a sudden, you're in a rush to move people home. So, some of my counterparts who do teams, they have some just amazing stories. They were up all night helping people set up ways to securely get their employees to a work-from-home type experience, so that they only had essential workers interacting with the patients, but then the office workers were able to effectively conduct business from home.
Greg Beaumont (00:30:25): Additionally, there were use cases that were amazing. So, Microsoft has now what's called the Cloud for Health where we're effectively taking our technology and trying to make it more targeted towards healthcare customers and their specific needs, because we see the same types of use cases repeat from customer to customer. One of those use cases that came out of COVID was called Virtual Visits. And I actually know the team that built that solution, but because of patients who were on COVID, they didn't know how contagious it was.
Greg Beaumont (00:30:56): There were people being put on ventilators, who weren't allowed to see their families and they were setting up a team's application, where people were actually able to talk to their family and see their family before they went under, right? There were chaplains who were reading people their last rites using video conferencing, and things like that. So, it was pretty heavy stuff, but I think from a healthcare perspective, it showed the value technology can provide.
Greg Beaumont (00:31:21): And from our perspective in the field, it's like we're not just out there talking about bits and bytes. It kind of hit home that there's real people who are impacted by what we're doing and it adds another kind of layer of gravity, I'd call it, taking what you do seriously, right? I had another customer, they were doing some mapping initiatives with some of the COVID data because they wanted to provide maps for their employees of where the hotspots were.
Greg Beaumont (00:31:46): And we were up till I think 11:00 at night one night working through a proof of concept. And they said, "Yeah, what's next is we also want to start mapping areas of social unrest." I said, "Wow, social unrest. Why are you worried about that?" And they said, "Well, we expect because of this lockdown, that eventually there's going to be rioting and issues in all different parts of the world." And at that time, I just kind of didn't really think about that, but then a lot of those things did happen. It was kind of just interesting to be working at night and hearing those stories, and then seeing how everything kind of unfolded.
Greg Beaumont (00:32:18): Another example, look it up, there's an Azure COVID Health Bot out there and then there's some information on that, where you can ask questions and walk through your symptoms, and it will kind of give you some instructions on what to do. Another one that is even popular now is looking at employees who are returning to work. So, when people return to work find out vaccination status, "Are you able to come back to work? Are you essential? Are you nonessential?" I don't think a lot of customers were prepared to run through that scenario when it hit.
Greg Beaumont (00:32:48): So, having these agile tools where you can go get your list of not only employees, but maybe partners that refer people to your network, because you might not have all the referring doctors in your system. So with Power BI, you can go get extracts, tie it all together and then build out a solution that helps you get those things done. I'd say it was eye opening. I think for customers and also for myself and my peers, that we're not just selling widgets. We're selling things that make a difference and have that human perspective to it.
Rob Collie (00:33:20): Yeah, that does bring it home, doesn't it? That statement from an organization that COVID was the catalyst, evaluating and investing in their analytic strategy?
Greg Beaumont (00:33:29): Yep.
Rob Collie (00:33:30): Being in BI, being an analytics is one of the best ways to future proof one's career because at baseline, it's a healthy industry, there's always value to be created. But then when things get bad, for some reason, whatever crisis hits, it's actually more necessary than ever because when you've been in a groove when a an industry or an organization has been in an operational groove for a long time, any number of years, eventually, you just sort of start to intuitively figure it out. There's a roadmap that emerges slowly over time. Now, even that roadmap probably isn't as good as you think it is. If you really tested your assumptions, you'd find that some of them were flawed and analytics could have helped you be a lot more efficient even then.
Rob Collie (00:34:14): But regardless, the perception is that we've got a groove, right? And then when the world completely changes overnight, all of your roadmaps, your travel roadmaps, none of them are valid anymore. And now, you need a replacement and you need it fast. And so, what happens is, is that analytics spending, BI spending, whatever you want to call it, or activity, actually increases during times of crisis. So, you got a healthy baseline business. It's an industry that's not withering and dying in good times, but it actually it's like a hedge against bad times.
Rob Collie (00:34:47): When I saw that research years and years ago, when I was working at Microsoft Corporate, we just come out of the dot-com crack up, we'd seen that BI spending it across the IT industry was the only sector that went up during that time where everything else was falling. It's like, "Oh, okay." So, not only do I enjoy this stuff, but I really should never get out of it. It's like one of the best future proofing career moves you can make is the work in this field. And so, I mean, we've seen it, right? The early days of the COVID crisis, you're right when no one knew the range of possible outcomes going forward was incredibly wide. The low end and the high end were exponentially different from one another.
Rob Collie (00:35:29): And so, we experienced in our business, sort of a gap in spring and early summer last year. We weren't really seeing a whole lot of new clients, people who are willing to forge a brand new relationship. Again, what happens when a crisis hits? You slam on the brakes. No unnecessary spending first of all. Let's get all the spending under control, because we don't know as a company what's going to happen in the industry, right? You see a lot of vendor spending freezes and of course, to other companies, we're a vendor, right? So, our existing clients, though, doubled down on how much they used us and how much they needed us.
Rob Collie (00:36:08): And then later in the year, the new client business returned, and we actually ended up, our business was up last year, despite that Q2 interruption and sort of making new friends. And this year, holy cow like whatever was bottled up last year is coming back big time. And so, yeah. You never really want to be the ghoul that sort of morbidly goes, "Oh, crisis." From a business perspective, yeah, anything that changes, anything that disrupts the status quo tends to lead to an increased focus on the things that we do.
Greg Beaumont (00:36:43): Yeah, I think something you said there, too, was when you don't know what's going to happen was when the business intelligence spending increased. I mean, the intelligence and business intelligence, it's not just a slogan. The purpose of these tools is to find out the things you don't know. So when there's uncertainty, that's when BI can provide that catalyst to sort of add some clarity to what you're actually dealing with.
Rob Collie (00:37:06): Yeah, I've been using, even though I'm not a pilot, I've never learned to fly a plane or anything. I've been using an aviation metaphor lately, which is windshield is nice and clear. You might not be looking at the instruments on your cockpit very much, right? You know there's not a mountain in front of you, you can see how far away the ground is. And you could sort of intuit your way along, right? But then suddenly, whoosh clouds. And oh, boy, now, you really need those instruments, right? You need the dashboards, you need the altimeter, you need the radar. You need all that stuff so much more.
Rob Collie (00:37:37): And so, and our business has kind of always been this. The reason I've been using this metaphor is really for us, it's like given how fast we operate, and I think you can appreciate this having come from a Microsoft partner consulting firm before Microsoft years ago, our business model, we move so fast with projects. We're not on that old model with the original budget and the change orders and all of that. That was all dysfunctional.
Rob Collie (00:38:01): It was necessary, because of the way software worked back then, but it was absolutely dysfunctional. It's not the way that you get customer satisfaction. So, we've committed to the high velocity model. But that means seeing the future of our business financially two months in the future is very difficult relative to the old sort of glacial pace, right? If there's a mountain there, we're going to have months to turn around it.
Krissy Dyess (00:38:26): To add a bit to your analogy there, Rob. I am married to a pilot and I have gone up in the small tiny airplane. And before the gadgets, there's actually the map. The paper map, right? So, you had the paper map, which my husband now would hand to me. And he'd tell me, "Okay, let me know the elevations of different areas to make sure we're high enough, we're not going to crash into the mountains."
Krissy Dyess (00:38:47): What's happened is people just they got used to different ways that they were doing things. They were forced into these more modern ways. And I think even now, this wave of seeing this catalyst we can change and how are other people changing is also driving the people to seek help from others in terms of getting guidance, right? Because even though you've had the change, it doesn't necessarily mean that the changes that you made were 100% the right way and you can learn so much from others in the community and the people that are willing to help.
Krissy Dyess (00:39:24): And I think that's one of the things too, that our company provides as a partner, we're able to kind of go alongside. We've seen what's works, what doesn't work, what are some of those pitfalls? What are those mountains approaching? And we're really able to help guide others that want to learn and become better.
Rob Collie (00:39:42): Yeah. I mean, this is us getting just a little bit commercial, but you can forgive us, right? That high velocity model also exposes us to a much larger denominator. We see a lot at this business that accumulates. The example I've given before is and this is just a really specific techy, so much of this is qualitative, but there's a quantitative. It's sort of like a hard example of like, "Oh, yeah, that's right. This pattern that we need here for this food spoilage inventory problem is exactly the same as this tax accounting problem we solved over there, right?" As soon as you realize that you don't need to do all the figuring out development work, you just skip to the end.
Rob Collie (00:40:22): And really, most of the stuff that Krissy was talking about, I think, is actually it's more of the softer stuff. It's more of the soft wisdom that accumulates over the course of exposure to so many different industries and so many different projects. That's actually really one of the reasons why people come to work here is they want that enrichment.
Greg Beaumont (00:40:38): Yeah, that makes sense. Because you see all these different industries and you actually get exposed to customers that are the best in the business for that type of, whether it be a solution or whether it be a product or whether it be like a framework for doing analytics or something like that. So, you get that exposure and you also get to contribute.
Rob Collie (00:40:55): Even just speaking for myself, in the early days of this business, when it was really still just me, I got exposure to so many business leaders. Business and IT leaders that, especially given the profile of the people who would take the risk back in 2013, you had to be some kind of exceptional to be leaning into this technology with your own personal and professional reputation eight years ago, right? It was brand new. So, imagine the profile of the people I was getting exposed to, right? Wow, I learned so much from those people in terms of leadership, in terms of business. They were learning data stuff from me, but at the same time, I was taking notes.
Greg Beaumont (00:41:33): Everybody was reading your blog, too. I can't count the number of times I included a reference to one of your articles to help answer some questions. And it was the first time I was introduced to the Switch True DAX statement. And then I'd print that.
Rob Collie (00:41:47): Which-
Greg Beaumont (00:41:48): Sent that link to many people. "Don't do if statements, do this. Just read this article."
Rob Collie (00:41:53): And even that was something that I'd saw someone else doing. And I was like, "Oh, my God, what is that?" My head exploded like, "Oh." Yeah, those were interesting days. I think on the Chandu podcast, I talked about how I was writing about this stuff almost violently, couldn't help it. It was just like so fast. Two articles a week. I was doing two a week for years. There was so much to talk about, so many new discoveries. It was just kind of pouring out in a way.
Krissy Dyess (00:42:24): Greg, you came in to the role around 2016. And to me 2017 was really that big year with the monthly releases where Power BI just became this phenomenon, right? It just kept getting better and better in terms of capabilities and even the last couple years, all the attention around security has been huge, especially with the health and life science space. And last year, with this catalyst to shift mindsets into other patterns, working patterns using technology, do you feel like you've seen any kind of significant shifts just compared to last year or this year?
Greg Beaumont (00:43:05): Yeah. And so something that burns my ears every time I hear it is when people call Power BI a data visualization tool. It does that and it does a great job.
Rob Collie (00:43:11): I hate that.
Greg Beaumont (00:43:12): But it's become much more than that. When it launched, it was a data visualization tool. But if you think about it at that time, they said, "Well, business users can't understand complex data models, so you have to do that in analysis services." Then they kind of ingested analysis services into Power BI and made it more of a SaaS product where you can scale it. There's Dataflows, the ETL tool, which is within Power BI, which is an iteration of Power Query, which has been around since the Excel days. So, now you have ETL. You have effectively from the old SQL Server world, you have the SSIS layer, you have the SSAS layer. With paginated reports, you have the SSRS layer. And you have all these different layers of the solution now within an easy to use SaaS product.
Greg Beaumont (00:43:55): So this evolution has been happening, where it's gobbling up these other products that used to be something that only central IT could do. And now, we're putting that power by making it easier to use in the hands of those analysts who really know what they want from the data. Because if you think about it, the old process was is you go and you give the IT team your requirements, and they interpret how to take what you want, and translate it into computer code.
Greg Beaumont (00:44:21): But now, we're giving those analysts the ability to take their requirements and go do it themselves. And there's still a very valid place for central IT because there's so many other things they can do, but it frees up their time to work on higher valued projects and I see that continuing with Power BI, right? But like we're adding AI, ML capabilities and data volumes keep increasing then capabilities I think will continue to expand it.
Rob Collie (00:44:46): Greg, I used to really caused a storm when I would go to a conference that was full of BI professionals. And I would say that something like, "What percentage of the time of BI project, traditional BI project was actually spent typing the right code?" The code that stuck, right? And I would make the claim that it was less than 1%. So, it's like less than 1% of the time of a project, right? And everyone would just get so upset at me, right? But I just didn't understand why it was controversial.
Rob Collie (00:45:19): Like you describe like yeah, we have these long requirements meetings in the old model. Interminably long, exhausting, and we'd write everything down. We'd come up with this gigantic requirements document that was flawed from the get-go. It was just so painful. It's like the communication cost was everything and the iteration and discovery, there wasn't enough time for that. And when I say that the new way of building these projects is sometimes literally 100 times faster than the old way. Like it sounds like hyperbole.
Greg Beaumont (00:45:53): It's not. Yeah.
Rob Collie (00:45:54): It can be that fast, but you're better off telling people, it's twice as fast because they'll believe you. If you tell them the truth, they'd go, "Nah, you're a snake oil salesman. Get out of here."
Greg Beaumont (00:46:07): Yeah. And I think the speed of being able to develop, too, it's going to basically allow these tools to be able to do things that people didn't even dream of in the past. It's not just going to be traditional business use cases. I know in healthcare, something that's a hot topic is genomics, right? Genomics is incredibly complex then you go beyond Power BI and into Azure at that point, too and Cloud compute and things like that.
Greg Beaumont (00:46:31): So, with Genomics, you think about your DNA, right? Your DNA is basically a long strand of computer code. It is base pairs of nucleic acids, adenine, thymine, and guanine, cytosine that effectively form ones and zeros in a really long string.
Rob Collie (00:46:46): Did you know it effortlessly he named those base pairs? There's that biology background peeking back out.
Greg Beaumont (00:46:52): I did have to go look it up before the meeting. I said, "Just in case this comes up, I need to make sure I pronounce them right," so.
Rob Collie (00:46:59): Well, for those of us who listen to podcasts at 1.5x speed, that is going to sound super impressive, that string there.
Greg Beaumont (00:47:05): Yeah. I should call out, too, though that I'm not a genomics expert, so some of what I'm saying here, I'm paraphrasing and repeating from people I've talked to who are experts, including physicians and researchers. So, this long string of code, if you sequence your entire genome, the file is about 100 gigabytes for one person, okay? At 100 gigabytes, you can consume that, but if you want to start comparing hundreds of people and thousands of people in different patient cohorts, all of a sudden, it gets to be a lot of information and it gets very complex.
Greg Beaumont (00:47:35): If you think of that strand of DNA as being like a book with just two letters that alternate, there's going to be paragraphs and chapters and things like that, which do different things. So, one of the physicians I spoke to worked with Children's Cancer. Here's kind of where the use case comes in. So, you take something breast cancer where there's BRCA1 or BRCA2, BRCA1, BRCA2 genes where if you have it, there's a measurable increased probability that you'll get that type of cancer within a certain age range. There's a lot of other diseases and cancers, where it might be 30 genes. And depending on different combinations of those genes, it changes the risk of getting that specific type of cancer.
Greg Beaumont (00:48:17): But this physician told me that there are specific children's cancers, where they know that if they have certain combinations of genes, that they have a very high probability of getting this cancer. And when the child actually feel sick and goes to the doctor, it's already spread and it's too late. So, if you can do this sequencing, basically run it through machine learning algorithms, so it will determine the probability, you could effectively catch it at stage zero. Because these cancers, it's something that could be related to growth hormones and as you're growing up, and as you become an adult, you're then no longer at risk of getting that childhood cancer. So, if they could identify it early and treated at stage zero, instead of stage 4, it sounds sci-fi, but the tools are there to do it.
Greg Beaumont (00:49:01): It just never ceases to amaze me that you watch the news and they talk about self-driving cars and identifying when a banana is ripe, and things like that. But it's like, you know what? These same tools could be out there changing people's lives and making a measurable difference in the world. I think just especially post COVID, I'll expect to see a lot more investment in these areas. And also, interest because I think that might be one of the positives that comes out of this whole experience.
Rob Collie (00:49:27): I do think that the sort of the worlds of Medicine and Computer Science are on a merging course. Let's not call it collision course. That sounds more dramatic. There is a merging going on. You're right DNA is biologically encoded instructions by an RNA. The mRNA vaccine is essentially injecting the source code that your body then compiles into antibodies. It's crazy and it's new. There's no two ways about it.
Rob Collie (00:49:56): mRNA therapies, in general, which of course they were working on originally as anticancer and sort of just like, "Oh, well, we could use it for this, too." And there's all kinds of other things too, right? Gosh, when you go one level up from DNA or some point of abstraction, you get into protein folding. And whoa, is that...
Greg Beaumont (00:50:15): Crazy, yeah.
Rob Collie (00:50:16): ... computationally. We're all just waiting for quantum computers, I think.
Greg Beaumont (00:50:20): Now, I'll have to call out that I'm making a joke here, so people don't take me seriously. But if you think about it, the nucleus in each of your cells contains an important model of that DNA, right? There isn't just a central repository that everything communicates with. You have a cache of that DNA in every cell in your body, except red blood cells, which perform a specific task. There may be more of the power automated the human body. But cheap attempt at a joke there, so.
Rob Collie (00:50:44): Well, I like it, I like it. Let's go in with both feet. I've also read that one of the reasons why it's difficult to clone adult animals is because you start off with your original DNA, but then you're actually making firmware updates to certain sections of the DNA throughout your life. And so, those edits that are being made all the time are inappropriate for an embryo.
Greg Beaumont (00:51:09): Yep.
Rob Collie (00:51:10): And so, if you clone, you create an embryo, right? And now, it's got these weird adult things going on in it. That's why things kind of tend to go sideways. It can all come back to this notion of biological code and it's fascinating. A little terrifying, too, when you start to think of it that way. I've listened to some very scary podcasts about the potential for do-it-yourself bioweapon development. There was this explosion back, in what, in the '90s when the virus and worm writers discovered VVA. Remember that? We call them the script kiddies that would author these viruses that would spread throughout the computer systems of the world. And a lot of them, the people writing these things were not very sophisticated. They weren't world renowned hackers.
Greg Beaumont (00:51:53): For every instance where you can use this technology to cure cancer, you're right that there's also the possibility of the Island of Dr. Moreau, right? You go look up CRISPR Technology, C-R-I-S-P-R, where they can start splicing together things from different places and making it viable. And 10 years ago, they had sheep that were producing spider webs in their milk and it's just, there's crazy stuff out there if you kind of dive into the dark depths of Biology. Now that we went down the rabbit hole, how do we correct course, right?
Rob Collie (00:52:23): Well, we did go down a rabbit hole, but who cares? That's what we do.
Greg Beaumont (00:52:26): Even you kind of step it back up to just kind of easy use cases in healthcare, so one of the ones that we use as a demo a lot came from a customer, and this was pre-COVID. But something as simple as hand washing, you don't think about it much. But when you're in the hospital, how many of those people are washing their hands appropriately when they care for you. And there's some white papers out there, which are showing that basically, there are measurable amounts of infections that happen in hospitals due to people not washing their hands appropriately. So, a lot of healthcare organizations will anonymously kind of observe people periodically to see who's doing a good job of washing their hands.
Rob Collie (00:53:04): I was going to ask, how is this data collected?
Greg Beaumont (00:53:06): This customer actually had nurses who were using a clipboard and they would write down their notes, fax it somewhere, and then somebody would enter it into Excel. So, there was this long process. And with another TS, who covers teams, we basically put a PLC together in a couple days, where they enter the information into a power app within teams, so they made their observation, entered it in. It did a write back straight to an Azure SQL Database at that time. Now, they might use the data verse. And then from Azure SQL DB, you can immediately report on it and Power BI. It even set up alerts, so that if somebody wasn't doing a good job, you could kind of take care of the situation, rather than wait for two days for the Excel report to get emailed out, and maybe lower the infection rates in the hospital.
Greg Beaumont (00:53:53): So, it saved time from the workers who are writing things down and faxing things just from a sheer productivity perspective. But it also hopefully, I don't know if it will be measurable or not, but you'd have some anticipated increase in quality, because you're able to address issues faster. And that's the simplest thing ever, right? You can spend a billion dollars to come up with a new drug or you can just make sure are people washing their hands.
Rob Collie (00:54:17): Both data collection and enforcement, they happen to be probably the same thing. There's like, "Oh, I'm being watched." The anonymity is gone. That's a fascinating story. Okay. What kinds of solutions are you seeing these days? What's happening out in the world that you think is worth talking to the audience about?
Greg Beaumont (00:54:38): We're seeing this ability to execute better where the tools are easier to use, you can do things faster, but there's still challenges that I see frequently out there. So, I know something that you all are experts in its data modeling and understanding how to take a business problem and translate it into something that's going to perform well. So, not only do you get the logic right, but when somebody pushes a button they don't have to go to lunch and come back, they get a result quickly. That's still a challenge. And it's a challenge, because it's not always easy, right? I mean, it's the reason cubes were created in the first place was because when you have complex logic and you're going against a relational database, the query has to happen somewhere, but like that logic.
Greg Beaumont (00:55:19): So take for example, if somebody wants to look at year over year percent change for a metric and they want to be able to slice it by department, maybe by disease group, maybe by weekend versus weekday, and then they want to see that trend over time. If you translate that into a SQL query, it gets really gnarly really fast. And that problem is still real. One of the trends I'm seeing in the industry is there's a big push to do everything in DirectQuery mode, because then you can kind of manage access, manage security, do all of those necessary security things in one place and have it exist in one place.
Greg Beaumont (00:56:00): But when you're sending giant gnarly SQL queries back to relational databases, even if they're PDWs with multiple nodes, it gets very expensive from a compute perspective, and kind of when you scale out to large number of users, concurrency is still an issue. So that's something where you look at recently what Power BI has come out with aggregations and composite models. That's some of the technology that I think can mitigate some of those problems. And even if we think about something like Azure synapse, right? You can have your dedicated SQL pools then you can have a materialized view. A materialized view is effectively a cache of data within synapse, but then you can also have your caches in Power BI, and kind of layer everything together in a way that's going to take that logic and distribute it.
Greg Beaumont (00:56:46): Does that make sense?
Rob Collie (00:56:47): It does. I think this is still a current joke. The majority of cases where we've encountered people who think they want or need DirectQuery, the majority of them are actually perfect poster children case studies for when you should use cash and import mode. Right? It turns out the perceived need for DirectQuery, there is a real percentage of problems out there for which DirectQuery is the appropriate solution and it is the best solution. But it's the number of times people use it is a multiple of that real ideal number.
Rob Collie (00:57:17): I think part of it is just familiarity. Still, I've long talked about how we're still experiencing as an industry the hangover from most data professionals being storage professionals. Everyone needed a database, just to make the wheels go round. The first use of data isn't BI. The first use of data is line of business applications. Every line of business application needed a database, right? So, we have minted millions of database professionals. this is also why I think partly why Power BI gets sort of erroneously pigeonholed as a visualization tools, because people are used to that. They're used to, we have a storage layer and reports layer, that's it, right?
Rob Collie (00:57:56): Reporting services was Microsoft's runaway successful product in this space. Paginated reports is still around for good reason. And I think that if you're a long-term professional in this space with a long history, even if you're relatively young in the industry, but you've been working with other platforms, this storage layer plus visuals layer is just burned in your brain. And this idea of this like, "Why do you need to import the data? Why do you need a schedule? Why do you need all this stuff?" It's like as soon as people hear that they can skip it, and go to DirectQuery, they just run to the comfort zone in a way, right?
Greg Beaumont (00:58:32): Yeah.
Rob Collie (00:58:32): I'm teaching DAX and data modeling to the Excel crowd. I have a real tortured relationship with the related function. Should I tell them about it in their first class? Because I know what's going to happen. They're going to use it and they're going to gravitate right back to that one giant Franken table model where they use the relationships and then use the related function to turn them all into one big wide table and miss the whole point. And so, it's like, "Do I even tell you about it?" It's like, "Do I even tell the IT director that DirectQuery is a thing?" Because, again, it has its purpose. I'm glad it's on the platform, but it's overused.
Greg Beaumont (00:59:09): I think people confuse single source of truth with a single source of data.
Rob Collie (00:59:13): Totally. I've heard people say, "How many copies of the data do I need in my organization?" Right? In a very folksy combative tone. Well, you like caches? What about caches? Are you okay with caches?
Greg Beaumont (00:59:24): And this is another analogy I sometimes use and it's intended to be humorous and keep people's attention. I'm not trying to make a direct comparison here. I just want to call that out. But I call it the Bitcoin problem. So, with Bitcoin, it can handle I think it's 4.7 transactions per second. And people want to use it as a currency the way you use a credit card where Visa may be handled 1700 transactions per second. So there's a problem with going DirectQuery against Bitcoin and that it can't handle the concurrency and the scale.
Greg Beaumont (00:59:55): And so, there's a lot of these crypto projects out there that are trying to create basically ways to kind of resolve all the transactions and then periodically true up with the source. And I'm not an expert in that area either. I just, I think it's fun to read about. During COVID, I watched some things on Bitcoin when I was stuck at home. I saw a presentation, if anybody gets a chance to check it out, called The Future Will Be Decentralized by Charles Hoskinson, who was the founder of Cardano. And that's when it kind of clicked that they're not just creating fund money, they're creating effectively a whole new economic system or they're trying to create a whole new economic system. And some of the technologies might actually someday replace the Cloud. It's really interesting stuff that they're doing.
Greg Beaumont (01:00:36): But kind of circling back to where I started, it's kind of the same thing with the database. If you just try to run all the logic directly against the source, you're running massive amounts of logic for massive numbers of users in parallel. And caching reduces some of that pressure and it also allows people to have kind of specialized use cases where you're not doing 20 joins every time you select a filter. You do it once, and then you filter from those results.
Rob Collie (01:01:03): The Vertapak Engine, the end memory column store, all my years at Microsoft, that was the only thing I was ever close to that felt like what you would expect from a software company in a movie. This was science fiction. This technology was developed. It was sci fi and it was real, and it's still sci-fi today. It's so amazing what it is capable of. It is mind blowing the performance aspects of what it can do, and how effortlessly it can perform them.
Rob Collie (01:01:36): And to leave that out of your implementation like this magic piece of software, it's impossible what it does. It's still impossible. I still, I don't even remember how it works anymore. To leave that out, you're really leaving a lot on the table. And so, let's talk about what would be some cases where DirectQuery is the right answer?
Greg Beaumont (01:01:54): Near real time, so when you need data quickly, and you need it to be in the hands of the users without anything in between DirectQuery is absolutely the best use case in that scenario. There's other solutions. I know you've dug deep with Denny Lee recently on Big Data, where when there's just massive amounts of information, you don't want to cache that information. The purpose of caching is not to go get everything. It's to reduce the complexity of the logic. So, if you have a gigantic database, and you need to go get details from it, absolutely DirectQuery is the best option.
Greg Beaumont (01:02:27): And just when you kind of hit the technical limit of the caching within a tool like Power BI, you have to go to DirectQuery. I mean, there's just a certain point where you get up in the hundreds of gigabytes for a cube and it's going to perform better on DirectQuery mode. Just because the technology kind of hits that limit, where the benefits you get kind of max out and start to degrade.
Rob Collie (01:02:49): I worked with Chris Finland, when he was in the field on a project where we ended up with, at that time, it was 2013 and maybe, it was 2014. Anyways, SSAS tabular and 3 billion records in the biggest fact table. And this thing was running on 32-gigabyte VM and it was all loaded in RAM. It was having no trouble at all. And this was despite it having an incredibly complicated fact record structure, such that every single fact record in the model, all 3 billion rows, every single one of them was an inception to date number. Not what happened to that month, or that day. As of that day, the number in that database was this was what has happened in this corner of our business rewinding 50 years. This is the grand total over time.
Rob Collie (01:03:37): And so, even to get the current activity in a particular timeframe, it was a time intelligence measure. The most basic measure in the entire cube like, "What happened that month? How much revenue came in that month?" It was time intelligence, right? You had to take current number and subtract the yesterday number to know what it was. It was like the lights should be flickering every time someone touches this thing. It worked great. I was just, it was stunning.
Greg Beaumont (01:04:05): So, the one thing I hear where people I work with are going to strongly disagree with me on this is a lot of people still think that caching and middle layers are a Band-Aid until the DirectQuery technology gets better. This is just my personal opinion based on what I've seen and what I've experienced. I see over time where I mean, just imagine this scenario, okay? So, you have a solution that requires row level security and you can have a little note of compute on your local computer that contains just the rows that you're allowed to see with kind of a distributed tabular model.
Greg Beaumont (01:04:37): That doesn't exist today, but it could potentially in the future versus taking all that data and putting it in one place inside of a data center somewhere and having everybody communicate. To me, it just seems like it would be at least something to consider, right? I'm not an expert in the area, but I don't think that caching and distribution of kind of the logic is going to go anywhere soon. I think it's here to stay for some time.
Rob Collie (01:05:00): And then you've got this technology whose primary purpose is storage and retrieval. And then you've got this technology that its primary purpose is analysis. And they're going to make different tradeoffs. They have to make different tradeoffs. In fact, one of the reasons why you consider not near real-time, right? Why is near real-time a good use of DirectQuery? Well, because you can't rebuild the Vertapak model multiple times a second. That's a tradeoff, right? It can't be updated at the individual record level like a regular database can be because it made tradeoffs.
Rob Collie (01:05:33): I think you could almost mathematically prove that the Vertapak engine is close to theoretically optimal, in terms of how fast it is at what it does. You just can't sideline that thing. And it's not like the storage engines are ever, ever, ever, ever going to support a mode of DirectQuery that's going to be that fast. So, yeah, I think that's the way to look at it, right? Is it like you want to use the magic engine, sometimes you just can't. And you should be disappointed at those times. And then happy that DirectQuery is an option, but you should be disappointed that you weren't able to use the magic thing that's going to make everything better.
Greg Beaumont (01:06:09): I'd add it's usually less expensive, too, but usually the cost of doing it that way is less expensive for the organization and the query performance is still usually better.
Rob Collie (01:06:19): Yeah. It's a funny story that when I was working on that solution with Chris, back in the day. This was a Christ's reaction. I'm pretty sure that somewhere in the account team, there was a bit of like, "Oh, really?" When we found out that we were able to, because this is back in a very different licensing model. The world back then was very, very different in terms of how Microsoft licensed their software and it was per CPU per machine, whatever, right? And the fact that this gigantic model, with the entire financial history of this Fortune 500 firm, been around forever, was stored in this one model, and was able to be run on a single 32-gigabyte VM was a bit of a bummer to the people who are trying to sell software, right?
Greg Beaumont (01:07:04): Yeah.
Rob Collie (01:07:05): It's like this absolute apex predator of a project. We get one VM of additional footprint, are you kidding me?
Greg Beaumont (01:07:14): Yep. Unreal.
Rob Collie (01:07:16): Hey, Microsoft's loss is your gain, customer.
Greg Beaumont (01:07:24): I do see still challenges with creating those cache layers. You look at a tool like Aggregations, where it's allowing you to have hidden summary tables sitting behind your fact tables or alongside your fact tables. It's, you really have to understand data modeling to set those up. And you have to understand how it works within the context of the tool and the context of what people are using, but if you go look at the roadmap, within Power BI, you'll see auto ags on the public roadmap, which is the Automatic Generation of Aggregations based upon query patterns. I'll be interested to see how that actually looks when it comes out. I don't have access to anything that's not publicly available. That's out there in the public.
Greg Beaumont (01:08:03): And then on the synapse side, materialized views is kind of the same thing. And you'll also see a roadmap item for the query accelerator with synapse where it's going to look at the queries. It's getting from Power BI and then spin up materialized views, which for all practical purposes, as I mentioned before, another version of a cache, that will then kind of self-tune the model to get better over time. And hopefully alleviate some of the need for people to actually learn how to do it manually. Again, it's moving from PaaS to SaaS as the other components have.
Rob Collie (01:08:34): And those sorts of improvements that are in the works. I mean, this is where some of your colleagues get the idea that we're just sort of sitting around waiting for the day of DirectQuery parity. There are developments being made to improve forever, right? We can always improve. We're going to get to Vertapak level.
Greg Beaumont (01:08:48): Yeah. I see those demos where they say that NLP will replace the data analyst. When you're younger, it's like, "Oh, no, I'm going to be out of a job." Now, it's like, "No, that's I'll be long dead before that ever happens."
Krissy Dyess (01:08:59): So, Greg, do you have any hobbies outside of work?
Greg Beaumont (01:09:03): Yeah, we actually kind of live off the beaten path a little bit. It's effectively kind of a small, almost a hobby farm. It used to be a horse ranch, so I spend a lot of time doing stuff in the yard. And this last weekend, residing my garage. I spend a lot of time doing family stuff and things like that. When I'm not working, I don't to be sitting in a desk. It's like I want to go build something with my hands or I want to go somewhere and do something and travel that kind of thing.
Krissy Dyess (01:09:32): Have you always been in Minnesota, too?
Greg Beaumont (01:09:34): No. So, I actually came from a military family. I think we moved seven times when I was a kid. Lived all over the country, but we ended up kind of landing here. And my wife's family is from here, so I've got roots that aren't going to be severed. We could be here for some time. Yep.
Krissy Dyess (01:09:50): Yeah. I kind of have the same thing here in Arizona. I mean, I was able to move around until again, I found my family here and it does make it hard to move when you had set your roots.
Greg Beaumont (01:10:00): Yeah, with things like teams, everything's virtual now.
Krissy Dyess (01:10:03): Yeah. It's not the same.
Greg Beaumont (01:10:06): Yeah, yeah, yep.
Rob Collie (01:10:07): What is it with Minnesota and BI, though? There's something to it, right? We have more consultants, full-time consultants working for us from the State of Minnesota than any other state.
Greg Beaumont (01:10:17): Yep. I know a lot of your employees are from Minnesota, and also a lot of people I work within Microsoft are from Minnesota in the data world. I don't know the full answer to that question. I do know that we have a lot of industries here that are very data-centric, right? So, you have a lot of device companies. You have large, probably one of the largest insurance companies that is based out of Minnesota.
Greg Beaumont (01:10:39): There's a lot of kind of medical innovation happening in Minnesota with the Mayo Clinic down in Rochester and University of Minnesota. And there's also a lot of schools that have very good math programs, and very good engineering programs, even all the way up to Fargo and North Dakota. They upgrade engineering schools up there. So I think there's just kind of a hub of education and technology and industry that kind of combines to kind of find those 5% that you talk about and give them that opportunity.
Rob Collie (01:11:11): Yeah. On a per capita basis, Minnesota has got serious game in the data space. It's tempting to think of it as, "Ah, it's just small sample size." But I don't think so. I think there is something in the water or equivalent. Maybe, it's the absolutely brutal winters. You've got to find something to... it's like, "Where do you go for football players, we go to the places where it's warm all year. Florida, Texas, California. What do you go for data people? Well, you need to go someplace where if you step outside, four-month window, you just die."
Krissy Dyess (01:11:41): It's longer than four months.
Greg Beaumont (01:11:44): It is, yeah. It's probably a five-month winter. The ideal situation would be able to stay here until New Year's, and then probably come back in April, right? I have friends and family, they're going to hate me for saying that because they snowmobile and ice fish and my neighbors across the street will put up an ice house, out on the lake. And so, some people actually love-
Rob Collie (01:12:06): They're building structures on a lake. Let that sink in.
Greg Beaumont (01:12:12): Yep. I've heard of stories where up on Lake Mille Lacs, you'll drive a mile or two out on the lake to get to your ice house. And I've heard stories of people who aren't from here going out to the ice house and saying, "Where's the lake?" And they're like, "Oh, the shore is two miles that way."
Rob Collie (01:12:26): Yeah. That dawning moment of, "Oh, my God. "
Greg Beaumont (01:12:30): Yep, yep.
Rob Collie (01:12:33): There were street signs.
Greg Beaumont (01:12:35): Well, yeah. They actually do have street signs on the lake in the winter. Some of these were posted.
Rob Collie (01:12:41): I grew up in Florida. And I thought when I went to school in Tennessee, I thought, "Oh, my God, is it cold here," right? And then eventually, I ended up living in Cleveland, I'm like, "Yeah, this is really cold." And then I took a couple of trips in the winter, in February, to Minneapolis. Like, "Oh, my God." I couldn't even keep the ice off of the highways underneath the overpasses. No amount of salt was going to do it. No, no, this is super frozen. Whatever that is, I don't know. Yeah.
Greg Beaumont (01:13:10): I had a co-worker once, who was born and raised in India, had never left Southern India, and came up here on assignment without ever having seen snow. And it was below zero when they got off the plane. So, I mean, you can imagine the shock, because it is something you have to acclimate to.
Rob Collie (01:13:29): I can't imagine. I would need, I would need [crosstalk 01:13:31]. And then I remember sitting there going, "Oh, that's right. There's an entire country north of here. What is wrong with those people?"
Krissy Dyess (01:13:38): It's just-
Rob Collie (01:13:41): It just seems like the absolute northern edge of human expansion, and then you realize, No, there's a whole industrialized nation up there.
Greg Beaumont (01:13:49): Yeah. People think of this as being the arctic tundra, but all of Canada is basically north of us.
Rob Collie (01:13:54): My friend, David Gaynor, who is going to be on an upcoming episode of the show, he grew up in Alberta, you know it's?
Greg Beaumont (01:14:00): Yeah.
Krissy Dyess (01:14:01): Every year we go up in Phoenix. We can go up north, just a couple hours in the Flagstaff in December, maybe January, get a little bit of snow. And the kids, they come up. Families, they bring their children to see snow for the first time and they all do it. They all stick their hands in it. And then that sensation, that burning sensation starts to kick in, and then two minutes later, they're all crying, going back into where they came from. And it's just like every year, you go up and you see these kids for the first time when they touch the snow, right? Like touch it and then immediately in tears.
Rob Collie (01:14:35): Right. Yeah, it's cool. Hi, Greg, so here as we're sort of closing up, what are some of the things that you see coming, whether they're new technologies or adoption trends that you think are most significant or perhaps you also find particularly personally exciting?
Greg Beaumont (01:14:51): Yeah, so if we look at some of the new capabilities we've seen in both Power BI and on the Azure side, there's a lot of focus on Machine Learning. AI And combining data from different places to get insights. Something that I think is kind of extremely valuable, but it's just not as prevalent in demos and presentations and things like that is the integration between something like Azure Machine Learning and Power BI, where it's still hard to create a good machine learning model. You probably want, especially in healthcare, you want real data scientists creating your machine learning models. But it used to be really hard to then put that into practice, right? You might have something that does a great job of predicting, but then how would an analyst use that data unless somebody else is just providing it to them.
Greg Beaumont (01:15:38): Now, you can literally go into Power Query your data flows, and select a machine learning model that you have access to, and then take the corresponding columns of data and map them to the inputs of that machine learning model. Hit go, it will do all the work for you. You don't have to configure any APIs or write any code. And then you're getting access to that predictive technology at your fingertips.
Greg Beaumont (01:16:02): There's also Auto ML if somebody wants to learn about machine learning, where you can start building simple machine learning models right in Power BI. What I found, though, is that by the time somebody really understands Auto ML, they're usually ready to graduate to the Azure side of the house. But I see that integration of, not only being able to get all of this data from all these different places and tie it together, but then be able to go beyond doing simple math and using machine learning algorithms is kind of the next big thing in both healthcare and beyond.
Rob Collie (01:16:36): That's a fascinating topic. Long time ago, when I was first working with PowerPivot, I had some friends who had left Microsoft and gone off and formed a machine learning startup, and some of them are back at Microsoft now. Really, really, really smart people. And it was natural for me to try and to collaborate with them and vice versa at that time. None of the PowerPivot models that I was building, it turned out none of them had anything interesting to be found with machine learning.
Rob Collie (01:17:04): And it was a hard lesson, which was by the time you're done aggregating, overwhelming majority of Power BI models and reports operating at an aggregate level, by the time you're done aggregating, you've kind of lost all of that grain level variation that is interesting to machine learning. So, I learned at that moment that a lot of these technologies are meant to operate at, you can think of it as being like operating at the fact record level, not on the aggregates.
Rob Collie (01:17:33): And so, whenever I hear about machine learning and Power BI coming together, my brain immediately goes back to those old days of, "Oh, no, these two are incompatible." And that's my first instinctive response. I have to think about it a little bit longer before I go, "Okay, there's actually ways they can interplay." And I haven't tried this thing that you were talking about, but it sounds amazing.
Rob Collie (01:17:54): At the Power Query level, you could be importing additional columns, you're mapping columns that I'm assuming that you would get back an additional column or multiple columns, with some sort of predictive score, right? Maybe like the percentage chance that this customer is going to be leaving. Attrition risk or whatever, or things of that nature. What's the easiest way to get started with that stuff?
Greg Beaumont (01:18:18): I think I'd add two things there. So, the easiest way to get started is right in Power BI Desktop. If you open Power Query, it's in the ribbon on the far right hand side, you might have to enable it. But you could start with cognitive services. And you could just say, "For each row of data, for this column, tell me what language that comment was written in?" And you can count how many responses are in Spanish versus English versus Portuguese or whatever it may be.
Greg Beaumont (01:18:41): Another example would be sentiment analysis, right? And this one is always funny in healthcare, because sentiment analysis is looking at words and then saying positive, neutral, negative from I think zero to one. But in healthcare, the words mean different things, so there was one that came out as being extremely positive. It was tenderness, right? Because in healthcare, it means you're sore and you hurt. But outside of healthcare, it's a positive emotional word, right? Yeah.
Rob Collie (01:19:08): There's also doctor speak in general, which is like it requires a completely different sentiment filter. I had a salivary gland tumor removed recently, which I'm fine now. But if you read the pathology report on what was going on with me, right? As a human being, non-trained professional.
Greg Beaumont (01:19:25): Scary, yep.
Rob Collie (01:19:25): And you read that, you'd be like, "Oh, man, Rob, you're going to die." So, I don't... yeah. I wouldn't want to the sort of the vanilla sentiment analysis looking at that.
Greg Beaumont (01:19:37): I wanted to add one more use case, too. So, you referred to doing predictions on aggregations. One use case where that might actually be applicable is let's just say somebody wants to do a simple forecast. Right? You can do this right in Auto ML. I'm actually working on a demo on for it right now. I don't have it ready to go where the analyst comes in and says, "I want to forecast at the level of the individual provider by day, by disease category, by department," something like that. And then you do the forecast and you find it's not very accurate.
Greg Beaumont (01:20:09): Well, you can maybe make an aggregation where you roll up the forecast to the level of by physician by week, rather than day, so on and so forth. And change the level of granularity. Rerun the Auto ML test to see how accurate it is. And then you could go back to your data science team and say, "Maybe we want to do the predictions at this level of granularity, because that's the accuracy level that I'm looking for." I agree with you that 99 times out of 100, you want the most granular data for those types of efforts. There are those scenarios we're kind of...
Rob Collie (01:20:41): Totally.
Greg Beaumont (01:20:41): ... coming up with summary tables to do the predictions to have that be more agile. I think it's going to create a lot of value.
Rob Collie (01:20:47): I'm mostly just reflecting my frustration from that era. We were failing to find anything useful. My friends at startup were telling me, "Yeah, Rob, you just don't understand how this stuff works yet or you can't aggregate like that." I was still very stubbornly insisting that "Okay, come on." There's still entities in the world, for example, like a store. Let's say you're a chain with 500 locations. You have all kinds of interesting attributes at each of those locations. "Is it a two-story store? Is it the deluxe store? It's blah, blah, blah, blah, blah. Does it have the pharmacy built in or not?" But all kinds of these aspects, right?
Rob Collie (01:21:18): And you would never predict future transactions on a transaction level? Doesn't make any sense, right? What would you forecast this store's revenue to be next month, right? So, completely valid machine learning problem. And so, I'm glad we did circle back to this because I never had the right kind of data to drive that sort of analysis, that sort of machine learning analysis. It just didn't happen to exist in the models I was using at that time. I wouldn't want people to come away from this going, "Oh, no, you can never. Machine learning and aggregate level are incompatible." That would be the wrong conclusion. It was just harder to get to that point than I had expected. I sort of naively expected it to just like, "Okay, here we go jump off the page." And it didn't.
Greg Beaumont (01:22:03): I agree with you. If they could ever find a way to combine multidimensional compute with predictive technologies that would be kind of the Holy Grail.
Rob Collie (01:22:12): Greg, I can't thank you enough. You brought so many really interesting perspectives. I'm really grateful for the thoughtful approach that you've taken and I think people are really going to appreciate this episode. So, many, many, many thanks. Thanks for being here.
Greg Beaumont (01:22:26): Yeah. And thanks for the opportunity. And I was listening to the show even before Krissy reached out. The service you're doing for the community here is absolutely fantastic. Thank you.
Rob Collie (01:22:35): Thank you very much. That's really gratifying to hear.
Announcer (01:22:37): Thanks for listening to the Raw Data by P3 Adaptive Podcast. Let the experts at P3 Adaptive help your business. Just go to p3adaptive.com. Have a data day!