Dr. Deming railed against performance appraisals, listing them 3rd in his Seven Deadly Diseases of Management and calling them "Destroyer of People." In this discussion, John Dues explains our cultural attachment to appraising workers and why it is a myth to assume that appraisals have any impact on performance at all.

TRANSCRIPT

0:00:02.3 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we continue our journey into the teachings of Dr. W. Edwards Deming. Today, I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. The topic for today is a continuation of our discussion about management myths that keep fooling us, and we are gonna be talking about performance appraisals. John, take it away.

 

0:00:32.1 John Dues: It's good to be back Andrew. I thought it'd be helpful first to connect back to what we've done, because it'll help listeners connect the dots between the various episodes that we've done together. I think this is the eighth episode, so episode one and two were all about the System of Profound Knowledge as a theory, and then episode three, we started working on understanding the concept of variation, special causes, common causes, that type of thing, and then four and five, we switched gears and talked about how to then apply the System of Profound Knowledge in our organizations. And so we talked about two powerful tools, process behavior charts, and then the PDSA cycles.

 

0:01:21.9 JD: Episode six, we started talking about A Nation at Risk and the Sandia report and how calls for education reform haven't always been built on a solid philosophical foundation. And then last time I introduced this idea of living in an age of mythology, and we talked about two myths. The myth about best practices and the myth of the hero educator. And so today, like you said, I thought we continue that discussion of the myths with a focus on performance appraisals, which is something that is a little bit hard to understand, I think it was hard for me to understand initially, but it's something that I thought was important because it's something when I listen to Dr. Deming's recorded seminars, it's something that he railed upon often.

 

0:02:14.9 JD: And I think tying all of the myths to a couple of key ideas is helpful. So I think that first idea is that when we see outcomes in a system, they're more than the skills and efforts of the individuals that work within the system. So those results come from more than just how the individuals within that system are working. The outcomes, that second idea is that the outcomes are mostly attributable to the system itself, and workers are only one part of that system. I think that's really important. That underlies all these myths and certainly underlies this idea of the myth of the performance appraisal.

 

0:03:00.8 JD: And I think that when we're talking about these myths, so we've covered the theory, we've talked about some ways to apply that theory, that System of Profound Knowledge in actual organizations. When we're thinking about the myths, what I'm thinking about is, dos and don'ts. And so the myths are the don'ts. There are specific prescriptions following the Deming philosophy that leaders should learn to stay away from and why to do so. And then of course, the do's would be a set of guiding principles to follow, and I thought, right now, we're focused on the myths and as we get through this episode and maybe one more on the myths, then we would then focus on the "what do you do?" That's where the guiding principles would come in, and so Deming outlined all of this for us. The theory, the application, the Do's and the Don'ts, and so that's where I thought we would start today.

 

0:03:55.6 AS: That's great. And we were talking before we turned on the recorder about how performance appraisals are such a fascinating area, and I know for a lot of people, there's nothing else. That that is the key of how you manage people. Like, you're talking about the core. Without performance appraisals, people are gonna be lazy. Without performance appraisals, people are gonna get distracted. Without performance appraisals people aren't gonna work hard because they're not gonna get compensated. Without performance appraisals, we can't get this organization to work and everybody to work together and this is the ultimate incentive that we need to motivate humans. So boy, you're taking on quite a tough topic here, John. Tell us a little bit more.

 

0:04:45.6 JD: We'll see how we do. And one thing to clarify when I say performance appraisal, in my world, in schools, this is typically called the Teacher Evaluation. So it has different names, but, an evaluation, an appraisal, some type of rating and ranking of employees basically. So I think one thing that, and you kinda just brought this up, is "if I don't do a performance appraisal, how am I gonna give feedback to team members?" And I think that's a good place to start is that, of course, I think that leaders and managers should, as a part of their job, provide direction and give feedback to team members.

 

0:05:33.9 JD: But I think it's a far cry to make the leap that giving direction and feedback is synonymous with administering performance appraisals. And I actually think that performance appraisals can actually work against giving good feedback. But like as a starting point, what makes up the typical performance appraisal? Thinking about four parts, just so we are all starting from the same place.

 

0:06:06.7 JD: First there's standards that are set. "Here's the standards that are gonna be outlined in this performance appraisal." Then there's a time limit set to meet those standards, then the manager makes observations and judgments, and then finally, the evaluation is given to the individuals by the person sort of in the organizational hierarchy. I think a key thing that I've learned in studying the Deming philosophy when it comes to performance appraisals is that they fail to consider the role of the system on individual performance. So that's one problem.

 

0:06:50.2 JD: They also fail to appreciate the variation in performance attributable to common causes. So that's why I was connecting our earlier episodes on theory and the applications to performance appraisal, 'cause you have to understand that to understand why Dr. Deming was railing against performance appraisals in the way that he did, and those are two of the key reasons.

 

0:07:20.2 AS: Right. So a person being evaluated or being talked to with a performance appraisal, a common thing is, they could say, "wait a minute, you're saying I didn't do this, but I couldn't do this because the system has this whatever." Or you get a boss that's focused on common cause variation going, well, "you did this, and then that, and this guy did that, and this is and then all..." What they're really doing is chasing their tail on all of these common cause variation, which is not going to improve the system and it's just rewarding and punishing what is just a natural outcome of the system.

 

0:08:01.1 JD: Yeah, that's a big part of it. So if I'm a teacher and part of my evaluation is something like outcomes of students and how well I deliver the curriculum, the effectiveness of the curriculum, those types of things. Well, the vast majority of teachers didn't select the curriculum. So that's a good example of something that's a part of the system that a teacher has no control over typically, but that it could play a role in an evaluation, and there's all kinds of examples like that. That second idea in terms of the appreciation of where the person is falling performance-wise within that common cause system, what that means is that, sure, people could be performing at different levels, and there could be slight variations in that, but it's very possible that those ups and downs, just like any other ups and downs that we study are just common cause. And so it's not one person different from another within the rating system?

 

0:09:06.6 JD: Are they far enough outside that they show up as a special cause. That their performance shows up as a special cause. Because if it does in the case of a teacher or maybe a student that is outside of the system in terms of performance, then there may be special help or special support that's needed. But I think many, many times that's not the case, and that the ups and downs don't represent anything meaningful. And I think one of the things that helps bring this into view for people is to say, "well, how did you experience teacher evaluations or performance appraisal or whatever you call it in your system, how did you experience as...

 

0:09:50.6 JD: How did you experience that practice as a receiver of those things?" Because that puts you in a different mindset. For me, performance appraisals, when I've been evaluated, have largely been positive in terms of the overall rating, but they've also, a lot of times not made a lot of sense on any number of fronts. And so I think of, as a teacher in Atlanta, and Atlanta Public Schools had a teacher evaluation system, the principal would come in for one hour across the school year and observe me and write it up and formally evaluate me, sit down with me and go over that evaluation. Well, if you think about that, one hour of observation, the typical 180-day school year, seven hours a day, that's about 0.08% of the school year that the principal observed. So that's a big problem.

 

0:10:52.1 JD: So we're saying that that represents the entire...my entire time teaching across the school year. That one hour observation. So that's a major problem. Another issue is, what is it that I'm being evaluated on? One that stands out for me, and granted we were in a different time, 23 years ago, 22 years ago when I first was evaluated, but they're still a technology category in the evaluation. And so part of the evaluation was to "use technology effectively in a lesson." And so one of my first questions would be, well, "do you have to use technology in a lesson for it to be effective?"

 

0:11:40.4 JD: I think that would be questionable at best. But what if I use technology in some lessons and not in others, and the one you happened to observe, I didn't use it, right? You didn't see the ones where I did. I was working in a large urban school system, I had seven computers in my room and five of them didn't work on a regular basis. So that's another obstacle, right? And so I get this rating, I take it, I don't really say anything about the computers not working, or what about my other lessons where I did use computers, I just listen to this, but...

 

0:12:13.3 AS: Otherwise you're gonna be labeled as argumentative.

 

0:12:15.4 JD: I'm gonna be labelled as argumentative and the rating was fine as it was, although I lost some points for those things. It's probably not worth...it's not.... You kind of pick your battles. But the point is, what does that leave me with in terms of the taste in my mouth about my school, about this evaluation system, granted it's one part of the system and maybe I didn't care about it too much as long as the rating without a satisfactory level, but the point is, it didn't seem fair, it didn't seem to make sense, it didn't seem to line up with what you would need to look at in terms of what you need to make an effective lesson.

 

0:12:55.0 JD: And how many people are experiencing evaluations in those same ways, whether it's ridiculous and being evaluated for something that doesn't work in your room, like the computer's not working, or a smaller like, do you need computers to be a part of the lesson in the first place. And so there's all kinds of things like that that I think are part of a typical evaluation system.

 

0:13:22.0 AS: So to summarize what you're saying is one way to think about performance and appraisal is to think about your personal feeling when you're receiving your evaluation, and I would argue that most people don't feel great, it's not something they're really looking forward to.

 

0:13:36.2 AS: And the second way you can look at this is look at the person who's delivering it. If you're having to deliver performance appraisals, is that like your favorite day of the year that you're working with that person? Yeah, so that's a good way to look at it so that you kind of understand that there's just something that doesn't feel right here, but continue on.

 

0:13:56.6 JD: Yeah, it takes a tremendous amount of time and effort. No one actually likes the process, generally speaking, and I think the thing that I wanted people would hold on to was that they don't get magically better when you're on the other side and being the evaluator. So my feeling wouldn't change whether I'm on the receiving or the giving side of the evaluation system now, I think for some leaders, unfortunately, I think that changes as long as they're on the other side, it's fine, but I think that's why I think putting yourself back in the shoes of the person receiving the evaluation is a good thing to hold on to. "I'm not special, there's not something about my personal characteristics that make evaluation unnecessary for me, but everybody else needs those things."

 

0:14:43.6 JD: So I think holding on to that as you move, especially if you move into a leadership role is a really important mental model. I think another key thing is after the evaluation, all of these people for the most part, are gonna still be working together, and so another key question that I learned from a Deming student named Peter Scholtes in a book called, The Leader's Handbook, a great book.

 

0:15:15.8 JD: He said, "what are the factors that differentiate highly effective versus lower rated people?" He outlined these five factors, so there's A, would be native ability and your early education, the second factor would be, B, your individual effort, how much work am I putting in as a teacher, as an employee. C, would be training, an orientation that I get as a part of the onboarding process or the ongoing professional development that I get as a part of the job. D would be variability of the processes and systems that are going on within my job, and E would be the system evaluation itself to some of those things that we just talked about, is it fair? Is it well constructed? Is it representative of my total work, that little sample that's seen by the manager.

 

0:16:17.0 JD: And if you look closely at those five items, really only one of them, that being, I think I call it D, that individual effort is under the control of the individual person working in the system. The other four factors really don't have anything do with individual performance, but what the performance appraisal system attempts to do is solve that equation, A plus B plus C plus D plus E equals my rating, let's call it 100.

 

0:16:54.5 JD: But if you can't solve that equation, if you don't know already what the variables A, C, D and E account for in terms of its contributions to the rating, the only thing you know as an individual effort, right?

 

0:17:11.2 AS: And you don't really know that either.

 

0:17:17.0 JD: Yeah, fair enough, fair enough.

 

0:17:20.1 AS: So it's a shifting sand that you're working on, which is what probably one of the counter-arguments to performance appraisals is that there's just... It's so subjective and difficult, particularly, okay, if you're a narrow-minded person and you've never thought about the fact that there is variable B, C, D and E as an example, then...but once you start to think about those things, you realize that not only is it difficult to quantify and all that what a person's doing, and how do you factor in the fact that that person just went through the loss of a parent over the period of time that you appraise them. How does that impact performance?

 

0:18:02.5 JD: Any number of things. Any number of things. Yeah, I can think of a lot of examples when you start to unpack those various factors, like when you're talking to the manager, "oh, well, we didn't quite onboard them like we typically do now, no one acclimated them to our curriculum system" or whatever it is.

 

0:18:27.3 AS: But they're still responsible for delivery.

 

0:18:29.8 JD: Yeah, they still move forward with that response. And again, it's not that there's not gonna be variation in performance amongst employees, it's just, are we getting what we think we are from this rating and ranking system. I think what we're doing basically is disregarding the contributions of the system on the performance of individuals that are working within that system.

 

0:18:57.0 AS: And I guess if you talk about that to people, they're gonna be like, "Now you're unleashing something that's just unmanageable." Okay, yeah, fine. We're gonna start talking about the system and the impact and that everybody's just gonna blame the system! John, don't you know people are just gonna blame the system, then if we start talking about why it's not your responsibility.

 

0:19:23.2 JD: Yeah, I think, yeah, we'll kinda get into what's the prescription in terms of...what would of the prescription be from the System of Profound Knowledge in terms of what to do instead, but one thing to do, if you did have some type of evaluation system, you could just remove the numbers and have a narrative feedback on characteristics or competencies or capacities that are important for your particular organization. I think that that would be one way to handle it.

 

0:20:00.4 JD: Another great tool that I learned from David Langford is a tool called the capacity matrix, where you outline what are the capacities that are important for a given role that you want to see develop. You define a series of dimensions of growing from more basic to more proficient in a particular capacity, and then you ask the person to track their own learning in those areas, and as they self-evaluate, they have to provide evidence, be it - maybe they give a presentation or incorporated a technique into their lessons on a regular basis, or maybe they presented at a staff meeting, something like that, but they have to link the capacity development to some evidence that it's been put into place.

 

0:20:51.0 JD: That's another way to handle... The point is to develop the person and build capacity, that's a much more powerful way to do it, and I think the goal of starting to use the Deming philosophy is transformation, and I think what Deming was talking about when he talked about transformation is this process from going from - starting to understand these assumptions and these myths and then working to move away from them. So one of the things...one of the lenses I have just in studying the Deming philosophy is to ask questions because so many of the practices like the performance appraisal, prior to studying Deming, I never even stopped and said, "well, what is the theory behind the performance appraisal? Where did it come from? Why do people think it isn't an effective practice? Are there practices that would be more effective?"

 

0:21:52.3 JD: So just as a starting place, you can start to ask questions about some of these things that you probably never even stopped to think about. I think that was true for me, whether we're talking about these myths or any other number of things that are common in organizations, work settings, and we have this...

 

0:22:14.0 AS: And for performance appraisals: what is the theory behind them?

 

0:22:20.0 JD: That's a good question. That is a good question. Where did they come from? Well, I don't know for sure, but I know that...a lot of corporate practices can be traced back to things like the military and early railroads, which were some of the first organizations to have a larger staff that had to be organized in a way, and I know that in terms of the rail companies that you know, when there was a crash, there did need to be somebody blamed. And so you had to nail down who in the hierarchy... Where did things break down? It had to be an individual to blame when two trains ran into each other or the train ran off the tracks. And I think what Deming is saying it is what was actually the system that led to that crash, that's what we needed to study in a lot of cases, and almost in all cases, whether it's the train running off the tracks or the Challenger space shuttle disaster, almost all of them were of system problems and it wasn't one single individual that you could pin those problems on.

 

0:23:37.2 AS: And you could argue that performance appraisals are not really there in that case, like what you're talking about let's say is a train crash, it's not really there to some extent to blame... To improve the system, it's there to blame someone and then, "okay, we got our scapegoat, now let's move on."

 

0:23:54.4 JD: That's right, yep. Yeah, so with performance appraisal, it's not quite as dramatic as the train crash, but what's happening is that it leads to this rating and ranking of teachers, we do the same things with students, students have their own form of performance appraisals, even schools within state accountability systems have their own rating and ranking systems. So they reward at the top and punishment at the bottom, that's the typical present practice. And I think the better practice, what we're trying to move towards when you're managing through the Deming philosophy is: abolish the ranking in favor of managing the whole organization as a system, and what you wanna do is study and understand how every part of that system, every component whether it's grade levels or departments, whatever, how do they contribute to the optimization of the system.

 

0:24:57.8 JD: And so that's... What's the aim of your system, how do you optimize that? And I think a big part of this performance appraisal thing is that that practice is running in the other direction from optimization. You are incentivizing individuals to look out for themselves versus contributing to the aim of the organization.

 

0:25:26.7 AS: One of the things that people say is like, "What do I replace it with?" Well, in a lot of cases in education, you may not even have that choice, but in private business, you do, and I always say that...I always say "imagine that you're lost deep in the woods, and after hours of walking in one direction, you realized that you're walking in the wrong direction. However, you're unsure of the right direction, but you've received enough information to know that you're walking in the wrong direction. What would you do?"

 

0:26:05.7 JD: Perhaps stop going in the wrong direction as a starting point.

 

0:26:12.3 AS: And the point is, is that you don't have to know the right direction if you've identified the wrong direction. And so that's one of the challenges that we often get with performance appraisal is, "what are we supposed to do if we don't do that?"

 

0:26:28.8 JD: Yep, yep. And I think... And that's - when you start to understand the System of Profound Knowledge, you start thinking about ways that it can offer you guidance on a practice like the performance appraisal. And so what I tried to think through is, in terms of performance appraisal, what do each of the components of the System of Profound Knowledge contribute in terms of learning about the way...in terms of your analogy: the way to move...start to move in the right direction.

 

0:27:08.5 JD: And so there's the four components of the System of Profound Knowledge, we have Appreciation for a System, Knowledge of Variation, Theory of Knowledge and Psychology. And so each of those components has contributions to make in terms of rethinking the performance appraisal. So I was gonna break those down as a way to round down out our talk today. Some of this is a recap, but when I think about Appreciation for a System, we've talked about this, but that system is responsible for most of the observed variation between the performance of the individuals. It's most...Deming said up to 94%, depending on the situation, even up to 97% of the results that come out of the system can be traced back to the system itself, and only 3% to 6% were attributable to the individuals.

 

0:28:08.0 JD: So the system is the overwhelming contributor to that ranking within that, doesn't help anyone, nor the system improve. Giving somebody a rank, sorting people into good and better and best does not point the way towards improvement for the organization. So that's the Appreciation for a System contribution.

 

0:28:34.1 AS: Yep. We could say changing seats on the Titanic.

 

0:28:37.2 JD: Yeah, exactly. Exactly. Then we have understanding variation or Knowledge about Variation. So we've talked about this a little bit, but ranking people, especially when we're ranking within a common cause system, is misleading. 'Cause remember, even if there's ups and downs in terms of the data, the question isn't: is it different? It's: is it meaningfully different? And in a common cause system, even though there are some ups and downs, there's no difference. It's not of a magnitude that you can say, "yep, that's significantly different from one point to the next." And another thing to consider is there will always be variation. [chuckle] There's always gonna be variation between students, between teachers, between schools, between school systems, between states, whatever that thing is, there is variation in a natural state of affairs. So we have to come to grips with that.

 

0:29:40.9 JD: In terms of Theory of Knowledge, when we rate and rank people, it's a snapshot. Kind of like what I alluded to my observation in Atlanta being less than 1% of the total time that I was with my students. So that ranking doesn't take into account and any performance appraisal system I've ever been aware of that temporal spread. So in other words, I'm really more interested in what's the performance over an extended time period. And so when people would ask Deming, okay, you're saying, the performance appraisal is something we should abolish. Well, how much data would you need on an individual worker before you could rate them? And what he would say is 15 or 16 years. [chuckle]

 

0:30:32.1 JD: And basically what he's saying is, I think, is that, that's the amount of performance data you would need to plot. If you're doing it once a year, once you have 15 or 16 years, you can kind of get a sense of how that data is performing over time. That last component, maybe the most important is Psychology. I think one big problem is that those performance appraisals at their worst are debilitating to people, at their best, they're perceived to be arbitrary, like what I talked about. Certainly wasn't debilitating to get my rating, my rank and my rating in Atlanta, but it did...I did see the rating and the points I lost as arbitrary and meaningless, to be candid.

 

0:31:23.1 JD: And then another big part of that psychological component, especially when it becomes to rating and ranking students, is that this thing called the Pygmalion effect begins and it can really start to destroy cooperation. Whether that's students or teachers, but you this is basically this idea that once, let's say a teacher has a set of expectations about students, they start to take on those characteristics, whether it's in a positive or negative direction.

 

0:31:57.2 JD: And they've done some pretty fascinating studies on this Pygmalion effect in classrooms. There's one where it's like in the late sixties, basically a teacher was told that a set of students had performed really well on a standardized test. In reality, there was no difference between this group of students and the rest of the students. But because the teachers thought that, over time what the researchers found is that they started treating the students differently and it actually resulted in those students scoring higher on the standardized tests at the end of the year just based on those teacher expectations. And so, talk about a powerful, powerful set of effects within a rating and ranking system. And I think that's something we really need to consider in any type of institution, but especially a school system. So, yeah.

 

0:32:55.0 AS: I was just reminded of a quote that Dr. Deming said, which is, and I'm gonna, I'm gonna read it for for a minute here. I'm gonna read it in my Dr. Deming voice. "So evaluation of performance, merit review, or annual review. The idea of merit rating is alluring the sound of the words, captivate the imagination, pay for what you get, get what you pay for, motivate people to do their best for their own good. The effect is exactly the opposite of what the words promise." [chuckle]

 

0:33:32.1 JD: Yeah. I think that's what I found in practice, before I discovered the stuff I tried to improve our teacher evaluation system. And in reality that's just an effort that's not worth my time.

 

0:33:47.9 AS: He also said, "Annihilates teamwork, and it's purely a lottery."

 

0:33:53.3 JD: Yep.

 

0:33:57.7 AS: I wrote a blog post on this many years ago entitled, Why We Stopped Performance Appraisals at Coffee Works, my company. And this was... I published in 2016, and I just would... Maybe I'll just read a little excerpt here. What I said is, "The system was an annual review during which bosses in our companies met employees and scored them as A, B or C. I read Jack Welch's book and I thought, yeah, kick out the C players, this has made sense in the days when I was buying into that. We would use this to allocate bonus mainly to pay, A's and B's. A's a lot, B's a little bit, and C's nothing. And for years we've been seeing the weaknesses of this system, but didn't have the guts to abandon it, because we didn't have something to replace it.

 

0:34:50.5 AS: And so before you ask what we did to replace it, let's consider what we didn't like about it. Number one, it was unfair. Number two, it was subjective. Number three, it fostered favoritism. We saw that certain employees were continually getting positive ratings from their bosses. Number four, it failed to recognize what quality godfather Dr. W. Edwards Deming taught was that the majority of the output of any one employee is attributable to the system. And number five, it was time consuming and costly. Number six, it did not enhance employee performance. Number seven, it increased fear and caused suspicion. Number eight, it caused employees and departments to compete against each other rather than compete against our competitors or just take care of the customer.

 

0:35:34.2 AS: Number nine, it was completely inward focusing, encouraging employees to shift their focus from the customer during the time of performance appraisals. It's like you can't take care of the customer 'cause you gotta get all these performance appraisals done." Is there anything that you would add to that list of what you see in the education environment?

 

0:35:51.3 JD: No, I mean that's spot on. I mean, I think the key summary or takeaway is for any of these practices, be it performance appraisal or otherwise, is that thing optimizing the system? Is it making your organization better at achieving its aim? And you just named a whole lineup of things that said, "no, this is why in this particular practice performance appraisals, this is why Dr. Deming railed against them." So I think that, yeah, I think that was a great synopsis of many of the things that Dr. Deming talked about in his seminars when he railed on performance appraisals.

 

0:36:33.4 AS: So, as we wrap up, the purpose of today's discussion is to open up people's minds to understand what are performance appraisals, what are the myths behind it? What are the weaknesses of it? We also kind of said, even if you don't have a substitute, you could argue that you don't have to keep doing something that's damaging if you know that it's damaging. But we also know in an education environment, you may not have the power to make that decision. Like we had in our coffee business, it was like, "this isn't working, we're stopping. No more resources to this." We have that flexibility. So, we haven't spent any time talking about ways to do things and the positive aspect. But let's just wrap up this whole conversation. How would you kind of wrap up the message that you want the listeners and the viewers to get from this specific discussion?

 

0:37:29.4 JD: Yeah, I think because a lot of this, a lot of Deming's ideas were targeted at leaders and leaders at all different levels. And I think, what constantly happened to me was someone would say "what's your leadership style?" And I would typically cobble together some type of jargon in response to that question. But what Dr. Deming offers is a management philosophy: the groundwork, the framework, the foundation, the philosophical foundation, the myths to stay away from, a set of guiding principles that actually, when you dig deep and you study these things, they offer a way forward, a lens through which to make better decisions for your organization. And that's really what I take from this.

 

0:38:22.3 JD: There are many better ways to run our organizations. Performance appraisals are just one component of this, but hopefully what people are taking is that, hey, at the very least, the next time I have a leadership team meeting, I'm gonna bring this up and say, Hey, why do we do this? Is it leading to the type of results - like what you're alluding to with your business - that we think it is. And if it's not, what else could we do? How can we replace this, make it better? How can we at least begin to ask these questions instead of just carrying on? Because it's the way that we've always done things. So, yep.

 

0:39:00.6 AS: John, on behalf of everyone at the Deming Institute, I want to thank you again for this discussion. For listeners, remember, go to deming.org to continue your journey. This is your host, Andrew Stotz. I'm gonna leave you with one of my favorite quotes from Dr. Deming. "People are entitled to joy in work."