In Their Own Words

What's the difference between education and training? Why is the distinction important? How does the Deming lens offer a new perspective on teacher effectiveness? In this episode, John Dues and host Andrew Stotz talk about why it's important to go beyond skills training and encourage education for personal growth.  TRANSCRIPT 0:00:00.0 Andrew Stotz: Here we go. My name is Andrew Stotz, and I'll be your host as we continue our journey into the teachings of Dr. W Edwards Deming. Today I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. This is episode 19 and we're continuing our discussion about the shift from management myths to principles for the transformation of school systems. John, take it away.   0:00:31.2 John Dues: Andrew, good to be back. Yeah, principle 13 today, Institute a Vigorous Program of Education. I'll just start by reading the Principle, "Institute a vigorous program of education and encourage self-improvement for everyone. The school system needs not just good people, but people that are improving with education. Advances in teaching and learning processes will have their roots in knowledge." It's interesting, when I was reading about sort of this particular principle, Dr. Deming took this actually pretty far when he was asked where would you draw the line? And he basically said, I would allow any educational pursuits that people are interested in. So that was his sort of take on this particular principle. But I think it's maybe the first thing is to differentiate between training and education. When he was talking about those things, we talked about instituting training on the job back when we talked about principle six, and he basically said the training is for a skill and a skill is something that's finite because it ends when performance has reached a stable state for a person when thinking about that particular skill.   0:01:51.3 JD: The differentiator with Principle 13 is that it's focused on education and it's meant for growth. And in the Deming philosophy, this is sort of a never ending process of education. So skills, so training is focused on skills, whereas education is focused on knowledge and theory. And this is really an important distinction in my mind, and you need both, training and education are complimentary components I think of an effective school system or really an effective organization in general. So I think, I mean, obviously training is important. It's something that's necessary, especially when you come into a new job. We have lots of new teachers that come to us 'cause we're a relatively young organization. And it's pretty typical for these new teachers to come even if they majored in education many times, they don't have sort of the basic classroom management skills, the basic lesson planning skills, the basic lesson delivery skills that they need to be successful in the classroom.   0:03:00.9 JD: So we have a training program, and in the absence of that training program the teachers would probably flounder or it would take a lot longer time to get their legs under them. So training is important, but we have to sort of shorten that runway. So we have to be good at training 'cause we're like a relatively young organization and we have students that come to us on average that are below grade level. And so they can't wait a long time for these sort of teachers to get up to speed. And I think we've talked about the fact that we have this sort of three week training program before the school year starts for new teachers for that reason. And so training is obviously important, very important. But I think what I've sort of come to appreciate is this idea of... And Deming stressed this, that leaders, systems leaders understand this idea of a stable system.   0:04:00.7 JD: One of the things that he said was that "The performance of anyone that can learn a skill will come to a stable state upon which further lessons will not bring improvement of performance." And this for me, reading Deming at this point in my career was really an interesting revelation because for many years I had heard sort of policymakers, education reform types sort of lament the fact that teachers improvement largely levels off in about year five of their career. Now, there has been some more recent longitudinal teacher research in terms of effectiveness over time. And basically people have found that that's not quite true. And that teaching experience is positively correlated with student achievement gains sort of across the teacher's career. But it's definitely true that the gains and effectiveness are steepest in those initial years.   0:04:55.2 JD: And so when you put those two ideas together that there's sort of this leveling off in about year five with Deming's sort of concept of stable systems, it really sort of dawned on me that it was this perfect explanation for this phenomenon. When a teacher is in their first five years there's a lot of foundational skills like the things I was talking about, like lesson planning, lesson delivery, classroom management, those basic things. There's sort of this period of rapid improvement or growth, and then it sort of levels off after you get the basics of how to be a teacher. And then after that happens, you have this... The potential for improvement sort of lies within the organization, within the system itself and not in the individual. So this really lined up with this thing I had heard for a long time, even though I think sort of it was misinterpreted.   0:05:52.0 JD: And I think a lot of those people that were talking about teacher skills leveling off after five years, they didn't have this lens of a stable system. They didn't have that part of it. And so they were saying, well, teachers aren't improving. Well, it really wasn't the teachers not improving. It was the fact that most of the capacity, like we've talked about here for improvement lies within the system itself and not the individuals. And I would also make the argument that this is not just educators, that this is other sectors as well, healthcare or whatever that thing is.   0:06:27.0 AS: Yeah. I mean, a good way of imagining that is a person who knows nothing that has the prerequisites, the education or whatever's necessary to get the job. And they know nothing about teaching and about the school system or anything that you can just imagine that so much of the initial phase is just understanding how the system, how they operate within that system to do certain tasks, which can be a process of trying to understand all of that. But then it's like they become, it's like entering the stream and then they become the stream floating down the river where everybody's kind of doing the same thing. And then you realize, okay, by this time now their, their, the amount that they can improve has been hit for some specific tasks and things like that. And then all of a sudden their output is a function of the system.   0:07:23.0 JD: Yeah. Yeah. And I think where this can really go off the rails is when people don't understand the stable state of systems. I think that, and I think a lot of the educators from reformers were sort of talking about it as if teachers were kind of replaceable because they didn't improve after those initial five years, especially 10 years ago that was sort of the common way people talked about this. And you could then sort of the next step is to draw the conclusion that experienced educators aren't that important since that improvement sort of levels off pretty early in their career. But I think that is the completely wrong conclusion to draw. I think experienced teachers are incredibly important because of the stability they provide a school. They can provide mentorship to inexperienced teachers, they have longstanding relationships with families as multiple students come through the system.   0:08:25.0 JD: That stability is really important for all those reasons, which are hopefully fairly obvious to anybody that's worked in a school. But I think even maybe more importantly is this idea that once teachers have that baseline level of knowledge and skills, they can run a classroom, they can deliver a well-planned lesson. The reason that it then becomes important for improvement to have those folks is because once those basic things are in place, now we can actually start to work on the system where the real potential for improvement lies. And I think that was a point that was missed or glossed over in a lot of those conversations about education reform and this idea of the teacher skills leveling off after year five.   0:09:23.8 AS: Mm-hmm. One of the the things about education that I have a story that's... I guess one of the conclusions is that the next level of improvement of the system oftentimes comes from outside the system. And that's where education takes the mind into another space.   0:09:40.9 JD: Yeah.   0:09:49.2 AS: From that other space, they're getting knowledge and theories of what's going on out there. And I had an example, John, that was... When I was the head of research at Citibank, and I had been head of research before taking care of a team of analysts, and analysts are always late in their reports, they're writing long reports about whether to buy or sell a company. They're trying to gather as much information, talk to the company, things get delayed. They set their deadlines and then they... The job of a head of research is juggling those delays so that the sales team and the clients need an idea day. And it's always the case that you're juggling around and okay, we don't have something this day, let's make something up with what we've got. Okay, this guy couldn't produce on that day, but he's gonna come in on Monday. So I felt pretty good about my skills at managing that process. And then I got a job at the number one foreign, the number one broker, let's say, or investment bank at that time in Asia called CLSA. And when I talked to them, I asked them how do you handle the flow and how bad is it here [chuckle] with the analysts being late? And they said, the analysts are never late.   0:11:13.3 AS: And I was like, that's impossible. My whole career it's been about handling the analysts being late. And they said, no, analysts are never late here. And I was like, how are you doing that? And they're like, well, we have a three week plan ahead. Everybody knows it. You know your day. There is no excuse, there's no shifting, there's nothing, it has to be delivered on that day. So it's up to you to kind of bring your project to a head so that you're ready to present on that day. And if you have some kind of major setback or problem, talk to another person and switch the day with them and sort it out. And every single day we had great stuff coming out. And I would've never, I mean, I was operating at a certain level thinking I was really knocking it out of the park, 'cause I was accommodating. I was careful, I was thoughtful. I understood the pressures that people were feeling. I was doing my best, but I didn't have a knowledge that it could be a very different way of doing it. And that's where I think about going outside of your own system to observe and learn and see. And then all of a sudden you're like, oh, [laughter] Okay. And that's where I feel like what you're talking about, about the education aspect is really the most amazing part.   0:12:33.2 JD: Mm-hmm. Yeah. That actually... I hadn't planned to talk about this, but I've been reading recently about the... Called the... Well, there's a book called Toyota Kata and Kata is from martial arts. It's the various movements that you have to do sort of repeated deliberate practice so you can sort of, they become ingrained in your muscle memory. Well, the same idea is in place in Toyota. They call... Well, they don't, but the author called it... They don't call... They don't have a name for it, but he sort of observed it and gave it the Improvement Kata name Mike Rother. Yeah, there it is. Yep. There it is. That one. And one of the things that was interesting, and it kind of reminded me of this as you were talking, is that part of the improvement kata is there's a sort of a target that's aligned with the organization's vision that guides anything that the folks in the organization are working towards.   0:13:27.0 JD: And so there's always a target condition. There's an understanding of sort of where each individual is and the departments are. And they're always setting a new target on the way to that sort of vision target and running these experiments all the time. And they constantly set those targets so that they are ambitious but within reach. And then they're coached on the way repeatedly. And in that way they're sort of always moving forward the organization. And so I think of when you've changed investment banks and you're at this new bank and they're saying, Hey, this thing is possible, it's possible to do this. Here's the way we do that. Here's how we work towards that. And so you can imagine a place like Toyota being so successful, because if everybody has got this mindset, this scientific thinking where they're constantly moving towards a target and there's a method for doing so, [chuckle] that is an incredible education right there if you're an employee working in an environment like that. So that just made me think of the Toyota Kata.   0:14:41.4 AS: Yeah. And it's a great example of how reading books is part of education because you're getting exposed to new ideas and exploring and thinking about things. And that's where, well, think about the repetition in let's say a martial arts as an example. And when Dr. Deming talks about opening up education to everything for everybody, there's something to learn in almost everything out there. Like if it is about... What is it about those repetitions and why is that important and could that benefit our business? And he talked about painting and other things, you know? Like education very widely can bring you new ideas that can come back to improve your system.   0:15:27.3 JD: Yeah. And I think you have to invest in that sort of broader education, 'cause it's sort of an investment in the future, you know? Especially right now, things are changing fast. And you could have the best training program in the world, but if you are not also sort of looking out for what's next beyond that, to adapt to whatever's changing in your environment... A good example is this, we have a much better understanding of cognitive science than we did 20 years ago. And so if we didn't adapt... If we didn't sort of learn that and then adapt that and sort of include that learning in our training system that we're gonna start falling behind pretty quickly. And I think this can get... This may be part of the most important responsibility of a leader on the learning front.   0:16:28.5 JD: Because what I also see is that education leaders are often getting enticed by many, many fads that sort of come along. And so how to sort of actually latch onto something that represents a potential advantage, that's a real important skillset to have. And I don't think... That's a key... I think a key function of systems leaders is sort of to know what to let go of or what not to latch onto at all and what to sort of sink resources into because if you're gonna go do these educational pursuits, you're obviously gonna have to sink time and money resources into these things. And so being able to differentiate between what is good and what is bad is a real key skill.   0:17:22.6 AS: And one of the things about Toyota is it's like the ultimate Asian family business. And although it's now a big public company, the largest automaker in the world, and the family's ownings in the company is relatively low, it still has the influence of the family. And I was thinking about another huge company that I know of in Thailand here that shifted its focus away from, let's say, Deming in this case, to when a new CEO came in, he said, well, there's a different way and this is my way. And one of the things that's interesting about what Toyota's done, you know, Toyota gets a lot of blame for being slow to progress and stubborn and all of that, but man, they have built a machine and a... You just can't change the direction of that quickly, you really nurture what has been developed and how do you not just throw away. I was presenting to my students last night in my finance class here at Sasin School of Management in Thailand and I was showing them the DuPont Analysis in the world of finances where you break down the return on equity of a company. And I explained why they call it the DuPont Analysis, and that's because the DuPont company bought shares in General Motors in 20- or 1912 or something like that and they instituted this method of financial controls on General Motors. And I said to my students in passing, General Motors has been going bankrupt since 1912.   [laughter]   0:19:00.9 AS: And it's like every... It's not a cumulative level of learning. And that's where I feel like Toyota, what Toyota has achieved is a cumulative learning process.   0:19:16.9 JD: Mm-hmm. Yeah. You know, and it's a part of their DNA. I think certainly there have been challenges as they've grown across Europe and the United States and the world really. And a lot of the challenges that I understand is because people... That improvement Kata is sort of combined with a coaching Kata, like an approach to coaching and managers at different levels coach folks that are sort of a level down from them. And everybody in the organization, especially early, had sort of this mentor-mentee relationship. And so part of the challenge with growth was the fact that there are only so many of these folks that are grounded in this scientific thinking in the coaching part of this. And so that was a challenge as they grew, you know, in California and Kentucky and other places across the world.   0:20:17.9 JD: They had to build this coaching capacity across all of these new production facilities and other types of facilities across the world. So... But I think that what I really like about this principle...I, you know, if push came to shove, I started this by talking about Deming would basically allow almost anything when it came to allowable educational pursuits. And I think I would be much closer to that than I would be to limit those things. I think that is a really... That's a good sort of approach to take as a leader. I think here where I am at United Schools Network, one of the things that I was able to do was go take an improvement advisor course which required significant resources and time and money at the Institute for Healthcare Improvement.   0:21:19.6 JD: And so someone could look at that very easily and say, well, why are you an educator going to a healthcare organization? And I think it's one of those things where people maybe don't realize that the Deming philosophy and some of the continual improvement stuff, it's sector agnostic. And so when you can learn the philosophy, the methods, the techniques, you can bring them back to your own organization. So I think had I not gone down this path to study Deming, I wouldn't have made it to IHI and then bring this stuff back to my organization. I think it's benefited our organization in lots of ways, even though that might not have been immediately apparent to folks, you know, initially.   0:22:09.8 AS: So how would we wrap this up for the listeners to make sure that they truly understand the idea of vigorous education, self-improvement, this type of stuff?   0:22:14.0 JD: Yeah. I mean, for me the main point is that systems leaders should really encourage education among the whole workforce with a pretty wide latitude for allowable pursuits. I think especially for educators, when we seek those types of opportunities, we're also modeling this idea of continual learning to students as well. They see that just because I have a degree or a master's degree or even folks here that have a PhD, we have I think an organization that's pretty hungry for learning. And that's a model for students. Oh, this doesn't end when you graduate high school. This doesn't end when you graduate college. It doesn't even end when you graduate from graduate school. People all across the organization have books piled up on their desks and we're sending people to various learning programs and stuff like that.   0:23:09.4 JD: And I think that's a good model for students. And I think within that another big thing is to think about do you have an understanding of the stable state of systems and understanding that training programs are only gonna take you so far? Individuals are gonna come to a sort of a stable state once they've sort of maxed out on any particular skill. And that's why this idea of education is so important. Skills are important, training is important, but this other side of the coin, you have to pay attention to education. What's on the horizon? How are you gonna push the boundaries within your system? And I actually think to your point about outsiders or having an outside perspective, that's sort of, I think the benefit of education, because I think without that sort of push from an outsider, the push from the education, breakthrough improvements aren't possible in our school systems. They're not gonna come from training programs. They're gonna come from this continuous learning, this idea of continually pushing the targets, having sort of an improvement mindset. Having a coaching mindset that's always pushing towards those things. And I think this requires not just skills, but it requires new knowledge and new theory continually. And I think that has to come from this vigorous program of education.   0:24:39.7 AS: And the beauty of capitalism is that if you don't go out and get the education, your competitors will, and you don't want your source of learning to be facing constant defeat from your competitors.   [laughter]   0:24:56.2 JD: Yeah, you can't sit around and wait, that's for sure. That's for sure.   0:25:00.0 AS: Exactly. Or someone's gonna take it. And that's the beauty of the capitalist system, the adversarial aspect between companies definitely gets people riled up when they see that all of a sudden someone's doing much better with some new technique or idea. Well, I think that was a great discussion to help us understand the difference between training and education and why it's so important. John, on behalf of everyone at the Deming Institute, I want to thank you again for this discussion. For listeners, remember to go to deming.org to continue your journey. You can find John's book "Win-Win: W. Edwards Deming, the System of Profound Knowledge and the Science of Improving Schools" on amazon.com. This is your host, Andrew Stotz. And I'll leave you with one of my favorite quotes from Dr. Deming. "People are entitled to joy in work."  

How do grading systems, teacher ratings, school rankings, and other programs like those create barriers to learning? Should we eliminate them entirely, or do they have their place? John Dues and host Andrew Stotz talk about how to preserve joy in learning. TRANSCRIPT 0:00:02.4 Andrew Stotz: My name is Andrew Stotz. I'll be your host as we continue our journey into the teachings of Dr. W Edwards Deming. Today, I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. This is episode 18, and we're continuing our discussion about the shift from management myths to principles for the transformation of school systems. John, take it away.   0:00:31.3 John Dues: Good to be back, Andrew. In this episode, we're doing the 12th principle. So we're on 12 of 14, remove barriers to joy in work and learning. So that's a certainly a concept that we've talked about, but I'll start by just reading the principle. "Principle 12, remove barriers that rob educators and students of their right to joy in work and learning. This means working to abolish the system of grading student performance, the annual rating of staff and accountability rating systems for schools and school systems. The responsibility of all educational leaders must change from sheer numbers to quality." There's two really great quotes I like from Deming. One on joy in learning where he says, "Our schools must preserve and nurture the yearning for learning that everyone is born with. Joy in learning comes not so much from what is learned, but from learning."   0:01:24.4 JD: And then for joy in work, he says, "Joy in the job comes not so much from the result, the product, but from contribution to optimization of the system in which everybody wins." So he is saying basically the same thing in those two quotes, but he is talking about the contributions to the process is where the joy comes from, not necessarily the outcome. And so much of the time, we're focused on the outcome, be it the work product in a work setting or the test scores perhaps in a, in a school setting. But he's really talking about what is that process that you're contributing to? And, and you know, how do you feel because of that, those contributions you're making? I think whether you're talking about joy in work or joy in learning, sort of unifying theme in principle 12 as it's, this concern with the pride of workmanship, whether that's the workmanship of making a product or in the learning that you're doing or something you're doing as a result of that learning, like a report or a poem that you've written or whatever.   0:02:31.6 JD: And so I think as a result, it's barriers that get in the way of joy in work and learning. And you know, maybe one of the most important obstacles to improvement of the quality of our education systems in the United States. And you know, just like, sort of, it says in the outline of the principle, there's really sort of three levels that these barriers exist at. You got the students and the grading of students. And then you have oftentimes some type of rating system, evaluation system for teachers, for principals, perhaps sometimes those rating systems use test scores or other similar metrics. And then that third level is, you have the actual schools or school districts themselves that are being rated within these state accountability systems. So you sort of have, you know, these three levels. And then there's this common problem at all three levels, regardless of which one. And that's basically this thing that we've talked about repeatedly, where you under-appreciate the contribution of the system to the performance of the people, whether you're talking about students, teachers or, or you know, school systems. So I thought that's where we could focus today.   0:03:49.8 AS: Yeah, you know it strikes right at the heart of everything that we believe, as particularly as Americans, but certainly spreading that around the world, that it's all about measuring, ranking, tracking. You know, when a parent puts a kid in school, what do they want to know? What was their grades? When a student's in trouble, it's 'cause of grades. And what a student wants to know, like everybody wants to know and rely on grades. So it's just so, it's so difficult. You know, I was talking with someone else talking about why Dr. Deming's philosophy hasn't been adopted as as widely as you'd hope. And I think it's part of, it's just because it's just sacred, the sacred heart of everything that we believe. And if you can measure it, you can track it, you can feed that back and give it to people and show them where they are and you deserve where you are based upon your efforts, and you've gotta move yourself from there. That is so ingrained. And I'm just curious, like what's the hope from your side that this can be seen. I think it can be seen if you stop and look, but it's so hard to implement.   0:05:18.7 JD: Yeah. Well, I mean, I think one thing that can be confusing is obviously Deming was a statistician. So he is, has no problem with using data to improve the quality of our schools or even an individual lesson that a teacher delivers, gathering some data on how students are doing and tracking that over time. There's no problem with that. There's no problem with that I don't think at the school level either. I think the problem comes in when you create a reward and sanction system around that data. And I think that's, Deming actually think he indicated that, that system of reward or sanction on the other side is one of the main constraints from being able to develop this win-win culture. Whatever level of the system that you're talking about, that student level, the educator level, or the school and system level. They're, all those grading systems are really reward and sanction systems. And I think when you take the data and use it in that way, that's when I think Deming is talking about the real problems, the manipulation, you know, the competition for top spots that leads to all kinds of strange behaviors, those types of things. That's really where he's focusing most of his attention.   0:06:49.1 AS: And if you had no constraints from governments or other outsiders and you were setting up a new school right now with zero constraints, the only thing was absolutely optimizing the learning of young people. How would you handle this - grading? How would you handle all of this? Would you do it in a different way? Would you just do it and de-emphasize it and say, oh, well, it's not so critical? It's just information feedback, or would you teach them how to use that data like Deming may, or like how he uses data? Or would you say, no, that's just that there's no redeeming benefit, if we're not required to do it, then we wouldn't do it?   0:07:43.4 JD: Yeah, I mean, I think it'd be some combination of the things that you mentioned. I mean, on just from a practical sense, Demings certainly understood that we live within the world that we live within. And so if that hypothetical school that you're talking about is a public school and I was in Ohio, I would obviously give the state test and take whatever data I could use and use that in a positive way that I can. So I'm gonna do the thing that I'm caught on to do as a educator in a public school system. So I wouldn't opt out or anything like that. I think in terms of how I set up internal systems, I think, ..GAP. Yeah, I thought a lot about this but I haven't maybe put to paper exactly how I would do it.   0:08:36.2 JD: I think...I certainly would use assessments. I certainly would track how students are doing on standards. I would involve students in doing so they could track that over time. In terms of grading, I don't know exactly what I would do. I would definitely de-emphasize that to the extent that I possibly could so that the emphasis is on the learning and not on the grades. That would be a key sort of guiding principle. There's certain things that I think are outlandish that schools do, where they do pep rallies or pep rallies or something like that to, as the buildup to state testing comes, I find those things ridiculous. So I would stay away from doing anything like that. Kind of how we've treated state tests in the past, even prior to discovering Deming was matter of fact.   0:09:47.2 JD: Like, this is something we're preparing for, we're gonna do our best. We're gonna try our hardest, we're gonna learn from the experience, we're gonna work hard on it, and then we're gonna move on. You know, that's the type of mentality we had. In terms of like the mindset of the school I led, there was a poster in the hallway that said, you get it wrong and then you get it right. So that was the mindset is, we learn from our mistakes. We talked about creating a culture of air in our classrooms so that students felt, you know, safe, I guess is the word I would use to, or willing to call out when they didn't understand something, or they did make a mistake, and then we work together to rectify that. So that's a little bit a long-winded answer. I don't have it all worked out. I have some ideas, but I think overall using data is fine. I think it's, when you get into the rating and the ranking, that's where the problems and the rewards and the sanctions, I think that's where the problems generally come from.   0:10:50.8 AS: Yeah. I mean, I'm kind of unconstrained in my Valuation Masterclass Bootcamp, because I'm not under any, there's no supervision of what I'm doing by anybody. It's just me trying to make a better experience for the students. And the idea of grading never really came into my process. It's interesting, John, that one guy who was a student of mine, he graduated and then I hired him to work with me to take care of the other students. One of the first things he did was come up with a matrix and a grading system, [laughter] just because that's what he knew. And now he uses that system and he has little points that he gives. That system doesn't have any connection to whether you're gonna pass the class or not, or there's no ranking or anything related to it. It's just that, okay, you only got six points out of 10, which means you haven't really done the assignment. He's clearly defined what are the things that you need to have done? And then he goes through and says, did you do them? So it's definitely a, I think it's a good feedback mechanism.   0:12:08.4 JD: Yeah.   0:12:10.2 AS: But, you know, whether it's valuable. I think what I'm trying to do is create the experience that young people are learning how to value a company through the process of learning and discovery and, and discussion and, and, and going online and going through my material, asking me questions, and then demonstrating, showing different things. And then they're slowly putting those pieces together. And I can also see that it takes time. You know, it can't all happen in one week. We have six weeks and where they're at at the end of the six weeks is so much further along than where they were at the beginning. But I guess my point would be if I was completely unconstrained, which I am, it's just that it wouldn't be pro or con grades and ranking, it wouldn't even really exist because it's not a core part of learning. Core part of learning is providing the environment, the excitement, keeping people on track, helping them see, okay, here's what you gotta do now, see if you can do it. You know?   0:13:18.8 JD: Yeah. And I think what I was gonna do today is bring this alive. You know, we've talked about grading of students and even the performance appraisals in some of the past episodes. So what I thought we would do here is, sort of, focus on why grading schools could also be you know, a barrier to joy in work, how this might play out. So I think generally most people are now familiar with, because in the public school system, we've had various types of rating systems. You know, each state has their own. And I think it's, what's important here is to look at examine if the ratings help the public differentiate between schools that are doing a good job of educating kids and those that are not, because that's the point.   0:14:09.7 JD: Right? At least one of the points. And on its face, it sounds simple, you know, up until this year, Ohio has like an A through F grading system for schools, and there are sub components and you get an overall grade A to F. Right? And so it sounds simple. Schools with more A's are better schools, except for it's not that simple when you go beyond like a surface level analysis. So I thought it'd be helpful to just zoom in on two schools located here in Columbus. One's called Jones Middle School. It's in the Upper Arlington School District, which is close by. And then Columbus Collegiate Academy, I'll call it CCA, it's also a one of our middle schools. Both the schools serve grades six through eight. They're less than 10 miles apart here in Columbus.   0:15:03.6 JD: So they're geographically proximate. And this analysis comes from an article I wrote in 2020. So it's from a few years ago. So the results are a few years old, but you know, I think they're fairly representative of how the schools have performed over the last decade or school or so. So let's start with just the grades that the schools received. So the schools get an overall grade. Jones Middle School has an A, CCA has a B, so you know, fairly close there, but Jones outpaces. And then there's the achievement grade. And that's basically looking at all the kids' test scores, how they do overall. Jones gets a B, CCA gets a D, right? So Jones has quite a bit better performance. And then there's a progress category. So how much progress did the kids make during that particular school year?   0:15:57.9 JD: How much growth did they make? Now this is interesting now, CCA gets an A and Jones got a B. So just to recap, the overall grade for Jones was A, achievement B, progress B, for CCA overall, B achievement D, progress A. So basically a higher percentage of students at Jones begin year on grade level 'cause they have that higher achievement grade, but they don't grow as much as the students at CCA once they're there. This difference between achievement and progress grades becomes even more interesting as you start to factor in not only the school characteristics, but also the neighborhood characteristics. So let's talk about inside the school just to start with. So in terms of student population, Jones and CCA are pretty similar in terms of students with disabilities. So those kids with special education needs tend to, as a general rule score lower on standardized tests.   0:17:02.9 JD: So those populations are roughly equal, but 100% of the kids at CCA are economically disadvantaged as defined by the state. At Jones, just 2.4% of the kids are economically disadvantaged. When you look at other report card measures such as attendance, chronic absenteeism, Jones has much better rates. So 97 plus percent attendance rate, just 2% of their kids are chronically absent. At CCA 93% attendance rate, 21% of the kids are chronically absent. But when you start to look at these, some of these metrics framed in terms of the poverty rates in the community surrounding CCA, these numbers start to take on a different meeting. And I think what they're, especially things like chronic absenteeism, that's all the rage right now, attendance, I think what you start to need to understand is these are indicators of inequity, housing instability, neighborhood violence, lack of access to healthcare.   0:18:15.5 JD: I think they're more an indicator of those types of things than they are of school performance. So as you start to think about things in those ways, what you realize is that the students at CCA are just as capable as the students at Jones, but they face sometimes overwhelming obstacles related to poverty. It's also interesting to take a look outside the schoolhouse. So the median family income in the census tract where Jones is located is $184,000. So the median family income in that neighborhood, so it's a pretty affluent area. In the neighborhood surrounding CCA in that census tract, the median family income is just over $20,000. So we're talking about an order, orders of magnitude higher family income in upper Arlington than in the neighborhood that CCA sits in. And then there's all types of factors. Some grounded in historical reasons that relate to this, but they're also compounded by funding disparities. So the per pupil revenue at CCA for this year is $10,600. In Upper Arlington, it's nearly $17,000...   0:19:35.0 JD: This Jones Middle School has almost no students living in poverty, yet gets $6,000 more in additional revenue per student than the students that attend CCA. So think of the implications of that.   0:19:52.3 AS: When you say they get more revenue, you mean the state or the government's providing them more money per student?   0:20:00.1 JD: Yes, all in. From all sources. So when you look at what the federal government provides, the state government, and then local funding sources. When you look at all those sources combined, this more affluent middle school gets $6,000 more students, dollars per student.   0:20:14.5 AS: Obviously, it's not based upon need. Is that based upon the grade or some other?   0:20:19.0 JD: Well, it's because the funding is heavily influenced by local property taxes. And because of the affluence of...   0:20:26.7 AS: They have the resources.   0:20:28.2 JD: They have the resources. And in Ohio, charter schools don't have access to local money. So that explains most of the gap. Most of the gap. But back to my point, when you think about CCA, having kids with more challenges, less money per student, less resources to pay for a facility, to pay teachers a competitive salary, extracurricular activities, all those types of things that we want to equalize are highly inequitable between those two schools. So then you start to ask yourself, well, what are the report card grades measuring exactly? Are those grades on those state report cards a fair representation of what's happening inside the school? Or can a significant portion of those grades be attributed to this larger context in which the school sits? And I think that's where you sort of put on this systems thinking lens and realize that, sure, what teachers and the principal is doing inside the schools, they are certainly making contributions to those state report cards. But you cannot ignore what is going on outside those schools and those neighborhoods when you're thinking about these grades.   0:21:54.1 JD: And so if you're sort of thinking about... Like a formula that would sort of lead to the school's results and you just... Let's just call it A+B+C+D+E=71, where 71 is the score that the school gets. Let's just call it that. And let's call the school's contribution letter F. A, B, C, D, A+B+D+C... Or A+B+C+D+E+F=71. The school's contribution is F. Well, that equation cannot be solved unless you know the values of A through E or at least some of those values. But what we try to do with this state report card system is that we assign this value to F, the contribution of the school, with no knowledge of the effects of these other variables.   0:22:57.3 AS: So it's, in other words, the contribution of the school is 100%. You mean you're responsible for your results? Is that what it means?   0:23:04.4 JD: Well, right. So if you're going to give me... If you're going to give CCA Main Street a D in achievement, that means the only thing that contributed to that grade was the school. But there's all those things that we talked about. Some, sort of, when you look at the variables A, B, C, D, and E, those other variables, you could look at things like, what are the state standards? Or what's the test design? What's the school funding? Household income? Home environment?   0:23:34.1 AS: Education level, maybe, of families.   0:23:37.2 JD: Education level of parents. Teaching methods. All of these things are variables that are outside of the school's control, or most of the ones that I just mentioned. But we don't, we don't see that when we look at these report cards. Right? You know, and just like I said at the beginning, despite all of that, I'm in favor of administering these state tests that are standards aligned, reported annually to the public. I actually think understanding how students are performing in a standardized way is actually... Could be useful information, I think. But when you extend those systems to the grading, the rating, and the ranking, I think that's a misuse of the information. Because too much of the rating and ranking comes from the system, as opposed to being directly assignable to the school or to the individual educators within that system. I mean, I think, if you analyzed report cards in this way, I think my opinion is that a reasonable person would conclude that the comparison between Jones Middle School and CCA is not a fair one.   0:24:56.4 JD: Because those students that arrive in those schools are not on equal ground upon enrollment. And I think our time would be better spent figuring out how to make things more equitable between those two groups of students than constantly recalibrating these rating systems that at best communicate confounding information, conflicting information.   0:25:27.5 AS: KPI experts around the world listening and viewing this are saying, "oh, come on, John. What? All you got to do now is you just got to break it down. And now we're going to do the KPI by adjusting for these factors. And now we're going to compare schools based upon that." Of course, what we've learned and I've learned over my life is that every time you think you're going to break it down and make it more comparable, it gets harder and harder to do that. And it just becomes less reliable and less useful in a lot of cases. Not completely. I mean, making some simple adjustments just for, let's say, yeah, I suspect that just one factor could probably represent A, B, C, D and E probably pretty well. Maybe that's the income of the area or the amount of funding that they got. One or two of those factors probably is enough to say, okay, we gotta compare schools that have these factors similar as a first step. But every time that I've ever gone down to go deeper into measuring, it just gets...it, it, the answer isn't there.   0:26:40.6 JD: Yeah, and I'm going to tie this back to joy in work. So if you think about that current school rating system, what we fall prey to is that fundamental attribution error that we've talked about before, where we have this tendency to underestimate the impact of situational factors on other people's behavior and overestimate the impact of individual factors, you know, when it works in our favor. But what happens is if I'm a teacher at CCA, there's a likelihood that I'm going to get blamed for the results. Let's say the achievement results. If I'm at Jones Middle School, there's a likelihood for praise. Because the school is doing pretty well. But in both cases, in both cases we're vastly underestimating the impact of the situational factors or the system on those results. So over time, I think this can have an impact on joy in work of educators working in these challenging schools.   0:27:49.3 JD: Even, even in the case where in many of these schools, like the one I just talked about with CCA, that there's solid evidence that staff and these schools are often getting better outcomes if you go beyond the surface level analysis. Because if you remember, they did quite a bit better on the progress, meaning they grew the kids more in a single year, even though they may have not hit the proficiency standard at the rate that Jones did, they grew the kids more. So you could make a solid argument that CCA is actually better, even though they got lower grades on the report card, right? I've often said, what would happen if you just switched the two staffs?   0:28:33.6 AS: Yeah. Problem solved.   0:28:36.3 JD: What would happen to the report card? You know. That's interesting. Obviously, it's never going to happen, but it's an interesting hypothetical experiment. My guess is a lot of teachers will find out that would go from Jones Middle School to CCA in a much more challenging environment would find out pretty quickly, that a lot of their methods don't work as well, right? So I think that these are the types of things that we're talking about. Imagine if you're at CCA year after year after year after year, get these lower grades. Right? And even if there's some evidence, like the progress score, who's digging in to find this? That score is often harder to find than the overall grade. That score is often not in the headline and what makes it into the newspapers. You know? And so you start to ask or you start to doubt yourself. You start to think about, am I really good at my job? Those types of things come in. And if you don't have someone there doing this deeper analysis, putting this in context, that's not easy to do.   0:29:52.1 AS: Yeah, when the pressure's on.   0:29:54.4 JD: When the pressure's on. And even if you're good at doing that type of analysis, sometimes people won't believe you because, well, that's not what I'm hearing. That's not what I'm... That's not what my family's saying. Those types of things. And then, and then, if you have those good teachers that at a certain point say, I'm just going to go somewhere where it's easier. Then those kids at CCA wind up in a worse place. And that's, I'm using CCA as an example, but I think this plays out at you know challenging schools all across the country all the time.   0:30:36.2 AS: Yeah, when you were talking about the morale of the CCA teachers, I was just thinking some brilliant bureaucrat would probably come up with the idea of why don't we post this grade right on the front of this school?   [laughter]   0:30:51.4 JD: Well, yeah, they're easy to find. That's for sure. These are all public, public reports. Sure. And in fact, actually, back during, I think, during the Obama administration, during Race to the Top, when it became really in vogue to rate teachers based on their progress scores, the individual teachers. The school report cards are easy to find, like a report card on any public school in Ohio or any public district. But in some cities, what started happening is they were, newspapers were getting a hold of the list of the progress rankings for individual teachers and posting those. I remember some of those were in the newspaper. And I think we've talked about this here as well, that what researchers have shown over time with these progress scores, these value added scores, is that some of the score is attributed to the teacher from before. Teachers that take on more challenging groups of students tend to have scores that are... Progress scores that are lower, all types of things And you want good teachers in those rooms. And what you're doing is disincentivizing that to happen when you have these types of rating and, rating and rating systems. So it's a tough thing.   0:32:15.2 AS: It's such an interesting topic. And I think, it got me thinking that we should start a new series on the Deming Institute podcast, which is, bad use of data. Like examples of, you know, here we have a misuse of data or just the simple thing of not making adjustments for situational factors and the misattribution. You could argue if you just improve that, maybe there's a little bit more meaning to this. But then, of course, there's also all the unintended consequences. And I just would imagine, I'm thinking about a book I have called the... By Terry Mueller, I think, or Jerry Mueller, which is the Tyranny of Metrics.   0:33:05.9 JD: Yeah, we got a lot of copies of that in our, right in this room where I'm sitting.   [laughter]   0:33:10.8 AS: Yeah. And I think that that would be kind of fun to bring out from the audience examples of what you're seeing.   0:33:17.4 JD: Yeah. Well, and one thing I didn't even mention that is also a key contributor here is, so let's say these two middle schools get this state report card. And another contextual factor is that most of the kids that go to Jones Middle School went to, I don't know the name of it, Upper Arlington Elementary School. And a very stable neighborhood. And of course, there's a few families here and there that will move in and out. But for the vast majority, I guarantee a vast majority of the kids that took these tests in sixth, seventh and eighth grade at Jones have been in Upper Arlington since kindergarten or preschool.   0:34:00.4 AS: Yeah.   0:34:01.2 JD: CCA Main Street, because of the nature of charter schools in Ohio, is a standalone 6-8 middle school. So that means 0% of the kids went to our elementary school during these years. And now whatever happened K-5 in a school, those kids school career, that certainly plays a big role in how they're going to show up when they enroll at CCA. So the only rule in terms of counting for CCA's test scores is that the kid had to be enrolled by October, let's say the first week of October. And they take the test in March. So six months later, let's say.   0:34:49.2 AS: Yeah.   0:34:49.6 JD: So let's say probably 50% of the kids at CCA, were brand new to that building, to that district, that school year, whereas the vast majority of Jones middle school students had been in that district for seven or more years. Because kindergarten is a year, and then when you're a sixth grader. So the time that they've been there, that's not taken into account either. And that may be the most important.   0:35:19.0 AS: Yeah. That's fascinating. So how would you summarize the one thing you want the listener, the viewer to take away from this.   0:35:30.5 JD: Yeah, I mean, I think it can be easy to start to think that data is bad. That is not the problem. You need data to help inform your decision making.   The problem comes when you then take the data and attach the ratings and the rankings to it, that's when the problem comes in. So you need to detach those two things.   We need to keep it public, keep it transparent, keep it known by all stakeholders, be it parents, the public, policymakers, students themselves. But it's the rating and ranking, that's the problem. That's the key takeaway.   0:36:11.9 AS: Great. Well, on behalf of everyone at the Deming Institute, I want to thank you again for this discussion. And for listeners, remember to go to Deming.org to continue your journey. You can find John's book, Win Win. W. Edwards Deming, the System of Profound Knowledge and the science of improving schools on Amazon.com. This is your host, Andrew Stotz. And I'm going to leave you with my favorite quote from Dr. Deming, which is totally pertinent to what we just discussed. And that is: people are entitled to joy in work.

Bill Bellows, an expert in leadership in Deming organizations, discusses leadership in Deming organizations, including a story about senior 'leaders' making a huge error in judgment at a conference of auditors. They also explore the role of language in maintaining unity, the importance of teamwork, management as resource allocation, and the concept of appreciating the system and the people in an organization.

The podcast explores the benefits of cooperation over competition in various contexts such as sports teams, classrooms, and orchestras. It discusses the importance of interdependence for high performance and the role of a manager in fostering cooperation. The concept of dimming and tracking progress is introduced to improve performance, and the power of cooperation and teamwork is highlighted through personal experiences and quotes by Dr. Deming.

The podcast explores the importance of informal conversations between managers and team members, focusing on aims, hopes, and fears. It also discusses the role of data in decision-making and the need for a safe environment. The value of camaraderie and meaningful workplace conversations is highlighted, along with the importance of relevance in education. The podcast emphasizes the significance of unheard conversations for fostering understanding and connection among team members.

Quotas, arbitrary targets, work standards with numerical goals - these don't seem to apply to schools. But, as John Dues and host Andrew Stotz discuss, quotas show up a lot in classrooms, causing harm and preventing improvement. TRANSCRIPT 0:00:02.4 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we continue our journey into the teachings of Dr. W. Edwards Deming. Today I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. This is episode 17, and we are continuing our discussion about the shift from management myths to principles for the transformation of school systems. John, take it away.   0:00:34.3 John Dues: It's good to be back, Andrew. Yeah we've been working our way through these 14 Principles for Systems Transformation. Last week or last episode we did eliminate slogans and exhortations. And so we're on to principle 11, which is Eliminate Arbitrary Numerical Targets. So I'll start with the overview. So principle 11, eliminate arbitrary numerical targets in the form of work standards that prescribe quotas for teachers and numerical roles for people in management, substitute leadership in order to achieve continual improvement of quality and productivity. And the first thing I wanted to start with was just this really powerful Deming quote on quotas. In Out of the Crisis, he said, "A quota is a fortress against improvement of quality and productivity totally incompatible with never-ending improvement." I just love that quote because it's just such a forceful pushback in the other direction. It's a fortress against improvement. There's really no gray area there in that quote.   0:01:43.0 AS: It's not a fort, it's not a barrier, it is a fortress.   0:01:48.2 JD: Fortress. It kind of brings together a mental image in your mind when you hear fortress, a fortress against quality, a fortress against improvement. So why did he say that? One of the things that's interesting is, especially thinking about work standards that prescribe some type of quota for teachers, it's like, well, when you think of a quota, you typically think of a worker and some type of production facility. And that, of course, is largely what Deming was talking about with his Point 11 'cause he was doing a lot of work in manufacturing and that type of setting. However it does, like all of this stuff, it translates into education. And, you know, so that's why I decided to keep Principle 11 'cause it does show up in different ways in the classroom setting. And I think examples are really good because when I initially read this quote and I was thinking, well, how do quotas show up in a classroom setting?   0:02:40.5 JD: And I thought of one that really stood out from when I was a principal here at one of our middle schools here in Columbus, at United Schools Network. We, we had this quota of sorts for homework. So in the middle school where I was, teachers had to assign homework nightly in their classes to students. And they had to grade two to three of those assignments a week and then return the graded assignments to students within 24 hours. And as I stopped and read this particular principle, and I thought about how it applied to my time as a principal, I really learned that that was the wrong approach. And as you start to think about that and reflect on it, you start to think about why Deming said quotas are a fortress against improvement. And there's this...   0:03:31.5 AS: And before you go... Before you go on, I just wanna highlight how normal that sounds.   0:03:38.4 JD: Very normal. Yeah. Very normal.   0:03:39.8 AS: And anybody here, like the first thing I'm gonna do or the first thing I do when I take over as principal is I'm gonna require that there's a minimum amount of this, and it has to be da, da, da, da, da. And it just seems like it is the responsible thing to do as a manager.   0:03:56.4 JD: Yeah. And there was a noble premise behind the quota, and that was that students needed frequent feedback on their work in order to learn. That was the premise, right? So it was this work standard, it was well-intentioned, but like a lot of these things that Deming talks about and quotas are no different, is the actual effect was that teachers spent less time giving feedback and more time grading this high volume of work. So this is what happens when you have a quota, is the focus became meeting the quota, grading the two to three assignments per week, rather than giving that quality feedback to students. So in this case, this sort of numerical target for graded assignments, then superseded the quality of the feedback. And this is, this is what happens, I think, generally speaking, with quotas and practice.   0:04:56.3 JD: And so when you step back and you think about that particular quota, you say, well, what was the teacher's job? Was it grading two to three homework assignments per week or was it giving students quality feedback? And it really couldn't be both. It couldn't be both those things. So then I started thinking about, well, what would've been a better approach? You know, had I had the Deming lens when I was a principal, I think the thing that I would've done is start with, well, let's come up with a sort of a well articulated aim for why we give homework. And included in that process, or included was, developing a process so that students received timely and high quality feedback 'cause that's really what this was about. So I think that sort of brings to mind substituting leadership, that second part of the principle.   0:05:58.4 JD: And in this case, I think, you know what I should have done is replace those work standards or that quota with some type of a better understanding of the job of the teacher. So, you know, I think in doing that, then I also... Leads to higher quality work-life for teachers. I would say, in thinking about this homework example, time spent grading homework was probably the number one complaint that I got from teachers. And this better approach to the two, three, assignments per week quota would've been to work with teachers to design a better system. Like how could we design a system that would give them the time to deliver high quality feedback to students on a timely basis? That was really the aim. And that's really where I should have concentrated my time as a principal. But again, I didn't have that Deming lens 10 years ago when I was serving as a, as a principal in our network. And reading the Deming stuff, it was very quickly like, oh, aha. Like, here's how I should have been thinking about this. Rather than being so hyper-focused on: you gotta grade those two to three assignments every single week.   0:07:29.9 AS: I like the word substitute leadership, you know, and Dr. Deming said that a lot. And the best way that I've kind of tried to explain it, and it just happened recently, where a client of mine was talking about having what they would consider to be underperforming staff. And they were older. They'd been with the company for a long time, and like the mindset is not there. And so their idea was to use KPIs as a way of basically catching these people out and then eventually firing them from the company. I'm making it kinda crude, but that's kind of the way it came across.   0:08:09.7 AS: Yeah.   0:08:10.1 JD: And I was like, wait a minute, let's just get down to the meat of this. The fact is, is that you hired these people [laughter] and you led these people for 20 years, who's responsible for this? And then I said, look, don't substitute leadership... Don't substitute KPI for leadership 'cause people say, so if I don't have KPIs or I don't have this, how am I gonna manage the people who aren't performing? I'm like, you know the people who aren't performing, they're probably in the wrong job. They may be in the wrong company, they may be the wrong thing, I don't know, but you need to talk to them and work it out and figure out a solution, that's leadership. But hiding behind some sort of quota or target and thinking that that's gonna solve the problem, no, that's why we need leadership.   0:09:06.4 AS: Yeah. And knowing the staff that I was working with at the time, the group of teachers, I am sure I am 100% sure that we as a group could have come up with a way, a better way to do our feedback system than the way it was set up. I have, I have no doubt. If we said, look, this actually isn't really working that well for teachers the time it's taking just to grade the homework. The kids, the students are, do they really need homework every single night? And when they're getting these papers back in the morning, do they have any time to actually look at whatever feedback is provided? Sometimes it was pretty minimal. Sometimes there was, depending on the teacher and the assignment, sometimes there was some feedback there. But are we giving kids time to look at that and actually learn from that feedback in any way? And so, again, you know, well-intentioned as it was, the volume superseded the, you know, the quality of the feedback. So I can think of all types of ways that I would sort of redo that system in retrospect with a clear aim is where I would start, what's the aim of this? Whether it's homework or classwork or whatever it is.   0:10:23.9 AS: And with technology now too, it's just such, it's gotten a lot easier. Such as give the students a five question online quiz that tests the topic that you taught that day. Then accumulate the data and understand what was the hardest one or two questions. Then in the first 10 minutes of class or five minutes of class, say, okay, last night's assignment, the hardest question was number three. And now I'm gonna randomly select one person to tell me how did you answer number three? And then let's have a discussion on that. And then that way you're getting feedback. It's the same thing I did with Feedback Friday just 'cause you were talking about feedback. Where everybody wanted, they requested in my Valuation masterclass bootcamp, they requested more feedback and I designed Feedback Friday where I gave them the exact assignment, then a certain number of them will present their work, the ones who volunteer in this case, and then they present their work on Friday. And then I give feedback that everybody witnesses and can learn from.   0:11:44.8 JD: Yeah. I mean, I think there's so many different ways, like what you're describing to set up the practice, to set up the feedback system, to have students pair up or someone present their work or you know, there's all types of better ways that would would've saved a lot of people, a lot of time, a lot of headache. There were many, many ways we could have redesigned that system.   0:12:06.4 AS: And why use the word arbitrary? You've said eliminate arbitrary numerical targets. We've talked about the numerical aspect of it, but why do you say arbitrary?   0:12:19.5 JD: Yeah. I wrote an article that actually called them arbitrary and capricious goals. So not just arbitrary, but also capricious, but yeah.   0:12:26.8 AS: What does capricious mean? How would you define it?   0:12:28.4 JD: Well, it's sort of the same as arbitrary. It's sort of like without any sort of grounding in logic or reality. [laughter] Flippant, sort of, I think I have that right. You can fact check me on that definition. But that's actually a perfect segue into that 'cause there's that second part of the principle that talks about also eliminating numerical goals for people and management. So not just teachers, but also school or network leaders. And we've talked a lot about, and across the series about various types of targets that exist in education. But I think it's still worth discussing a few points. Like what does arbitrary mean in this setting?   0:13:07.2 AS: And capricious means "given to sudden and unaccountable changes of mood or behavior." [laughter]   0:13:12.5 JD: Yeah, and I think in that article I've used capricious because in education, you know that the targets are changing so often, especially associated with test scores or other sort of parts of state accountability systems. One year it's some type of label, another year it's letter grades that schools are rated on. Now in Ohio we're given star ratings. So they've gone away from A to F and now it's schools get rated based on a five star system. And so that's what I mean by capricious. It's just...   0:13:42.4 AS: So five is A?   0:13:44.5 JD: Five is good. Yeah, five is good. Five is an A, I suppose. If there's a difference between a star, a five star and an A, I don't know, [laughter], but that's the new thing here in the Buckeye state. Yeah, but so back to the management. I think, well one thing I think when you're talking about teams or departments or a school or the whole organization, I think they should have an aim. So I think that's really important. But by aim, I'm talking about some type of clear purpose statement, but that's not so specific in detail that it stifles initiatives, that's one part. And I think a clear aim statement sort of in that spirit is something very different from a numerical goal. So that's one part of it. I think another part, and I'm getting to the arbitrary part, is that so many times that internal goals either set for management or set by management of an organization are basically a burlesque,you know, if they don't include a method.   0:14:52.8 JD: And that's what I've seen so often where, we've sat down to create a scorecard or whatever, or we see these things imposed on us from some type of organization that holds us accountable like the state department is, we have these goals, like increased student attendance rates by 10% or increase math scores by 5%. And the thing is, is part of the arbitrary is that sort of natural variation in those attendance rates or in those test scores are viewed as a success if they're going in the direction of good, but fluctuations in the other direction sort of send everybody scurrying around looking for explanations. And we end up sort of writing fiction to explain away or to explain causality when there's not actually causality there. And I've been just as guilty. We're gonna improve test scores by 10% next year without some clear plan.   0:15:53.7 JD: But then what Deming would say is, if we can do it next year with no plan, why didn't we do it this year? [laughter] And why did we stop at 10%? Why not just make it 20%? That's again part of the arbitrary. Now there is a caveat and Deming I think makes this caveat Out of the Crisis that's important. And that's when... When we are setting numerical targets or talking about numerical targets that he categorized as, he called them, facts of life. Meaning, they're just sort of plain statements of fact with respect to survival. So a good example would be, for a school would be, unless student enrollment improves by 10% next year, the school will have to shut down. But that's not arbitrary, that's a fact of life.   0:16:43.3 JD: If we don't get the revenue associated with increase in enrollment, then we're gonna have to shut down. That's not an arbitrary target. That's something sort of something, you know, altogether different that we should pay attention to. But I think getting to this idea of arbitrary, I would probably characterize it this way, and this is probably the main point of principle 11 for leaders is that they have to understand system capability. And that's often what's sort of missing from the understanding when these arbitrary targets are set. So if the system is stable, you'll get what the system will deliver. If the system is unstable, then there's no way to predict capability. So I think it's fine for organizations and individuals to have goals, but the problem is so often just like what you zeroed in on is they're arbitrary.   0:17:50.8 JD: And that's been my experience during the vast majority of my career is, why is this target set at 80%? And then no one can really tell you why that was picked as the target. For whatever the thing is, I just picked 80%, sometimes it's 75%, sometimes it's 80%, sometimes it's 90%. But I think that type of goal setting is really inevitable when as a sector, we have really no understanding of the theory of systems, we have no understanding of the theory of variation. And so without that understanding, what we tend to do is blame individuals working within school systems instead of working to improve the system itself. And I think that's really the key for me and why I'm so zeroed in on the Deming philosophy 'cause it does offer this other way. And so thinking back to system capability and before we as leaders set goals, there's really four kind of things that I've tried to focus on. And one of them is: what is the capability of the system or process under study?   0:19:13.3 JD: And when I say what is the capability, the easiest way to think about this is, if I display my data, let's say it's third grade reading state test scores. By capability, what I mean is, if I've look at the last dozen years of those third grade test scores, there's gonna be an average. And that average of those twelve years is basically the capability of my system. So I'm not just looking at last year or even the last two years, I'm looking at a dozen or more data points preferably, to see how capable that particular system is. So that's sort of the first thing I'm gonna look at is, what's the capability of that particular system or process over time? The second thing I'm going to do is ask, what's the variation within that system? So we've talked about the process behavior chart a lot. I do think I'm sort of in the Donald Wheeler camp and thinking that that is the most important tool. And so it's very easy using a process behavior chart because I have the limits, the upper and lower limit.   0:20:29.8 JD: And however wide or narrow those limits are, that's gonna tell me the variation I can expect in that particular system over time. The tighter those limits are, the less variation, the higher the quality, or at least the higher the predictability of that system or process. The third thing I am gonna wanna know is, is that particular system or process stable over time? Do I see any signals in the patterns in the data that would say that this is a stable system and therefore it's predictable or is it unstable and therefore it's unpredictable? So that's the third thing I'm gonna ask. And then once I've answered those questions, the last thing I'm gonna say is, do I have a logical answer in thinking about whatever goal we're setting to the question by what method? And so I think if you don't have that sort of picture in your head, the goals that we set are sort of arbitrary and capricious. If we are setting a goal, we can answer those four questions, then the goal is probably reasonable, logical, and grounded in some type of understanding of our systems. That's what I think Deming meant when he talked about arbitrary targets or arbitrary goals.   0:21:58.1 AS: So let me review that for the listeners out there. Number one, what's the capability of the system? Number two, what's the variation of the system and understanding a process behavior chart? Is the system, number three, is the system stable over time? Is it predictable? And number four, do I have a logical answer to the question by what method? And what you get from that is that, clearly if you can answer those questions, you understand your system pretty well. And therefore it's less likely you're gonna come up with an arbitrary goal. You're gonna go to the, say, here's what I think we can do with a deeper understanding of the system. But if you have a bureaucrat from the state education department, as an example, say, I want 5% more. Why not 5% less? [laughter]   0:22:50.4 JD: Right. Based on what?   0:22:51.4 AS: But where? Where does that come from?   0:22:53.5 JD: Yeah. Yeah. I think another thing I mentioned Donald Wheeler, he said, goal setting is often an act of desperation. And I think I talked about earlier in this series of episodes, I've talked about third grade reading test scores in Ohio. The goal is 80%. The system right now is capable of about a 60% in terms of looking at the system of third grade reading test scores in the state of Ohio. So 80% is a hope and a dream. It's somewhere out in la-la land. You know, and and what's happening is schools are being held accountable for that number in a system that is not capable of meeting that target, far from it, as a state. So I'd wanna know, like who set that and on what basis was that goal set for third grade, for third grade reading in the state? And that sort of thing is happening over and over and over and over again. What's the latest thing that we're gonna focus on? I think chronic absenteeism is one of those things right now that everybody's talking about. Kids aren't coming to school like they did pre-pandemic without any understanding of the theory of systems and the theory of variation. And so people are just running around talking about it without any understanding of what that data looks like over time.   0:24:27.2 AS: Yeah. And I'm looking at Donald Wheeler's goal is often an act of desperation, part one. And some of his discussion on that is great, great stuff. I was thinking about, in my own case with my Valuation masterclass bootcamp, when I first started the bootcamp, now we're on bootcamp number 12, but the first ones, I just told the students, okay, pick any company and then you can write a report on that company. And the outcome of that was disaster. Like it was just so... And I realized I didn't have that much teaching involved in how to get them to where I wanted to get them. So I had to... First I had to start to improve my teaching knowing that I'm trying to narrow the outcome to be, you know, somewhat consistent. And then I realized I can't just let them do any one company by choice, I have to kinda give them a list 'cause there's a certain companies that just don't have much information.   0:25:23.3 AS: If they choose it, it's a bad company for them to work on. So then I would give them a list of a hundred companies and say, pick one out of this, and each person had a different company. So that started to improve it that I had more information. And so I'm iterating through this and then I realized, some people just state that it's harder for them to do this assignment. It takes six weeks to do it and it's just overwhelming for some people. And I thought, what if each team did the same company? And I assigned it. And so what I did is I set up teams and now I encourage the teams to work together. They each wrote their individual report on that company, but now they start sharing information. And now I'm narrowing down, and I'm getting my system more and more narrow and the outcome is getting more and more narrow.   0:26:07.1 AS: And then I have a deadline that by the end of the fifth week, you've got to submit your draft. If it's not up to the standard, I can't put you in a time slot. We're gonna have to figure out something else to do.   0:26:21.5 JD: Yeah.   0:26:22.0 AS: And so that prevented someone who's postponing until the last minute, we're giving them a deadline that's a week before to give them some time to wake up and fix, make sure they got the stuff fixed as much as they could.   0:26:35.6 JD: Yeah.   0:26:37.4 AS: And then recently, so we've been iterating through this and improving the system. And now the outcome is better and better and better of what they're doing and the way that they're presenting. The way that I'm able to... I can't teach about a hundred different companies, I can teach about a few and help them in that process. And then in this particular bootcamp, my idea for improvement was what if we... Instead of starting the first week by assigning them that company and the team gets all excited and they start working on that company, let's say the company's Tesla, as an example. Why don't I pick an industry? And in the first week, everybody in the bootcamp works on an industry report, which is just a one-page report. What are the key features? And now everybody's going out and getting industry analysis, third-party research, and that's helping them.   0:27:26.1 AS: And I had four groups that are later going to be assigned the actual company. So we looked at the automotive industry and then there'll be assigned companies like Ford or Tesla or Toyota or whatever. And so by the time we get to the second week, they've now got a really good picture of the industry. And all of a sudden it adds a lot. My hypothesis is it's gonna add a lot of context to their assumptions in the final report. The reason why I'm explaining that is the idea of a process, a system. And in this case, I have to say, I'm not like measuring it very, very specifically, I'm judging the outcome based upon my experience in the prior outcomes. But if somebody came along and they said, hey, why don't you improve this? Why don't you do this with the system and set an arbitrary goal? They would have no understanding of what we've been through, what we've learned, how we've iterated through it. And without that understanding, almost anything would be arbitrary. They're interesting ideas and I listen to what people say, but almost everything would be arbitrary in a system that you're not studying or that you are studying in detail.   0:28:33.9 JD: Yeah. I also think about our previous conversation about, I don't know how you evaluate the final project, but I think of the Deming admonition to abolish grades. And you can very clearly picture if you... How many cohorts have there been of the, that class?   0:28:53.8 AS: We've had 12.   0:28:55.1 JD: 12. So let's say you lined up the sort of reports, a representative sample from cohort one to cohort 12, and you looked at the quality of the reports from cohort one compared to the quality in cohort 12. It sounds like because of all these iterations you've done, the quality is much higher in cohort 12 than cohort one. But then the question would be, who do you assign that evaluation of that grade to? See what I'm saying? Like now you can start to see why Deming said that. Because the lack of quality or the lower quality in earlier cohorts is as much attributable to you and what you were doing as the instructor as it was to the students.   0:29:42.8 AS: It was almost 100%.   0:29:44.5 JD: Yeah, so you can see... [laughter]   0:29:45.0 AS: Because I was setting the whole system. And I think that's where you get the idea of substitute leadership.   0:29:54.3 JD: Yeah.   0:29:55.7 AS: Like it's you, for the listeners, for the viewers out there, it's you, it's your responsibility. The outcome is your responsibility. The outcome is an outcome that's happening because of the system that you're running and participating in and operating. And I could go back and look at my bootcamp number one students and go, they were terrible. [laughter] But the fact is, there's no difference between the raw material that came into the bootcamp in the first group versus the one that came in in the 12th group, but they're just so... The output's so much better, so you can't argue that it's the students, it's an improvement in the system. And let me just add one thing about grade. I really don't know how to grade them truthfully. So what I just say, first, you've got to hit that deadline of having a good quality draft. I said, it doesn't have to be perfect, but it has to show that you've put time in. Otherwise, I can't spend time in your presentation with you during the final presentations, which means you're not gonna graduate. You can come back and try it again, and we can do it in another way.   0:31:03.4 AS: We'll talk about that later. But then the second thing that I do is say, if you can submit on time and you can present on time and according to the guidelines that we give, which they can do, then you pass. So I guess it's kind of pass/fail. Now, what I do is I pick out what I think was the best one of that particular cohort. And I have never announced that, and I've thought about it in my team. My other team members have said, wait a minute, what about... All you talk about a Deming, and here you are highlighting this one person and all that. And I was like, yeah, that's a good point. So we haven't really done it, but what we did do is take that one and we use that in the next cohort to say, this is the high bar. That this is one of the best ones that was done in the last group and my goal is to have you exceed that.   0:32:03.6 JD: Yeah. Yeah. To me it sounds like sort of what you've been doing to improve that particular, that class, that system, follows the Deming philosophy to a T, really. Yeah. Yeah.   0:32:21.4 AS: And I think in the end, I think the key thing and maybe we'll wrap up on this is just the... I like the idea of the arbitrariness because what it tells you is that really to set, as you said, Dr. Deming didn't particularly, he's not against goals and he's not against plans and all that, but it is that arbitrary nature of somebody just coming into a system that they didn't really know much about and setting some arbitrary goal. And really that just disrupts the system. And so for the listeners and viewers out there, if you are setting some arbitrary goal without having a clear understanding of the system, then what's holding the system back could be you. And that to me is a big takeaway from this. Any last thing you would add?   0:33:08.3 JD: Yeah, and I was looking at that article that I wrote. I was in a series called Goal Setting is Often an Act of Desperation. And the definition I used for capricious came from a law dictionary 'cause that's where capricious most often shows up is in the legal world. And it's "a willful and unreasonable action without consideration or in disregard of facts or law." And so that's what I was feeling is that often what's happening is educators are given these targets that have no basis in reality and that can only cause consternation and we're seeing churn. And so, and the people that work in education, teachers leaving and those types of things, and I'm not saying it's all for this reason, but it certainly doesn't help when you're constantly being given goals that are not set in reality. So I think if we took those steps to do those four things when we're setting goals, what's the capability, what's the variation? Is there stability and do we have a method? I think we'd be far ahead of where we are now.   0:34:09.8 AS: Boom. John, on behalf of everyone at the Deming Institute, I wanna thank you again for this discussion. And for listeners, remember to go to deming.org to continue your journey. You can find John's book, Win-Win, W. Edwards Deming, the System of Profound Knowledge and the Science of Improving Schools on amazon.com. This is your host Andrew Stotz and I'll leave you with one of my favorite quotes from Dr. Deming. People are entitled to joy in work.

The podcast discusses the importance of understanding downstream processes, the Taguchi Loss Function, the impact of variation on teaching, focusing on positive outcomes, and the significance of staying centered in different situations.

The podcast discusses the adverse effects of slogans, exhortations, and targets in education, including cheating scandals. It emphasizes the importance of prioritizing systems over individuals and highlights the unintended consequences of slogans and targets. The speakers also discuss the significance of proving assertions and providing evidence, emphasizing the need for evidence-based approaches in education.

The podcast discusses integration excellence and the human side of integration. They explore the contrast between red pen and blue pen companies, share stories without revealing names, and discuss survival skills. They also touch on taking responsibility for success, understanding variation, and encouraging independent thinking. The focus is on aligning organizations with Deming's principles and preparing for an uncertain future.

Importance of active listening in a managerial role, focusing on understanding and learning rather than just responding. The impact of prepared and unprepared discussions on critical thinking in education. Adapting to online teaching and enhancing audience participation through video materials and debates. Making time to listen and learn as a manager, prioritizing individual sessions and small group sessions to make others feel valued.