Setting the Challenge: Path for Improvement (Part 2)

In this episode, John Dues and Andrew Stotz discuss the first part of John's path for improvement model - setting the challenge. Using an example from United Schools Network, John explains their aspirations for cutting chronic absenteeism rates.

TRANSCRIPT

0:00:02.4 Andrew Stotz: My name is Andrew Stotz. And I'll be your host as we continue our journey into the teachings of Dr. W. Edwards Deming. Today, I'm continuing my discussion with John Dues, who is part of the new generation of educators striving to apply Dr. Deming's principles to unleash student joy in learning. And the topic for today is "set the challenge." John, take it away.

0:00:24.6 John Dues: It's good to be back, Andrew. Yeah. Last episode, we kicked off this new series. I introduced this improvement model that we can use to help us set ambitious goals backed with a sound methodology. I think I made this disclaimer last time. I'll make it again that this is sort of like showing listeners a peek behind the curtain because we're sort of talking about this as this model is being built and used for the first time in my network of schools here in Columbus in United Schools. So I think that caveat's important, and I think maybe starting with just a quick review of the model we looked at last time would be a good refresher for this episode for those that are reviewing and I'll talk through it for those that are only listening. I'll go ahead and share my screen quick. You see that all right?

0:01:17.1 AS: Yep.

0:01:18.4 JD: All right. Yeah, so this is the model we kind of stepped through kind of an overview last time. I think it's important to remember a few things. One, basically the core idea of the improvement model is it gives us the scientific way of thinking so that we can work in a way that makes sense to close the gap between our sort of current conditions in our organization and sort of our aspirations. So we frame those two things as the voice of the process, as current conditions, what's happening right now. And then the future aspirations, that's the voice of the customer. That's sort of what we or someone else wants those conditions to be. And what we're doing throughout this process is stepping through the four steps that you can see displayed in the model. So kind of just stepping through those quick.

0:02:06.5 JD: The first thing is that we set the challenge or the direction, and that's gonna be... We're gonna dive deeper into that step today. Then the second step over on the left hand side of the model, for those that are viewing, you work to grasp the current condition, so what's going on currently in your organization. And then the third step is we establish the next target condition. So think of that as like the intermediate goal that we're working towards sort of on a more proximate timeline. And then fourth, what we're doing is once we understand those things, then we're experimenting to overcome obstacles or impediments. And so all of those things we talked about, doing that with a team that includes someone or people working in the system, in our case students, a lot of the time, those with the authority to work on the system like the teacher in a classroom or the principal of the school building. And then that System of Profound Knowledge coach that has that awareness of the System of Profound Knowledge and sort of brings that lens to the improvement efforts. So that's sort of a quick rehash of the model that we went over in episode one. And then I'll stop. Well, you want me to leave that up or I can stop sharing?

0:03:29.9 AS: Either way. I don't mind.

0:03:31.0 JD: Okay. Well I'll stop sharing for now and then we can always pull it back up if we want to. Yeah. So with that sort of in mind, what I thought would be helpful is then do this deep dive into step one. And so kind of what we'll do through the next several episodes is focus on each of the steps. So we'll take this deeper dive into step one. Set the challenge or direction. So last time I mentioned that in step one of the model we asked this primary question, where do we want to be in the long run? And this is... Think about this as a overall challenge or direction that's set by organizational leadership typically. So senior leaders, the CEO, typically the board, they're gonna be the ones framing this challenge, setting the direction for the organization.

0:04:27.2 JD: And we can also think about it as a sort of a longer range goal that if we accomplish it, it will differentiate us in our case from other schools or if you're in the business community, it would differentiate you from competitors in some way. But even though it's something that we're striving toward currently we're setting this in a way that it's gonna stretch us and right now it almost seems impossible to accomplish this thing far out into the future. And this direction or challenge, I think it's fairly typical. Sort of set this on a six month to three year timeframe. So that kind of gives you a sense of sort of how far out we're looking and the timeframe we're looking to sort of achieve this challenge is. And last time I shared an example from our most recent strategic planning where we're trying to reduce our chronic absenteeism from the current state, which is 52% chronic absenteeism. We're trying to take that down to something closer to 5% chronic absenteeism.

0:05:44.4 AS: Seems nearly impossible.

0:05:44.5 JD: It does if you're on the ground in schools right now, especially schools like ours, it really does seem nearly impossible to sort of cut it at that large of a rate. So.

0:05:55.6 AS: I have an ethics class that I teach here, an ethics and finance class here in Bangkok. And I tell the students it's not mandatory to attend class from my perspective. School may require you, but it doesn't matter to me. My job is to make it so exciting and interesting that you wanna be here. That's hard.

0:06:16.7 JD: Yeah. That's a good frame. That's a really good frame. We have, unfortunately we have so many obstacles that our kids are going through to get to school. Even something as simple as consistent transportation from yellow school buses is a major impediment to school attendance here in Columbus. So there's all kinds of obstacles that... Challenges in front of us that we're gonna have to improve and solve to get down to that 5% rate.

0:06:43.0 AS: When I was growing up in Ohio, in Hudson, outside of Cleveland, the farm that I worked on in the summer, Barlow's farm. Mr. Barlow was our bus driver because there wasn't much going on in the wintertime. And so, yeah, he never missed [laughter] I don't ever remember a bus not arriving in my whole youth.

0:07:07.6 JD: Yeah, yeah. It's something in many places you take for granted. And then in a number of other places, it is a major, major challenge for sure.

0:07:13.0 AS: Interesting.

0:07:14.8 JD: That's certainly the case in Columbus where we are, for sure. So when we think about this challenge it's this... I kind of think about it and this is why it needs to a lot of times be set at the leadership level or the senior leader level is it's this new future condition and it serves as a sort of compass for us to follow. And that's important 'cause we don't have an exact roadmap for how to get there. So it's sort of like a general direction, but the specifics are what we're gonna have to be filling in as we go through some of those other steps. Some of those intermediate goal setting, some of the experimentation that I talked about. But think about this direction or challenge is it's really the purpose behind our efforts. And then when you sort of put together the tough challenge and then the scientific thinking, that's really powerful. And those two things together can really sort of move you into this new territory. I think that you're looking to... I think this map territory compass metaphor is really sort of spot on for this particular model. Oh, go ahead.

0:08:29.8 AS: I had a question because I have a client of mine and one of their objectives is to list their company on the stock market here in Thailand. Like many companies, but in some ways that's kind of an owner's goal. Like we've talked about some of their other goals, like being the leading company in their field in asia, Southeast Asia or something like that, and/or maybe to have a happy workforce or whatever. And I'm just curious, like how do people think about goals? What is a good way to think about it? Is there such a thing as like an internal or a higher level goal versus a goal for the company versus an external goal? How do you guys think about those types of things and target conditions?

0:09:17.2 JD: Yeah, I mean, I think... I don't know the exact answer. I mean, I think when you set a goal at the leadership level, then you're gonna... Well, one, you're gonna have to explain it throughout the organization, whether it's a 100 people or a 1000 people or 10,000 people. And then depending on the size, there's probably gonna be different types of goals that are in different business units, I'm guessing by business type. We're a pretty small organization and so we're pretty close to the... In fact, our office... My office, it is on the ground floor of one of our middle schools, so we're very proximate compared to like a bigger company. So I think this can look different in different places. I think the consistent thing is it's gotta be clear, it's gotta be spelled out, it's gotta be clearly communicated. It has to be something that you're talking about frequently. Otherwise you're obviously not gonna move in this direction.

0:10:11.2 AS: Yeah, I mean, that kind of answers it too. 'cause I thought... When you said that, I also thought about how people care about with them, what's in it for me. And so, as you said, you gotta explain it and you're gonna have to do a lot of talking about it. So people need to see that that goal is something that's gonna bring them value, otherwise they're not gonna be excited to go do the hard work that it takes to get there.

0:10:37.0 JD: Yeah. Maybe in that respect, like sometimes in education is... Some of the goals that we have set are so self apparent that there's just sort of immediate buy-in 'cause like who's against kids coming to school? Almost nobody. So that's I think a fairly easy one to get buy in. And maybe in other settings more time needs to be spent on the buy-in part, the explanation part. Maybe... This can be kind of hard, but who's involved in setting the goals in the first place? Maybe there's ways to get more people involved in that process.

0:11:10.6 AS: Well, and maybe the kids aren't involved in the buy-in.

0:11:13.8 JD: Yeah, that's true. That's true. Although, yeah, like the yellow bus thing that's out of their control. And I'd say that's actually a major obstacle. But I think...

0:11:26.5 AS: That's true. Nowadays, I'm sure there's plenty of kids that wanna be there.

0:11:29.5 JD: Oh yes for sure.

0:11:30.9 AS: But there's obstacles all over the place for them.

0:11:34.3 JD: The vast majority wanna be there, actually, I think. But your point is good, and that goes back to that team they're the ones working in the system and so they're gonna be the best at identifying the obstacles. So to stay in our setting, they certainly need to be a part of the experimentation that happens to improve the chronic absenteeism rate. One other important caveat to point out at this step in the process, and we've talked about this a little bit but I got this little chart that I think will help sort of explain when we were setting this direction or challenge, it's what I would call like an improvement goal. And it's not an accountability goal. And I think it's really important to be explicit about the difference between those two things.

0:12:24.4 JD: 'cause they often get conflated. And so I had built this chart for another improvement project, but I think it does a really good job. So I'll share my screen again. I think it does a really good job of sort of outlining the difference. And it's not that one is necessarily better than the other, it is just really important to know what's the purpose of this particular type of goal and what's it used for. And so I was just gonna take a moment to kind of run through this. So on the left you have sort of some key questions that are answered either by... And here it says measurement for accountability, but you can sort of replace that with an accountability goal and improvement goal over there on the right. So you have measurement for accountability or accountability goal, and then improvement goals or measurements for improvement.

0:13:20.7 JD: And you have some questions that that particular type of goal or measure will answer. Then you have in the next row their specific uses. And then why quality measurement matters. So just starting with accountability goals or measurement for accountability what those types of goals are gonna do is answer questions about merit or status or accomplishment of someone or something. Who's performing well, who isn't, who should be considered knowledgeable enough to do X. We're talking about end of line outcomes, like end of year outcomes. They're often... Goals for accountability often happen once a year. So I've talked about this repeatedly, but state tests would be a very good example of an accountability goal. The point of doing that is to separate the good from the bad basically, when you look at state test. Down there in the use sell for those that are viewing, it says the purpose of, or the use for accountability goals is to determine the applications of rewards or sanctions. Right? And so it's none of this really has to do with improvement.

0:14:38.8 AS: Sanctions, what a word, [laughter]

0:14:40.4 JD: Sanctions, right? Yeah. I mean, this happens in schools all the time. Schools can be sanctioned depending on what the law is at the time as it relates to state testing and accountability system.

0:14:50.9 AS: I thought we only sanctioned Russia [laughter] Okay. So there's even sanctions in schools. Okay, got it.

0:14:58.0 JD: Definitely sanctions in schools. And then we can juxtapose the accountability measures with the improvement measures over there on the right or improvement goals. When we're talking about improvement goals or setting the challenge, we're really talking about questions about specific changes as potential improvements to systems like our systems. So we're thinking about questions like, are the changes I'm making leading to improvement? How are my changes affecting other parts of my system? And what's measured is outcomes and processes relevant to the object of change. You know and how often are we doing this? Frequently. Much more than once a year, like the state test. And the whole point is to learn our way to a better system, right. And so with chronic absenteeism this could be both a measure for accountability and a measure for improvement, depending on how it's framed.

0:15:57.8 JD: Chronic absenteeism is actually reported on state report cards, but in this case, I'm talking about an internally created goal that we have for ourself that we've created for ourself that our organization is gonna work towards. And there's gonna be various things that we do to see if things that we're trying as interventions, experiments work in improving those rates, basically. So I think it's really important to call this out that when we are talking about this particular improvement model and the four steps, we are not talking about accountability goals at all. We're talking about improvement goals, two very different things.

0:16:37.6 AS: Interesting. And the improvement goals is the type of thing that it seems like is not as common as the accountability. Like everybody's trying to, you do this, you've gotta achieve this, that type of thing. Whereas this is such a bigger picture.

0:16:51.3 JD: Yeah. I mean, I think the key difference is because you could actually have an internal accountability goal. You could set up a similar system internally as what the state does when they're looking at schools. And if our mindset was, we're gonna set this goal and you people over here go do this, figure out how to do it, that would be much more like an accountability goal. But our mindset is like hey, this is something we all gotta take a look at. This has to get better. This isn't working for kids, so what are we gonna do? How are we gonna figure this out? That's really the key difference, you know? You're not doing this for some other group of people. You're a part of that group that's trying to make this thing better.

0:17:36.5 AS: So I'm just curious too, as I think about the listener or the viewer out here is how do they get started in this concept of setting the challenge or direction and maybe there's people at the top of the company. I mean, the first thing that I thought when you started talking about it is Oh, there's so many target conditions, there's so many challenges. Like there should be this one and that one. And all of a sudden I started coming up with like three to five challenges. And then I thought, oh no. Now this is overwhelming.

0:18:11.8 JD: Yeah. So in our strategic plan, we have 13 of these key metrics. And each of one of those could be its own challenge or direction, but they are divided up sort of roughly in like different department areas. And so some of 'em have to do with our fiscal responsibility raising funds and stuff like that. And there's a specific team that does that. And that would be different from like an academic team or a school-based team that was working on something like chronic absenteeism. So there is sort of a divide and conquer. The CEO maybe our superintendent are focused on all 13, but there are different teams that are actually running the experiments and working towards improving these things. And then there's someone like me that's sort of serving as the System of Profound Knowledge coach across multiple teams that are working on each of these key metrics basically. But in terms of where to start, I think that's a good segue. I mean, I think we've answered the question or we've said that basically that this strategic challenge answers the question, where's our organization going next? And I think one good... One simple way to start is to think of completing this sentence: Wouldn't it be great if we could dot, dot, dot.

0:19:37.0 JD: What is that thing or what are those things in your organization? It's, again, going back to it seeming nearly impossible, it's something we can't achieve with our current systems and processes. It's not easy, but not impossible. We think it's achievable even if we're not quite sure how we're gonna get there. It's something that's gonna be measurable. So we know if and when we get there. And another thing is that, especially when you're talking about, you were talking about like communication of these challenges across the organization, these challenges or directions are often expressed as some type of catchy statement. So just a quick statement that brings to mind this entire sort of area of work. And so I was kind of brainstorming because we've been talking about this chronic absenteeism example and I think we can just kind of keep that going throughout these episodes. So I was thinking of something like, every student every day, and then everybody knows that we think it's important for every student to be at school every day. And we're sort of working to get back to that post pandemic.

0:20:52.3 AS: Yeah. I was thinking, wouldn't it be great if every parent was fired up.

0:21:00.1 JD: Yes, absolutely.

0:21:01.2 AS: About every student every day. Like, that's the way I was just thinking about it 'cause I think that, and I just like the kids. I'm sure there's plenty of parents that say, I want my kids to get a good education and I want them to get more than what I got, but I can't reach it or I can't do it. I got too much on my plate. But if somehow they were a party to this.

0:21:26.1 JD: Yep. I think, similar to the students, I mean, I think parents absolutely can be a part of an effort where you work. And in fact almost have to be, especially 'cause we're a K-8 system and certainly at the very least at the K-5 level, kids are almost entirely dependent in most cases on parents getting them to school. So, certainly parents...

0:21:49.2 AS: What is the catchment area of your schools? Like what's the farthest?

0:21:57.8 JD: Yeah, we have pretty, because we don't have a specific geographic assigned area assigned to our school buildings, like a traditional public school district would typically. So we have much wider areas. So let's say, a 15 mile radius around a building would catch the vast majority of the kids that attend. So, yeah. Yep. Well, I think, let's look at an example. Let's look at chronic absenteeism as our focus here. So, I've mentioned, we've just updated our strategic plan it includes these 13 metrics, and there's this one focused on chronic absenteeism. And when we sort of outline the key metrics in the strategic plan, each metric has four pieces of information that we're listing explicitly right in the strategic plan. The voice of the process, the voice of the customer, the operational definition, and then some type of visualization of the data, basically.

0:23:04.4 JD: So those four things go with each of the metrics. So just as a refresher, a lot of people know this, and we've talked about this, but I think it's good to refresh. The voice of the process is the metric that tells us how we're currently performing. The voice of the customer is the direction or challenge we have set, so that's the step one. The operational definition for the metric puts communicable meaning into the concept and includes a method of measurement or test, as well as a set of criteria for judgment. So basically we wanna make it clear to anybody that's looking at chronic absenteeism, that they know exactly what it is we're measuring, and they could come up with that same measurement independently. And then the fourth thing is this visualization that illustrates the performance of the metric over time, because that time factor is really, really important. So I'll share my screen one last time so that people that are viewing this can see what this actually looks like in our strategic plan.

0:24:07.4 JD: And I'll kind of walk people through this visualization for those that are just listening. So you sort of see this chart over on the left or on top of the chart it says key metric three student success, chronic absenteeism rate. So that's the metric. The voice of the process is 52%. So that means that 52% of our kids are chronically absent and the voice of the customer is 5%. So that's that far off thing. That's six months or probably more like three years off that we're working towards. And we're not quite sure how we're gonna get there right now. And then we have the operational definition of the concept of chronic absenteeism. So this says "a student is considered chronically absent if they miss at least 10% of instructional time for any reason. Our chronic absenteeism rate is the percentage of students at United Schools who are considered chronically absent."

0:25:08.9 JD: Now, this particular definition was fairly simple because there's already a sort of a federal and state definition of chronic absenteeism. And then down below the operational definition, you have the data that we have thus far charted over time. So in this case, we only have three years of this particular type of data 'cause that's when it sort of started getting measured at the state level. And so the y axis is the chronic absenteeism rate, the x axis is the school year. So this chart has 2021/2022, 2022/2023, and 2023/2024 school year data. And you can see the data is fairly similar across those three years. Not too unexpected, but it's right around, 53%, 54% in the '21/'22 school year, maybe 52% in the '22/'23 school year. And then slightly less than 50%, let's say 48% of kids were chronically absent.

0:26:11.5 AS: Just outta curiosity, where was that before covid? Let's say 2018/2019.

0:26:17.9 JD: Yeah. Lower. Definitely lower. I think going back historically, if we had that data, I would guess, and I'm sort of guessing based on overall attendance rates compared to what overall attendance rates are and like what chronic absenteeism probably was, it was down probably closer to like high 20s, low 30s, that type of thing. And obviously a chronic absenteeism rate of 52% is very high. But when you look at even the school district buildings that are sort of around these schools, generally speaking, their chronic absenteeism rates are even higher. They're in the 60, even into the 70% range. And so not that this is like a comparison, but you can kind of get some context there that these rates are even higher in the neighborhood schools that are closest to our campuses.

0:27:16.7 AS: It would be a bit shocking for someone in Asia, like myself in Korea, Japan, China, Thailand, who's, it's a much different view of education. But just for the purposes, for someone who doesn't know much about what's happening in the US, what are these kids doing? Are they out working or are they at home?

0:27:40.6 JD: It's hard to know exactly. I think one thing is, is that in some cases, older siblings, like in our middle schools, are often taking care of younger siblings for various reasons. I think that can be a common way, but I think it's hard to just pin on one or two things. I think this is a very complex problem with lots of causal things, causal variables that are going into this. So I think that's why it's so important to study it in our context and try to figure out besides the things that we already know, like busing being inconsistent, those types of things. What else is it that's contributing to this? And so that's sort of what the process that we're starting now is trying to figure this out.

0:28:32.3 AS: And would you equate the voice, is voice of the customer equate it to target condition? Or is there a difference between that?

0:28:42.1 JD: Well, in this case, the way I'm framing it is like the voice of the customer is that direction or challenge, that's step one. So we had to, because this is an internal improvement goal, we decided for ourselves, like what do we think that vision, that purpose, that challenge is out there on the horizon that we're not sure how to get to, but we want to get to in the next two or three years. So I would equate those two things in this case.

0:29:07.1 AS: And who is the customer in the case of a school, how would you view that?

0:29:15.0 JD: Well, so that can be a little tricky based on how you're asking the question. Just from the point of who's the customer and the voice of the customer. Well, it's the leadership that set this like a customer with a capital C, that's the voice of the customer in terms of who's the customer of the school system it depends on exactly what aspect of the school system that you're talking about. But in general, our families and our students, are customers of the school system. But then so are the high schools that we feed our eighth graders into. Those are also customers of the system. But in terms of who the customer was that set this challenger direction, that was our senior sort of leadership team.

0:30:00.7 AS: Okay. And the voice of the process where we're looking at the 52% absenteeism rate, would you call that the current condition?

0:30:11.5 JD: Yes. Yes.

0:30:14.0 AS: So voice of the process, current condition, then you have to have some operational definitions so that we know what we're really talking about. And then a visualization that helps people see kind of where things are at.

0:30:28.3 JD: Yeah. And ideally we would have more data than this, but this is the data that we have. And you can actually see there's this note here 'cause right now it's just a run chart. It's just the data points with a central line running through. But there's this note that says the natural process limits are not included until we have at least five data points. And so we won't include the control limits until we have more data, basically.

0:30:50.4 AS: And for the person looking at this from outside, they're like, so wait a minute. You gotta wait a full year before you get that. But I guess there's a lot of data underneath this that is input data that will eventually drive this output.

0:31:07.8 JD: Yeah, I mean, I think what we do with any data like this that comes once a year, you have to find some proxies. So some proxy outcome data that maybe you're measuring on a every other week or maybe on a monthly basis. And then you have some process data that you're measuring, here's the things that we think will move the outcome needle, and are we doing those things? So you set up different types of measures, sort of intermediate outcome measures, process measures that are sort of measuring the different things that you're trying. And then usually a sort of a third component to that measurement system is a balancing measure where you're making sure that other things in your system aren't, [laughter] going astray because you're putting all your focus on this chronic absenteeism concept. So it's complicated. It can be complicated.

0:31:54.2 AS: It is. And in the area of education, you're under so many different constraints set by government. I was having fun in my mind imagining like when we were young, the Keystone cops, and they were kind of funny, crazy cops. But I was imagining getting an old ambulance with a flashing light and arriving at student's homes and saying, we gotta get you to school urgently. [laughter] And all the fun things that you probably can't do.

0:32:27.4 JD: Yeah. Yeah. Well, that as a brainstorm is not too far off from some of the things that we're thinking about as possibilities in terms of different forms of transportation, taking more control over the transportation where we can 'cause this is a service that's provided by the school district.

0:32:51.0 AS: One thing I did with Google Maps many years ago is I uploaded the zip code or address of my students, and so that I could see clusters of where they were. And then from those clusters, I started recommending, Hey, why don't the five of you guys form a group here and you're in the same area? And then that would help them to make a connection that they may not have made themselves.

0:33:24.9 JD: Yeah, that's a great idea. We have the geo mapping already. Yeah. We.

0:33:29.1 AS: And there's pods basically. So pods are out there of 50 students in this area. So when one has a problem you've got 20 of them that got a problem. If a bus doesn't arrive, is there a way we can get those 20 students communicating with each other and say, we want to get to school, how do we do?

0:33:50.0 JD: Yeah. We said we have some beginning of the year challenges right now with busing and we were making some calls to some of the families, and one really awesome grandma actually said, I'm gonna look into how much it costs to rent a bus and I'm gonna go round. And so that type of problem solving is certainly happening on the ground. It's how to sort of make that systematic and consistent. That's the tougher thing.

0:34:12.5 AS: Yeah. Exciting. Great one, and I look forward to the next one. I'm learning a lot and I know the listeners and viewers are learning a lot. Is there anything you would just add to wrap this one up?

0:34:25.6 JD: Yeah, I mean, I just a couple points maybe to bring it home on set the challenge, I think, one thing is we have to have this model to bridge a gap between current conditions and future aspirations. So that's this improvement model as a whole. There's always gonna be this gap between current conditions and our aspirations. And I mentioned this improvement model has this combination of scientific way of thinking and working to close the gap. And what we did in step one was ask where do we want to be in the long run? And this overall challenge that we set is really set at the sort of senior leader level, becomes a key priority. And I'm really thinking about this. If we can figure this out and some of the other key metrics, it's gonna really differentiate us from other schools. And so I think that's sort of those four or five things are the key frames for "set the challenge."

0:35:19.9 AS: Exciting. Well, I'm looking forward to the next one. Well, on behalf of everyone at the Deming Institute, I want to thank you again for this fun and interesting discussion and really opening up what you guys are doing there and the challenges that you're facing makes it even more real for the listeners and the viewers, and for the listeners and viewers, remember to go to deming.org to continue your journey. You can find John's book Win-Win W. Edwards Deming, the System of Profound Knowledge and the Science of Improving Schools on amazon.com. This is your host, Andrew Stotz, and I'm gonna leave you with one of my favorite quotes from Dr. Deming. People are entitled to joy in wo