Analyzing Data Over Time in Education

In this episode, David and Andrew ask: should we tell people when they make mistakes? How do educators manage mistakes in a classroom setting, after their organization/classroom is transformed by learning and implementing Deming?

0:00:00.0 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we continue our journey into the teachings of Dr. W. Edwards Deming. Today I'm continuing my discussion with David P. Langford, who has devoted his life to applying Dr. Deming's philosophy to education, and he offers us his practical advice for implementation. The topic for the day is, "should we tell people when they make mistakes?" We are continuing a discussion about Dr. Deming's section of the book, The New Economics. If you have the 3rd edition, that's page 86. If you have the 2nd edition, that's page 125. And this is a list that Dr. Deming has given us of 14 points. It's not The 14 points that we normally talk about, but these are... The title of this 14 points is called Role of a Manager of People. This is the new role of a manager of people after transformation, and we are on point number six. And I'll just read it before we get David to start talking on it, and that is this.

 

0:01:02.7 AS: "The role of a manager in a new style," basically he's saying, "If he understands a stable system. He understands... " And I know we can also say she, "understands the interaction between people and the circumstances that they work in. He understands that the performance of anyone that can learn a skill will come to a stable state upon which further lessons will not bring improvement of performance. A manager of people knows that in this stable state it is distracting to tell the worker about a mistake." David, take it away.

 

0:01:41.2 David P. Langford: Thank you, Andrew. It's good to be back again. Yeah, I was just reflecting on this list is... It's such a great list. I'm sure that when people first read through this book they kinda just take 30 seconds to read through the list and then you go on. I've been studying Dr. Deming's work for over 40 years now and still find so much insight into all these things. So if you go through this and you start thinking about, "Well, what can I do differently and where do I begin?" I was also thinking that, when I got my Master's in Administration, I never heard of any of this. I never heard of a stable state, control charts, theory. I never got any of this or had a list on to how to manage people, which would have been very helpful, very insightful. So if you're working at a university right now and you're a teacher of classes of administration, here's some good advice, take people through this list and they'll actually come out with capability of what to do. But now back to the list. So, the first thing he says is you have to understand a stable system.

 

0:03:01.1 DL: So we did a couple of previous podcasts on stable and unstable systems, and on face value, a lot of people think: "Okay, well, stable system, okay, well, it's working well." But Deming's thinking around a stable system is much, much deeper than that, and it has to do with statistical stability. And so if you understand a stable system, then the first thing you're gonna have to do is you have to find out, "Do I have a stable system?" And so often when I work with people, I'll just start with sort of disarming questions and say, "Hey, how are things going?"

 

[chuckle]

 

0:03:45.4 DL: And they'll usually say, "Well, it's going pretty good." And, "Okay, well, how do you know?" "Well, 'cause people are telling me that it goes pretty good." "Well, how many people do you manage? Oh well, I have like 30 people on the staff." So you're telling me that 30 people are telling you every day that things are going really well? "Well, no, one person told me." "Well, do we know what the other 29 think?" So, right there you realize a manager does not understand a stable system. They have no idea what they're doing, and the phrase in America is "you're flying by the seat of your pants" which basically means you're just…whatever is happening from day to day, you're just going with the flow, but you're not managing a stable system nor do you really understand it.

 

0:04:39.7 DL: So the first thing I wanna point out is that this is telling you, before you do anything, you have to understand the system. So you have to figure out, "Okay, what am I gonna collect data on? What's really important? How do I really understand if I have a stable system?" Then you're gonna have to set up that process and you're gonna have to do that for at least 12 data points, if not 20 data points. And in education that could mean 12 days, 12 months. It could mean long periods of time. Or the other thing is you have to go back into history and get that data for previous years. That's another way you can get the data points, put them on a run chart and see, "Is this system stable?"

 

0:05:32.7 DL: So what does that mean? Well, once you do that, you'll find out you have average performance over a long period of time. Okay? And usually there will be variation in that. Some data points will be higher and some will be lower. There are only three possibilities for a data point, up, down or the same. And that's called variability in a process over time. And so you have to understand that, and you can't understand that just by intuitively sort of going day by day going through things, because psychologically you're constantly reacting to a situation.

 

0:06:17.6 DL: You go home at the end of the day and your wife or your husband says, "How was your day, dear?" And, "Oh I've had a terrible day," and duh, duh, duh. But they usually don't say, "Well, how do you know it was a terrible day? How do you know it was worse than any other day, or better than any other day, or..." Well, until you actually understand a stable system and understand what's going on, you don't know really. You're reacting psychologically to that, and that's part of human nature.

 

0:06:47.5 AS: I wonder, David, would it be...do you think it would be...would it be proper to say that most systems that are kind of running, and haven't been looked at as to whether they're stable or not, are probably unstable?

 

0:07:04.3 DL: Not... My experience, I'd say 90% of my work has been in education, not business, etcetera, but what I found out is that intuitively people will stabilize a system over time. If they don't unintentionally do that, they kinda go nuts, they go crazy. Because one day they're in euphoria and the next day they're in hell, and then they're just... The roller coaster swings are so great that they'll usually leave the profession, go find something else. I used to see that a lot with new teachers, brand new teachers. And they call it the Fall Wall. So you start school and everything seems really great, and then you start to realizing, "Wow, this is work, and this is managing people, and this is every day, this is..." You're on and you're working through that, and then there's the Fall Wall of this huge depression cycle, and...teachers that stay with it and really have a love of kids and a purpose behind why they're there, they'll start to normally just stabilize the system. They may not get great results, but on average they're no worse or not much worse today than they were yesterday, or vice versa. But they're not gonna get a breakthrough in that system.

 

0:08:35.4 AS: Yeah, and I'm thinking about even when I recently did like a fast and my weight went down, and I had it down for a little while, and then I went back to eating, of course, and then all of a sudden you've realized there's this rebound and now that you're back, and now I'm back up to where I was and it's stable. It's not what where I really want it to be, but it's stable. And so you realize like there's a...as you're saying, people can't deal with chaos every day, so it stabilizes at some point, but that point may be far from the optimum of what that system could produce or something like that, I guess.

 

0:09:13.4 DL: Well, that's a good example, 'cause the person says he understands the stable system. Well, before you started to do that fast and actually track your weight, you were probably...your weight was probably in a normal range. That's the way it is for me. If I don't track it at all, it's usually within five or six pounds, given what's going on over a long period of time. But as soon as I track it and I start to understand what's happening and I see what the average weight is, then I have to think about, okay, am I happy with the average? So, again, this is back to the stable system. I always tell teachers, if you're happy with your average and you know it, clap your hands.

 

[laughter]

 

0:10:00.8 AS: If you're happy with your average, and you know it, clap your hands. [singing]

 

0:10:03.4 DL: And you know it, clap your hands. Yeah.

 

0:10:06.7 AS: There you go.

 

0:10:06.8 DL: So yeah. And they all laugh and everything else, but how do you know if you're happy with your average? Well, you're gonna have to collect some data. So it really doesn't matter what system you're thinking about managing, you have to understand...do you understand a the stable system? So let's say that: "Well, these kids today, they're always late to class." Okay, well, set up a run chart, track that for 12 days or 20 days, and find out what you're talking about. Find out. Are all the kids late to class? "Well, no, they're...no, they're not all late to class." Well, okay, well how many are late to class and what's happening with the data over time?

 

0:10:46.7 DL: So you have to figure it out what's important to you and how you're gonna go about that. The example I often give is, I worked with a middle school principal and he said, "Oh well, the teachers are complaining that the buses are always late." And I said: "Okay, well, that's pretty easy to track, so let's set up a run chart and you track the buses for 30 days, the arrival times of buses." Well, he did that, and not only were they not late, they were consistently really good. It was a stable system.

 

0:11:23.6 DL: 20 buses were arriving within a five-minute span. These people knew their jobs. But then one day they had fog, and one of the buses got delayed with the fog for like two hours before they arrived, and then all...so all these kids get off the bus and they're two hours late, and everybody in the front office has to get them caught up, the teachers have to get them caught up, it's a big emotional deal. But that's what everybody is responding to. They're not thinking about the stable system. They're responding to this one special cause, and psychologically it was a big problem, therefore "these buses are always late."

 

0:12:05.3 AS: Recency bias.

 

0:12:07.0 DL: Yeah, there you go.

 

0:12:08.3 AS: And it's an emotional attachment. I had two quick stories I wanna tell you, David, about this, and then maybe you can help me understand them. But the first one was that I was teaching at...I teach a program called Masters in Marketing at a university here in - Thammasat University in Thailand. 75 students that are studying in the class. And then I teach at another program in another place. And one of the things you notice is that in the other program, the students are late. You just count on it. And so you kind of don't start until five minutes after, or 10 minutes after, and let them drift in. And with the Master's in Marketing students, I've never ever seen any of the 75 students late. And here we have Bangkok traffic, you got something to blame it on. And it's never late. So is it different students or is it a different system? Well, when I investigated it more, many years ago, to try to understand it, I realized that they set a rule. They said if you're not in the classroom five minutes before, we're locking the door. Come back next time.

 

0:13:14.1 AS: And for some people that was really harsh, but once people signed up and they knew that that's the way the system worked, the output of the individuals was very different, the activity of what they did, just because they knew what to respond.

 

0:13:29.4 DL: Well, you bring up... Yeah, you bring up a good point. One way that people often stabilize systems is to make more rules. Well, let's just make a rule like that, and if you show up late something bad is gonna happen to you, or you can't get into the door or something like that, and if it doesn't work then just make it harsher. And public schools...or not public schools, but all schools, K through 12 schools especially, go through that. So if somebody's late, what happens to you? Well, you know, this is gonna happen, and then if you're late three times then this happens, and if you're late six times then this, and then so many times you're gonna lose credit and.... Does it work? It will stabilize the system to a point, but every administrator knows, there's just students that are like, "I don't care. Do whatever you want. Because I don't wanna be here in the first place."

 

0:14:31.8 DL: "The system you're running is so terrible and I hate it so bad, that [laughter] only reason I'm here is to avoid the punishment". And if that's the system that you're running, and soon as you stop doing those things, the variation is just gonna go back to where it was before.

 

0:14:51.2 AS: And is there a difference when it's...in the case of the Masters in Marketing students, they're all kind of voluntarily there, they paid a lot of money to get this education. Does that make any difference? Do you have to handle it differently or would you do pretty much the same and say, well, just squeezing down on people may stabilize, but it may not actually solve the root problem?

 

0:15:16.3 DL: Well, the answer to it actually is in this point that we're talking about here. Because the next sentence, he [Deming] says, "The manager understands the interaction between people and the circumstances that they work in." Okay? "Understands that the performance of anyone...they can learn a skill in a stable state." And so the answer to the question is right there. I would like to think about, you're starting class, if you want everybody there immediately, you're starting class immediately. So if I'm late to class, and you better make darn sure that what you're doing immediately is really important and really fun and really interesting. So if I walk into class late, I know immediately I've missed something. But the reverse...

 

0:16:06.5 AS: So it makes me think of start with a hook, start with something that's a grabbing activity that they wanna be a part of.

 

0:16:12.7 DL: Yeah, it could be. And sometimes I've been in situations where I had to manage like that, and so I would just start the class outside. If you're late, you came to class, you'd show up and there's nobody there, and you're like, "Where is everybody, what happened?" So the next class, you're probably thinking, "Well, I better get there on time because they're gonna be doing something." Or what I used to always, or still do, is tell teachers is, "Start class before class starts." What's that? Well, that means, well, when people are on their way to class, what do you want them to be thinking about, getting ready to do? Well, that's probably gonna have to start at the end of the previous class.

 

0:17:02.4 AS: Hold on, David, that's so valuable. I'm just thinking about my own students tonight, that I have my Valuation Masterclass Bootcamp, and starting class before class with the idea, I have a communication channel and I know what I'm gonna talk about tonight, and I know what we're gonna be doing. So maybe in that channel I should be throwing out some things that get them excited about, "What are we gonna do tonight?"

 

0:17:31.2 DL: Yeah, or you actually go through a quick process with them to set up a little flow chart, how to start class before class starts, and get their feedback about, "Well, what should you be doing when you're stuck in traffic in Thailand and trying to get to the university on time? What should you be doing?" I should be going over about, "Hey, what are they gonna be talking about this time and what's gonna be happening?" "Well, how am I gonna know that?" Well, somehow through the syllabus or the previous class. Or that's why ending a class by going over what are we gonna start with next time, is really a good thing, because people are like, "Oh yeah, okay, and now I know what's gonna happen as soon as I hit the door."

 

0:18:14.1 AS: So that's another practical thing right there, and I learned this from attending an online course of a guy named Brandon Gale, and he had at the end of each of his presentations, he'd say, "Up next, I'm gonna show you da, da, da." And I was just like, "Okay, that just absolutely made me wanna go to the next one." So now in all of my presentations I always have this one slide that says, "Congratulations, you've made it to the end of this section. Up next, you're gonna learn the one thing that da, da, da." And then that really helps people. And I have another...

 

0:18:47.4 AS: Okay, so now here's another actionable thing that a friend of mine does. He issues podcasts out to the world, but they're directed at his students. And think about following up - a preview podcast about this material that then is published out there to the world, but his students know they need to listen to it when they're in the car on the way to the class. So, yeah. Okay, these are some great, great ideas, let's continue.

 

0:19:15.6 DL: Yeah, So that's a really great example that: here's the process that we're gonna go through. And so you need to have listened to this podcast because when you hit the door, you're gonna be put into random groups and you're gonna be asked to start a discussion immediately on the podcast that you were supposed to listen to. See? So now the responsibility is shifted from me as the instructor to them as the students: it's your responsibility, as soon as you hit the door, get together with three other people, you got 10 minutes to go over the podcast, and what were the significant points that you got out of that.

 

0:19:53.0 DL: Well, somebody that didn't listen to it, is gonna be in that group and they're gonna feel very foolish once or twice, and then they're gonna start actually paying attention and doing the podcast.

 

0:20:05.8 AS: So for the listeners and the viewers out there, here's a challenge, here's a challenge to you: take some of these actionable ideas and play with them, enjoy them, bring them to your students, bring that into the classroom. You highlighted something that I didn't think about either, but the idea of starting class with something like pleasurable, rewarding, something like that, and when I start...

 

0:20:31.3 DL: It doesn't necessarily even have to be that. It has to be something that needs to be relevant, needs to be timely and it needs to be engaging. So I have to do something as soon as I hit the door. I have a responsibility as soon as I get there or online, if it's an online class, and what do you want them to do immediately as they're sitting there waiting for the online session to start or something, right? Those are all processes that's gonna get you a different stable state than if you just wait and you're gonna tell them what to do when things start. And people learn from that too, because they'll learn that, "Oh well, don't bother to read the syllabus or understand what's going on because he's gonna tell us what to do as soon as we get there." They'll learn really quick that they don't have to think or plan, or do anything, because you're gonna tell people when they get there.

 

0:21:38.7 AS: Right. One of the other things that I tried something kind of bold, I know my team was a little bit worried about it when we did it with the Valuation Masterclass Bootcamp, we do live sessions at 6 o'clock, two or three nights a week, and basically, what we started was a breathing exercise and I recorded it as an audio clip where I just say, breathe in for four seconds, count one, two, three, four, hold it for four seconds, one, two, three, four, let it out for four seconds, one, two, three, four, and then we go through that cycle three times and it was a risk to bring that out. I felt like some people may think, "Oh, come on."

 

0:22:15.2 DL: It's a yoga class or what?

 

0:22:18.3 AS: What's that?

 

0:22:19.4 DL: Is this a yoga class?

 

0:22:21.8 AS: Exactly, so I went... By the time we got to the end of that first class where I tested this new idea, I asked them, "What did you think about that?" And they just absolutely loved it. And I know that breathing can actually stop your intensity that you've had from your day and kind of separate that from the actual class, but I can say for myself as the teacher, I enjoy that moment that we all have together where we breathe together. And I know it puts me in a different state, and I know that the value of that is, so that is something that I think students like to be a part of, they see the value in that, they feel the value of it, and therefore they wanna be on time to be in that. So that's an idea.

 

0:23:03.8 DL: There you go, there you go. So I wanna get back to this. So he [Deming] says he understands that the performance of anyone that can learn a skill will come to a stable state. So that's what we've been discussing, you've taught people a skill and now you stabilized it and it's coming to a stable state, that this is what we're gonna be doing. "Further lessons will not bring improvement or performance, a manager of people that in this stable state, it is distracting to tell the worker about a mistake." And so that's the topic for our session today, are way supposed to be telling people about mistakes?

 

0:23:40.4 DL: Well, Deming's thinking around a stable...the interaction to create a stable system goes also to why he was so adamant against grading people. That it's not just you doing your own work in a system, it's the interaction of all these people within the system that gets the result. So if you set up a system where you are doing that, you're actually forcing individual, individuals to work alone, don't share anything, don't talk to anybody, don't... Well, what's gonna happen is you'll have an isolated few people that'll really do well, right? They get the A's, they get the top marks, the top grades, whatever it might ever be, and then you'll have everybody else stratified layers, B, C, D and F or whatever you wanna say, on down within that, because what you're really doing is shutting down the interaction of people to create a different result.

 

0:24:51.1 DL: Well, even in that kind of a world, teachers learn to create a stable system within that. "I got rules and no, you can't do make up work and you can't do this, and you can't do that, because I've got my stable system that I must maintain the system over everything else," which flies in the face of understanding why people are there in the first place, right? They're supposed to be there to learn not to play this game or figure out what the game is for people to go through the system.

 

0:25:31.1 DL: So what [Deming] he's talking about is that if you're in a system and somebody's making mistake within that system, you're actually...and you're pointing it out or whatever you're doing, you're actually blaming them, the individual.... This is what I believe he's talking about. You're blaming the individual for that mistake when the mistake may have come from the system itself and they can't change the system, only the manager can change the system, whether that's the teacher or a supervisor in a company, or a vice president, a principal, whoever it might be. The individuals typically can't just change the system, right? They can't just come up to the teacher and say, "well, I'm just not gonna play your system anymore, but I'm gonna do this system 'cause it gets better results." Most teachers would kind of freak out about that and, "No, you're not. You're not doing that."

 

0:26:26.3 DL: If somebody did that to me though, I'd say, "Well, tell me about it, tell me about what you're talking about and why does it get better results, and maybe we should try that as a class and maybe we should learn from that and try to figure out...maybe we will all do that, if we can figure out that's gonna get a different result."

 

0:26:46.9 AS: And what would happen if you have a worker who just makes a mistake, they're supposed to have done a particular step and they didn't do that step or something like that, some people would feel like, "Oh, well, he should be blamed or she should be blamed for that."

 

0:27:03.5 DL: Deming often talked about, he went and visited a company, I think in Ireland or something like that, and a big sign in the company, that says "465 days without an accident" in the company, right? And then they go to walk up the stairs and the railing on the stairs was so loose that he almost fell off the stairs, going up the stairs. Well, that would have been an accident, and is that the individual's fault or is that the system? And when you start thinking like that, especially in a classroom, it's the same kind of thing: if somebody doesn't understand a concept, let's say you go through a process where you train people and everything else, but then you've got one or two people out of the group that didn't do it right, don't understand, etcetera.

 

0:28:00.9 DL: Well, whose problem is that? Well, that's a level of variability, a variation, that eventually you want to limit that variation, so instead of having one or two people every class period that don't understand, now I'm down to once a month, somebody doesn't understand. Now I'm down to once a year, somebody doesn't understand. Now I'm down to about once every five or 10 years, I'll run into somebody [chuckle] that doesn't respond to the process I've set up, and they really are a special cause and I need to treat them like a special cause. I'm not going to change the whole system for that one individual, but I am gonna help them individually because they're not part of the stable state that I've set up and that I'm working within that over time.

 

0:28:51.8 AS: And so would it be right to say that, okay, once we reach a stable state, really a lot of all the variability and the mistakes, people are gonna make mistakes, that's all kind of noise that's not.... That there's nothing meaningful about that, and therefore pointing out this mistake or that and why did this happen? It's just chasing around common cause variation, and truthfully, the job of a manager should be able to be to think way beyond that, and you wanna encourage everybody through education, not training, we're talking a lot about training here of getting people to a standard stable state, but then the idea is, come on, well look at the opportunities are huge ahead of us, and that is just normal variation.

 

0:29:37.2 DL: Yeah, I'll give you a personal example [chuckle] from me. I was in an honors English class when I was a junior in high school, and we're all supposed to be top kids in the school, and these English class and working very hard. Well, I was sick for the whole week. And we were studying Arthur Miller, the famous playwright.

 

0:29:58.5 AS: Death of a Salesman.

 

0:30:00.1 DL: Yeah, and I came back and I had missed what the assignment was and everything else, and I went up to the teacher real quickly and said, "What did I miss? And what am I supposed to do?" And she said, "Well, everybody in the class was assigned a different aspect of Arthur Miller where things through...so she said, "What I like you to do is I'd like you to make a bibliography of Arthur Miller." Well, whether it was because I was sick or whatever, I heard her say, biography. So I went home and spent a week writing a 14-page biography of Arthur Miller's life and everything, and even hand wrote it out three times to make sure it was all clean and everything else, I was so proud of it at the end of the week, brought it back in and handed it to her before her class started.

 

0:30:54.5 DL: And she looked at it, she says, "What's this?" And I said, "Well, it's my assignment," and she looked at it and she threw it in the trash can, and she said you were supposed to write a bibliography," and [chuckle] I was just like stunned. I'd made a mistake, but whose mistake was it? So verbally, those two things sound a lot alike and etcetera. I turned around and walked out of the class and went up to the principal's office, and I was gonna change this situation and get it changed. So I mean there's a good example that she was pointing out a mistake that was the interaction of parts to the whole within the process. Simple change would have been to have all the students listed out and all the things that they were supposed to do on a piece of paper and not just verbal announcements about what you're going to do, right? Because there's a certain amount of variability, obviously in a verbal announcement. That make sense?

 

0:32:04.1 AS: Yep.

 

0:32:05.2 DL: And so if you keep operating like that, it makes you as the manager in total control of everything, but it actually turns everybody else into victims to some degree.

 

0:32:17.3 AS: Well, let's wrap this up. I just wanna go back through this one and read it again, because there's just so much to it. Number six: "He understands a stable system. He understands the interaction between people in the circumstances that they work in, he understands that the performance of anyone that can learn a skill will come to a stable state upon which further lessons will not bring improvement of performance." And finally, to wrap it up in relation to what we titled this, "A manager of people knows that in this stable state it is distracting to tell the worker about a mistake."

 

0:32:51.0 AS: What a great discussion. I appreciate it. I got some actionable ideas out of it, and I think for everybody out there, put some of these ideas into place. And David, on behalf of everyone at the Deming Institute, I wanna thank you again for this discussion. For listeners remember to go to deming.org to continue your journey. And listeners can also learn more about David at Langfordlearning.com. This is your host, Andrew Stotz, and I'll leave you with one of my favorite quotes from Dr. Deming, "People are entitled to joy in work."