Tracking Progress and Characteristics of Quality in Education

David and Andrew continue their discussion on how to track student progress when you don't use grades or other conventional methods.

TRANSCRIPT

0:00:02.3 Andrew Stotz: My name is Andrew Stotz, and I'll be your host as we continue our journey into the teachings of Dr. W. Edwards Deming. Today, we continue our series of Deming in Education with David P. Langford, where we explore Deming thinking to create joy in learning. David Langford has devoted his life to applying Dr. Deming's philosophy to help everyone get the most out of learning. Today's topic is a continuation of the discussion on tracking progress in learning. David, take it away.

0:00:34.4 David Langford: Thank you, Andrew, it's great to be back again. In the previous podcast, we were discussing tracking learning and the typical way to track learning is grading people; A, B, C, D, and F, and Deming was very adamant that we could significantly improve the education system if we just stopped grading people. So, in my work with education over the last 30 years, a lot of educators get that, and they don't like grading and they've never liked having to do it and being the final judge. And then there's another whole group that thinks it's their right to judge people and give them a grade about what they could do. So, I mentioned in one of the earlier broadcasts that Deming said, "Why would I wanna judge somebody today when I don't know who's gonna turn out to be great in the future?" So I wouldn't wanna do anything that's gonna limit them.

0:01:33.2 DL: So as a teacher myself, having to think through that and having to actually work inside of a grading system and try to figure out what you could do, I think you first have to go through the thought process; is it possible for everyone in a class, for instance, to achieve. And if you say to yourself, "No, it's not possible." I had some students that said, "It's just not possible," they can't do it, you're probably never gonna get there. But if you start to say, "if it was possible, what would we have to change in the system in order to optimize everybody getting to that point?" Well, it always turns out that through neural science, every educator, even parents, will tell you that everybody learns at a different rate. You give somebody a complex problem or something somebody might be able to answer that in three seconds, and other people it might take them a very long time, but they could eventually get it, it just might take a lot longer for you to get there. And so, we sort of truncate that in education, and we talked about, last time, about deadlines and what deadlines mean, and those are mostly for the person managing the class to keep the class moving, right?

0:02:57.1 DL: Because if I just sort of make it open-ended and say, "Okay, well, everybody has to get to a certain level of performance, and we'll just keep it open until you get there," most teachers will tell you it would just be chaos, so the idea of changing a deadline to a target date, so... Yes, here's what you need to know and learn, or the process of what you need to go through. And our target date for you to finish this is this Friday, so then we run into the problem, well, what happens if somebody doesn't do it, or they don't do it at all, right? Well, in the current systems, if somebody doesn't do it at all, some teachers actually like that, 'cause then you don't have to grade people, you just give them a zero, right? And you go on. But if you think about, "no, my job is to optimize that child's performance." So if you didn't get it done, then we're gonna have a conversation. "How quickly can you get it done? When can I expect to see this?" That you're not getting off the hook, so to speak. I observed this with high school classes I was teaching when I first met Deming, and students would just tell me, "just give me a C," or "just give me a C or a D or something", and sometimes they would be basketball players or something like that, and they'd say, "Well, I just need a D so I can play basketball."

0:04:29.6 DL: "So that's all I really want. What do I have to do to get a D?" [chuckle] So all this thinking theory comes into play when you think about, "Okay, well, how do I have to change the system?" So if I change the system to one in which I say, "Well, there is no such thing as sub-level of poor performance." [chuckle] That make sense?

0:04:54.2 AS: [chuckle] No. What does that mean?

0:04:57.2 DL: I want everybody to do A-level type work, so if I'm gonna try to get everybody as close as I can to that, then we're gonna have to define that, and Deming would call that an operational definition. So it has to be very clear to everyone; what do you have to do to get to this level of performance? And then if people understand clearly what that operational definition is, I called it a quality standard; here's the quality standard for this, and I learned over a period of time to always ask students, "If you were to do this really well, what would it be like? Or what would it look like?" And boy, they're really strict on it, and they were often more strict than I. But first getting their input also, I got their buy-in on it, they said, "Well, it should be this and it should be... The writing should be clear." "Well, what does that mean? Writing should be clear?" So we're gonna have to operationally define that and that process could take a while. But Deming talked a lot about that prevention is the key to quality, so I'm probably gonna spend more time up front when we have an assignment, or task to be done, or a project, or whatever you wanna call it, and defining the standard for quality, because as we go through the process, I basically want people to self-evaluate themselves, right? And whenever they...

0:06:35.6 DL: Yes, we have a target date that we're gonna need to get this in by Friday to stay on our overall plan for the whole quarter semester, whatever it might be, so I'm gonna have to share that with them also. Right? 'Cause there's ramifications if we don't get this done by Friday, that's gonna cause us to fall behind as a whole class. If we keep on that track, we're gonna get further and further behind. So this thinking all comes into play because you also have to understand special and common cause variation that Deming talked about, and in other broadcasts we'll get much deeper into what that means statistically in education. But briefly, so here I have my target date on Friday, and then the projects come in, then I have to take a look as a teacher, do I have common cause variation with that, meaning that probably 90... Deming talked about 90%, 94% of the students all attempted to do something. And then I have special cause variation, so I have some kids that didn't do it at all, or they did it so poorly that they're gonna need special help.

0:07:50.2 DL: That's what Deming talked about. They don't need more rating and ranking, it's not gonna do any good. They need special help, which means I'm gonna have to spend time with them one-on-one and go through, well, What didn't you understand? And what can we do and how can I help you? And so and so forth. That's gonna take my time. Then I have the common cause variation, which is that 94% of students who didn't make an attempt. I wanna take a look at all of that work and start to say, Okay, are there common cause problems within that? So probably most of the reasons that you're getting common cause variation or problems from a whole classroom of students has to do with your process as a teacher. [chuckle]

0:08:35.7 AS: Interesting, it makes me think about delivery of products in a company.

0:08:42.8 DL: Yeah, it's the same principle. Same principle.

0:08:42.9 AS: Yeah, you've got an objective that you wanna deliver this exactly two hours after you've packaged it in the warehouse or whatever, you want it to arrive, but there's a lot of different factors. But let's say you set a target time based upon the location that you're delivering to, and in the queue of where that is, but you've set an approximate time and your objective is to try to hit as closely to that time. Now, many people may say, Oh no, actually your objective is to hit earlier. But not really, I think to make it a really robust system, you need to be really accurate. And so when I think about it with education, I would say that from a... You want everybody to submit at a certain time, but you're gonna have a small number of people that are just super stars, they're gonna submit early, and majority of people that there's gonna be this long tail of submitting late.

0:09:32.2 DL: So you can have special cause variation on the high achieving end in education, and you can have special cause variation on the lower end, not cheating. And both of those you wanna handle it as special causes. You remind me of... I had a college professor one time that very clearly told us all, "Do not hand anything in before it's due, even if you're finished." And I went up to talk to him afterwards. I said, "Why do you tell people to do that?" And he said, "Well, you just give me more time to grade your work."

0:10:08.7 AS: [laughter]

0:10:09.4 DL: "And I will find something wrong with it." So it's like inspectors in a house, if your job is to be an inspector, that's what you're gonna do. You're gonna find something wrong. Otherwise, why do we have you.

0:10:22.7 AS: Yeah. "I didn't find anything there. I didn't find anything there." What are you doing?

0:10:26.2 DL: Yeah. Why do we need you then? Right. So, I want to get back to this process of, Yes, you have a target date, etcetera. So if I have common cause variation and a large percentage of the students are not meeting the target date and hitting the target standard for that, that's probably a systems problem, a common cause problem. And when you go back and you ask students why, Why is your work not meeting the standard? It could be internal forces, outside forces, there could be all kinds of things like that, that... Maybe the common thing is, Well, I didn't have enough time. Alright, then, where's your time going? What are you doing with your time? We can track that. We can figure that out, etcetera. And you may be right, the task that I'm giving this group actually requires much more time, so in my process upfront, if I'm going to the students and saying, Okay, here's the quality standard for this assignment, how much time do you think you're gonna need to get your work to this standard and turn that in? And so I created a tool for doing that, called the Loss function to figure out, and I got that from the Taguchi loss function and Deming.

0:11:55.0 DL: And I'd just ask students, How many days is it gonna take you to get it to this level? And that was also fascinating too, because many, many times they would take a look at everything that needs to be done, look at all of the other things that they need to do, and they would set a timeline shorter than what I would have done. So I'm actually improving the quality of all the work and shortening the timeline at the same time, which is gonna enable me to move kids to a higher and higher level than ever thought possible before in the same process, simply by asking them. And some things, yes, it might be shorter and some things it might be a bit longer. But I'd rather err on the side of a bit longer and have more students get to an A-level or a quality standard for this than do the opposite process, just arbitrarily set a due date and then grade all of the people performances that comes in. So if we get to the target date, and I look at a child's work. I'm looking at it, I'm not grading it, I'm looking at it to see, does it meet or exceed the standard for this assignment? If it meets or exceeds. Great, [chuckle] right.

0:13:16.2 AS: Yep.

0:13:16.7 DL: And then we can talk about you know how do we put that in a marking book or whatever it might be. What if it doesn't meet or exceed I asked Deming this question, because he taught at New York University, I said, What do you do when you get papers in, and clearly it's not to the standard that you think it should be. He said, Well, I have a conversation with them. I teach, a very, very strange practice, right?

0:13:42.6 AS: Yeah.

0:13:43.7 DL: I'd go back to them, I start to say, Well, I need more explanation about this, or I don't think this is quite right, and I think if you corrected this, you might be right or this, and then they get time to correct that and make it right. It turns out psychologically, which is part of Deming's profound knowledge right... Psychologically, this has huge impact, the power that if I didn't quite get there in the time that I was supposed to get there, I can go back and fix it. I can do it, yeah because...

0:14:14.6 AS: Welcome to the real world.

0:14:16.6 DL: Yes, because my job is to make sure you learn this. Right. Not to play some grading game or... Time game, right? My job is, you see that you learn this material and basically remember it for the rest of your life.

0:14:30.9 AS: Yep. So in the last couple of minutes, let's try to wrap up what we've learned, what we've discussed, I just think about some of the things that I wrote down while you were talking... Right? You made me think like, Okay, the goal of the system of education is to help each kid optimize their learning rather than just a classroom or a teacher optimizing their job. The second thing is, we talked about that you can let go of deadlines, and once you do that, you need to set better maybe operating, operational definitions or what you call quality standards, to more clearly design what it is or define what it is, is a good outcome this is the book, it's called Tom Sawyer. And these are the things that I want you to get out of it, or you internally, or as you said, you can go to the students and talk about what you want out of it, and then special help. The other thing you said is there's special help or special causes where we have someone, for instance, that's struggling, needs special focus, not rating and degrading them with that. What else would you add to the summary of the learnings from that.

0:15:58.0 DL: When you hold everyone to a high standard, you're actually improving performance for the whole system and for the whole standard for everybody and then what people quickly learn is that... Well, if I turn something in on the due date, just because I wanna turn it in on that date, what's gonna happen to me? One, I'm just gonna get a whole bunch of feedback and I'm gonna have to fix it, and now I have twice as much work to do, because now we're going on to the next thing that I need to be working on, and I have to get this fixed up. So what you're teaching people is try to do it right the first time, build quality into your processes, and that was partly my job as a teacher too, is to teach them how to do that, so they get closer and closer and closer to that target date and so they always feel gratified. And in doing that, basically, you can get to a point where almost everyone in your class is getting an A or what we would call A-quality work, right? And when that happens, you have joy in learning, and you have it on a massive scale because everybody's very happy and so...

0:17:13.6 AS: That's great, I love that. And the other thing I was just thinking about is you said something really kind of mind-blowing right at the end, and that was the idea of once you start working with someone and you start... You know you forget about the deadline, like what if we let go of the deadline I can imagine parents thinking that, but what you just said is, you open up a whole new world to this kid who may have been just always struggling with this deadline, and instead you're saying, I want your contribution, no matter what the deadline says, and that... Then as you said, opens up a whole new world for that student to think, Wow, okay, this is different, now I really wanna contribute and so.

0:17:58.1 DL: Well, it's that you... Some people think that maybe this is semantics, etcetera, but by not using the word deadline and I change that to a target date... Makes much more sense. This is the target date. Did I hit the target date? Yes, great. I have nothing to worry about and my work meets or exceeds the standard. Awesome, great. You have nothing to worry about, right? I didn't meet the target date, okay, I got some worries, I gotta get this fixed up, or it didn't meet the standard, Oh, I gotta get this fixed up. So anyway, my goal is to get them to have that joy in learning, because if you think about a student that sort of slopped through it and didn't do a good job and is continually getting Cs, Ds and Fs on things, there's not much joy in that right, right?

0:18:52.7 AS: Definitely. Definitely.

0:18:52.8 DL: You want people to feel really proud of the work Deming talked about that too. Pride of workmanship.

0:18:58.1 AS: And ranking doubles down on that unhappiness feeling that they're feeling right then.

0:19:02.7 DL: Absolutely.

0:19:04.9 AS: So, David, thanks for your contribution, and on behalf of everyone at the Deming Institute, I wanna thank you and our listeners for striving to bring joy in learning.

0:18:52.8 DL: You want people to feel really proud of the work Deming talked about that too. Pride of workmanship.

0:18:58.1 AS: And ranking doubles down on that unhappiness feeling that they're feeling right then.

0:19:02.7 DL: Absolutely.

0:19:04.9 AS: So, David, thanks for your contribution, and on behalf of everyone at the Deming Institute, I wanna thank you and our listeners for striving to bring joy in learning.