Item Analysis and Histograms for Feedback

What if more quizzes created more joy—not stress? Lee Jenkins shows host Andrew Stotz how Deming-inspired practices like random-concept quizzes, student-led charts, and "all-time best" celebrations turn classrooms into true learning systems that build confidence, motivation, and real understanding. A simple shift in method—massive shift in joy. (View the powerpoint referenced in the podcast.)

TRANSCRIPT

0:00:02.2 Andrew Stotz: My name is Andrew Stotz and I'll be your host as we dive deeper into the teachings of Dr. W. Edwards Deming. Today, I'm talking with Lee Jenkins, who is a career educator in public schools, completing his full-time work as a school district superintendent. During that work, he was introduced to the teachings of Dr. Deming and has been applying those teachings to his life and work since. In his business, Crazy Simple Education, he helps people apply Dr. Deming's principles in their schools to bring joy back to learning, to help kids learn more. The topic for today is how educators have applied Dr. Deming's ideas to learning. Lee, take it away.

0:00:42.8 Lee Jenkins: Thank you so much, Andrew. It's amazing what Dr. Deming taught in five minutes. I've been able to teach that for over 20 years. It's just amazing. And then you see in the next slide, it was Lou Rhodes. And this is just a short little review of what we did on the first podcast. But he's the one that said, I think you're going to enjoy this. Little did he know how much I was going to enjoy that in 1990 when he said that. And then in 1992, heard Dr. Deming in person as the statistician. And he described in five minutes just a little touch of what was different about a classroom as opposed to all the other systems that he was teaching. And so over time, you're going to see how it's been implemented with great joy with so many people. He taught that education should have a learning system instead of an inspection system. And that's what we have, is an inspection system. The state departments of education inspect the schools and the teachers inspect the kids. We don't have a learning system. So if you think about that distinction, it's truly a learning system. And you're going to see that as we go through this today.

0:01:51.2 Andrew Stotz: Lee, I was just... After listening to you in the last episode and listening to some of our other great guests on the show, I talked to my students about this. And one of my students, after I went through it and talked about the random sampling as an example of questions to understand the level of knowledge that students as a group are getting, one of my students at this prestigious university I teach at in Thailand said, "So why are you grading us? "

0:02:26.1 Lee Jenkins: Yes. Yes. That's it.

0:02:27.4 Andrew Stotz: And I said... Lee, I need help. I gave my best answer and that is, "I decided that right now, the fight with the university to change the way it's done is not a fight I'm prepared to take. But what I'm going to do is try to deliver the best experience I can in the room." Now, that was a bit of a cop out, but that's part of... People who are listening and viewing this are also caught in a system, in a trap, an inspection system. So it's just great to hear you talk about this and it can help us think about how we can handle it.

0:03:09.9 Lee Jenkins: People say that education hasn't been improved for 50 years. Then think about it. We've had an inspection system for 50 years. Maybe that's the problem, right? So here's what Dr. Deming taught. Tell them what you want them to know first week of school. Here it is. You're going to give them a weekly quiz. The quiz is going to be the square root of the total number of concepts you want them to learn. So a teacher teaching a second language, 400 vocabulary words, they had 20 words a week at random out of the 400. It's simple, but it's crazy that you don't... People say, "How can you assess them on something you haven't taught yet? " You can, if you have a learning system. And then he said to build a scatter diagram and a class run chart. And let's look at those two just to review. The scatter diagram, and if you can't see this, it's just across the x-axis on the bottom. It says 1 to 14, which is for half a year. The y-axis goes from 0 to 10 because there are 10 questions every week in this classroom. And we have a dot by how many kids got 0 right, how many kids got 1 right, how many kids got 2 right. And if you look at over a semester, you can see all the dots moving from the lower left corner up to the upper right corner. So that's the scatter diagram.

0:04:29.7 Andrew Stotz: That's all the students in the class. That's not one individual student.

0:04:33.0 Lee Jenkins: That's not one student. It's the whole class because you're the manager of the learning of a classroom. He taught that. And then he said graph the total correct for the whole classroom.

0:04:46.6 Andrew Stotz: So you just did what he said.

0:04:49.8 Lee Jenkins: Yeah, add it up. It is simple and it is crazy. I mean, all the coaches who are listening to this know when you go to a game, you add up the total for every athlete. You add it up to get a total for the team. Then that same coach is in the classroom on Monday and they never think about that this is a team of learners. It's the same thing. Add it up. And they love it. And they help each other and they contribute and they celebrate when a struggling student helps the class out as much as a student that's advanced.

0:05:24.2 Andrew Stotz: Yeah. I mean we're social creatures, right? We want to be part of a group. We want to contribute. It's just such a clear principle.

0:05:35.0 Lee Jenkins: Yeah, it's simple. So then here's the expansion. Here's different things that happened over time with the process, and we'll share those with you. One was people said, well, ya know, my problem is not... It's partly they don't remember what I'm teaching well enough, but they clearly don't remember the prior courses. So this is a high school math teacher teaching geometry, and so she has half of her questions are coming from geometry because they're teaching geometry. But the other half of the questions come from the four courses they had in math prior because she knows they don't remember it. And then there's a secondary science department. The same thing. They said half of our questions for every quiz have to be from the prior courses, not just the current one. Because students think...

0:06:29.6 Andrew Stotz: Wow! That's fascinating. And before you go for that, so let's look at geometry. You've got these buckets. Before geometry is algebra two, and before that is algebra one, and before that's pre-algebra, and before that is math seven. I remember my pre-algebra class at school with Dr...Mr. Tyler. He was the football coach, and that guy was a slave driver. Even if you got the question wrong, if you structured your answering process right, you would get half points.

0:06:58.9 Lee Jenkins: Oh, okay. Yes.

0:06:59.6 Andrew Stotz: He helped me learn the structure and the order of solving algebra problems, but if I didn't do that well or I didn't have him as a teacher, I could end up in geometry not actually knowing that. But what the heck is this geometry teacher supposed to do if they find out that the class doesn't really understand some of the prior core principles?

0:07:21.7 Lee Jenkins: Well, they, obviously, they need to teach it, and so part of it they do. The other part of it is the kids don't want to forget the prior courses. If you just throw all these into a bucket and they don't say where it's from, they don't... Well, okay, I missed a question. But when you say, you're in 11th grade in geometry, and you missed the 7th grade question, they don't like that. So it builds, it's a visual. It's right in front of the room every day. They can see, I need to know all of this. And the science teacher is the same thing. The kids say, I'm in chemistry now. I don't need biology. Why do I need that? Until you see it right there in front of you every day, and you think, oh, I'm supposed to learn this.

0:08:12.9 Andrew Stotz: Gosh, it just brings me back to when I was in high school, and I really got frustrated because the pace was really fast, and I felt like I didn't fully understand the prior material, and now I'm on to the next. And that was, and I felt like I was building on a shaky foundation, and this is a part of addressing that.

0:08:33.7 Lee Jenkins: It is, absolutely. So that's one of the changes that was made. Teachers took and expanded that to the whole curriculum as opposed to only the course they're teaching.

0:08:43.0 Andrew Stotz: And just to think about that, is that in order to truly do that, you really want to have the math, the pre-algebra, the algebra, the algebra 2, and the geometry teachers all working on the same playbook.

0:08:56.2 Lee Jenkins: Yes, yes. And when we do make those lists for each class, there's no duplicates.

0:09:02.7 Andrew Stotz: Right.

0:09:04.3 Lee Jenkins: I mean, like with the science, I remember the biology teacher saying to the chemistry teacher, "You teach that? I teach that also." And they'd been teaching next door to each other for 10 years and didn't know it. So they have to say, who owns that one? So it's all a system that's tightly designed.

0:09:25.1 Andrew Stotz: And in the academic world of universities where I've taught, there's this thing that they want to give you independence to teach what you want in the way you want. I don't know about what's happening in schools these days, but is the curriculum pretty much set and therefore the teacher can't veer from that and therefore this would not be a problem? Or is it that, hey, every teacher's doing something different and it doesn't all work together?

0:09:53.6 Lee Jenkins: Right. What's the "what." The essential "what", needs to be agreed upon no matter who's teaching it. Now, on these lists, we don't put trivia. And trivia should be in the classroom. It's fun. It's interesting, but they're not accountable for it.

0:10:11.3 Andrew Stotz: Right.

0:10:11.7 Lee Jenkins: So it's what's essential for the kids to know. And the teachers, when they have time, the principal sets aside a day and said, okay, science department, get together, get this listed, what you want. They like that discussion and the agreement of what's expected.

0:10:30.1 Andrew Stotz: Yep.

0:10:31.8 Lee Jenkins: The next thing that was added, Dr. Deming did not talk about students graphing their individual progress. So this is a student run chart, not a class run chart. So you can see...

0:10:46.3 Andrew Stotz: Yeah, that's interesting. Before you even go into that, it makes me think about the factory. It was kind of accepted that the statistics guys would kind of run the run charts and management would look at it. It would be public, it wasn't hidden. But the idea of really bringing the accountability to the people on the production line is what this reminds me of.

0:11:10.0 Lee Jenkins: It's exactly the same, and the kids like making the graphs. When you see, this is a younger child, but it's done by a high school child, not all of them, but some of them, but who like to doodle, they become very, it's kind of pieces of art, but they own it. They own that learning. They can see how they're doing, and they're so happy when it goes up, but it goes down at times. Why does it go down? They went down because bad luck, because it's random. Sometimes you choose the hard ones, but overall, you see a progress of going up and up and up, and so that's why it's not an inspection chart. It's a learning chart. It's showing a picture of my learning.

0:11:58.8 Andrew Stotz: And just to be clear, the first two charts we saw were looking at the overall classroom, but now the chart you're showing is one student mapping their progress throughout the quizzes.

0:12:11.7 Lee Jenkins: Yes, every student does their own, and if the teacher is scoring the papers to give them back to them, the results, they have to change, a slight change, instead of putting how many, they put a plus at how many correct, because you're graphing the number correct.

0:12:30.6 Andrew Stotz: Yep.

0:12:32.8 Lee Jenkins: And then another thing is kind of a celebration, a thank you, when students do better than ever before. So if a student had two right and then three right, and then they finally had five right, they never had five right all year long, they do something for the kid quickly to just say, yay, this child went and hit the gong. Just means I did more than, got more right than ever before. So what's the power of that? Dr. Deming wants every student to win. And I've been in classrooms six weeks after school started, maybe four or five in that time, and say, "Is there anybody in here who hasn't had a personal best? " I've never had a hand go up. They all have by then. Now, so you can be a struggling student, you can be an advanced student, but they all have a record of doing better than ever before, and we have ways of celebrating that.

0:13:32.4 Andrew Stotz: And that also is the idea of the objective really here is to improve ourselves relative to our prior selves.

0:13:43.7 Lee Jenkins: Yes, you're in competition with your prior self, that's it, yes. And I would say it's even 1% of the time that I saw somebody twist that and make it into a bribe. It's not a bribe, it's a thank you. I'm so proud of you, it's a thank you. It's a completely different mindset. They want to do that. And if we look at the next one...

0:14:09.8 Andrew Stotz: And just to understand this one last thing is that, are you saying that in a classroom when a student hits an all-time high, they go up and bang the gong or the teacher bangs it or what?

0:14:19.3 Lee Jenkins: No, the kid does it, the kid does it. Or whatever's done. One, you know that in sports where they make a tunnel and the athletes run through that tunnel of other athletes. There was a classroom that did that. The kids made a tunnel and the ones who had an all-time best that week ran through the tunnel. Okay? And there's...

0:14:41.0 Andrew Stotz: Yeah. And you could do simple things. You could also just say, if you did an all-time best, stand up.

0:14:46.6 Lee Jenkins: Yeah, it could be... But we try to make it something fun.

0:14:51.3 Andrew Stotz: Yep, yep.

0:14:52.3 Lee Jenkins: Something that's enjoyable for them. And it depends on the age. Here's one, another classroom, they wrote their name on a shape when they had a personal best. If you go to the next slide.

0:15:05.4 Andrew Stotz: Okay.

0:15:05.8 Lee Jenkins: You will see there's a collection of probably 200 shapes. With individual kids, they wrote their name on it when they had a personal best. And see, it's everybody. And it's a graphic in the hallway that lets all the other classrooms see, look how much we're learning.

0:15:29.9 Andrew Stotz: Right.

0:15:30.3 Lee Jenkins: Because every time you have a personal best, you put your name. This happens to be a star instead of a feather, but they put it up there.

0:15:36.7 Andrew Stotz: Yep.

0:15:39.1 Lee Jenkins: And then here's a middle school. When they have a personal best, they write their name on the whiteboard. And the kids have made kind of a Scrabble out of it, a crossword puzzle, where they can use the letters from somebody else's name to make their name. They love it. And they particularly like it because their friends who happen to be in that classroom but a different period, when they come in, they see their friends' names. Again, it's everybody. It's simple. Write your name on the whiteboard when you have a personal best. And then this is a high school. They had the game Kerplunk. And if anybody's not seen that, it's a cylinder. And it has holes. About halfway up it has a bunch of holes. And you put straws through the holes. And then you put marbles on top. When a kid has a personal best, they pull a straw out. When you pull enough straws out that all the marbles on top come crashing down, that's why they call it Kerplunk. And then the class does something for a couple minutes of fun. But it's everybody.

0:16:49.0 Andrew Stotz: Right.

0:16:49.8 Lee Jenkins: Then here is, they added the word all-time best. That was an addition.

0:16:57.7 Andrew Stotz: Right.

0:16:58.1 Lee Jenkins: And this is a class run chart, like I showed you last time, where you add up the total for the whole class. But when the class has more correct than ever before, it's an all-time best. We use that word for kids also, and you'll see in school that the initials ATB are very common in the schools.

0:17:22.1 Andrew Stotz: Right.

0:17:23.3 Lee Jenkins: It's one of the most common things. And you can't see it, but I'm looking at this when they had 28 quizzes in the year, and there are one, two, three, four, five, six, seven, eight times during the year out of 28 that the class had an all-time best. Also, if you look at the x-axis, it's 28. Dr. Deming said every week, and it was changed to 28 instead of every week.

0:18:03.6 Andrew Stotz: Right.

0:18:03.9 Lee Jenkins: That was a huge deal for me because I knew that every week was too much. There's snow days. There's things that happen, and you just... In the schools, it's too much going on for it to be every week. But I also knew that every other week's not enough. Not for kids to really prove that they're learning. Plus, they like them. They want...

0:18:29.6 Andrew Stotz: So, what does the 28 mean? Why 28?

0:18:33.5 Lee Jenkins: It's seven times a quarter instead of nine times a quarter. That's why.

0:18:37.1 Andrew Stotz: Okay.

0:18:38.3 Lee Jenkins: So, out of a quarter, two times they didn't. And actually, the complaint the kids had was, why aren't we doing one this week? And so, in a sub-sense, it's only for the teacher to just kind of a sense of... It just eases up a little bit. For the teacher, not for the kids.

0:18:55.8 Andrew Stotz: So, in other words, rather than strictly tying it to a week, you tie it to the number of quizzes that you're going to do, and then you manage that.

0:19:08.6 Lee Jenkins: Yeah, and I've never heard anybody say they couldn't get the 28 in. It's reasonable.

0:19:12.5 Andrew Stotz: Right.

0:19:13.1 Lee Jenkins: Yeah. It's a reasonable...

0:19:16.3 Andrew Stotz: Just for people that don't recall, like myself, I can't even remember what numbers of days in the classroom and numbers of weeks in a class and stuff like that, can you just remind me what that is?

0:19:29.6 Lee Jenkins: Okay, in a year, the school is divided into quarters, and there's 36 weeks in the year. So, there's nine weeks per quarter, and we're quizzing seven of those nine weeks.

0:19:42.8 Andrew Stotz: Perfect, okay, got it. Okay.

0:19:46.5 Lee Jenkins: Now, here is something else that has been added, and it is the goal. And so, Dr. Deming talks against numerical goals, and we agree with that. That goal is not an artificial number. It's the best from the prior year. So, it's a real number. So, the students are trying to outperform the prior years.

0:20:18.6 Andrew Stotz: So, this is the best that the system could produce in the past period?

0:20:23.8 Lee Jenkins: Yeah, are we smarter than the kids that you had the last several years? Are we smarter...

0:20:29.5 Andrew Stotz: Am I teaching better? Are you learning better?

0:20:33.5 Lee Jenkins: No, it's a challenge. It's a challenge, and they are so excited when they do better than the prior years. So, how did they get so high up there? Part of it is because there are kids who get, on the quizzes, they get perfect, perfect, perfect, perfect, and it's kind of boring for them. And so, we've come up with... When you get them all right seven times, it could be five, it could be six, we've usually gone with seven, then you don't take the quiz anymore in the room because you've proven you know it. And then we give you a harder one.

0:21:17.0 Andrew Stotz: Right.

0:21:18.3 Lee Jenkins: The class gets credit for the quiz you didn't take, plus how many you get on the next one. So, that helps it to go on up because you've got kids that are, the word we're using is they test out. They've proven they know it.

0:21:34.9 Andrew Stotz: Yep. Yep.

0:21:36.5 Lee Jenkins: We use the, when I talk with the teachers, the flip of the coin statistics. If a kid gets a perfect score, you have a 50% chance they're lucky, and a 50% chance they know all the content for the year.

0:21:49.7 Andrew Stotz: Yep.

0:21:49.9 Lee Jenkins: You don't know what it is. After seven times, you're up to 99% sure they really do know all of it.

0:21:56.3 Andrew Stotz: Right.

0:21:57.1 Lee Jenkins: Yeah. Oh, this day, this is a run chart from a middle school, and they had one more right than ever before. They are beyond happy. And you will see kids in the rooms doing a chest bump.

0:22:20.2 Andrew Stotz: Yep.

0:22:20.4 Lee Jenkins: A kid that's struggling, and says, it was me. I'm the one that put us over the top. If it hadn't been for my two questions right, we wouldn't all be celebrating. And of course, if you don't count it, you'd never know as a student or a teacher that you had your best. Nobody'd never know.

0:22:43.3 Andrew Stotz: Yeah. Yep.

0:22:44.0 Lee Jenkins: Count it out and graph it. Oh, they're so happy.

0:22:48.2 Andrew Stotz: Yeah.

0:22:48.5 Lee Jenkins: So that... And then here is a run chart by grade level. This is 16 classrooms together.

0:23:01.5 Andrew Stotz: What does that mean, 16 classrooms?

0:23:03.9 Lee Jenkins: There's four science classes, four English, four math, and four history. And we took all of those questions right from 16 rooms and calculated a percent correct.

0:23:19.2 Andrew Stotz: So in other words, how we're learning as a school or how we're learning all the subjects, how would you describe that?

0:23:25.9 Lee Jenkins: This was grade seven.

0:23:28.3 Andrew Stotz: Right.

0:23:28.9 Lee Jenkins: This was for the grade seven teachers. They wanted to have a total for their grade level.

0:23:35.5 Andrew Stotz: And so it starts off on quiz number one, that students got 16% correct. That's quiz number one.

0:23:46.7 Lee Jenkins: Right.

0:23:46.9 Andrew Stotz: Or quizzes number one.

0:23:50.7 Lee Jenkins: For quiz number one. Right. You can't say week one, it's quiz one.

0:23:53.2 Andrew Stotz: Yep, yep. Sorry.

0:23:53.8 Lee Jenkins: And this is for first semester, because there's 14 right there.

0:24:00.4 Andrew Stotz: Yep, yep. And then by the time they get to quiz number 13, that's, or quiz number 13 for all four subjects brought together into one measure, they're at, say, they've gone from 16 to 55.

0:24:14.5 Lee Jenkins: Yes. So you can say that at halfway through the year, the seventh grade class, 16 classrooms, but seventh graders know half of the content. And you know it's in their long-term memory. They couldn't study the night before.

0:24:31.9 Andrew Stotz: Right.

0:24:31.8 Lee Jenkins: Because you don't know what's going to be chosen at random. They know half of the content.

0:24:37.8 Andrew Stotz: And interesting that we see kind of a linear rise. I wonder if there's an exponential rise towards the end as the students get totally pumped up and into it and they're learning more.

0:24:47.8 Lee Jenkins: They are. They want to get as close as they can. It won't land on 100%.

0:24:54.0 Andrew Stotz: Yep.

0:24:55.0 Lee Jenkins: Somebody's going to miss something, but it gets really close.

0:24:57.7 Andrew Stotz: Yep.

0:25:01.3 Lee Jenkins: Now here's something else we've added. Because Dr. Deming talked about the classroom, this is a whole school. And they're all taking a math quiz. It's an elementary from kindergarten through fifth grade. On Thursday afternoon, the teachers go in to their computer on a Google Doc and they put in how many questions their classroom got right on the quiz that week. It's all set up in advance and there's a total. And then on Friday, the principal announces if they had an all school time best, all-time best for the school. And you can see...

0:25:45.8 Andrew Stotz: And the number here is 3878 I see in quiz number 28. Is that the total number of correct answers out of accumulating all the different quizzes of quiz number 8, all the different classes that do quiz number 28?

0:26:00.4 Lee Jenkins: Yes. On quiz 28, they answered 3,878 math questions correct.

0:26:06.2 Andrew Stotz: And somebody could look at this and say, "Oh, come on, kids are just going to game this, right? It's just quiz questions and all that." Now, I think I understand why that's not going to be the case. But how would you explain to somebody that says that?

0:26:21.4 Lee Jenkins: Hey, as the kids get older... Let's go back. This is math.

0:26:28.0 Andrew Stotz: Yep.

0:26:28.5 Lee Jenkins: So the concepts are the same, but the questions are different. So they can't game it. And other subjects where it's not math, teachers tell me that three different questions per concept is enough and they don't game it. They can't. But if you only had for every question for the year, I mean, for every concept, if you only had one question, they would game it. They just remember the answer to the question.

0:26:58.7 Andrew Stotz: Right.

0:26:58.9 Lee Jenkins: Yeah. As they get older.

0:27:00.7 Andrew Stotz: And what would you say to some people that may look at that and say, "Oh, you're just teaching to the quiz or teaching to the exam? "

0:27:13.3 Lee Jenkins: Well, we're saying, here's what you're going to learn this year. University professors give out syllabuses. A syllabus is what you're going to teach, which is different from stating this is what the kids are going to learn. And so when you list what you want them to learn, this is evidence they learned it. Now, yes, we're teaching to what we said we want them to know. It didn't come... When you teach to the test, that often means that somebody else made up the test that I've got to teach to that test they made up because there's high stakes.

0:27:55.3 Andrew Stotz: Right.

0:27:55.6 Lee Jenkins: But when we as faculty say what we want the kids to know, we're not teaching to the test, we're teaching to what we said we want them to know.

0:28:05.5 Andrew Stotz: Yep. Yep. And then the other thing I would say is when you get students so deeply involved in the whole process, ultimately young kids actually are not going to necessarily celebrate cheating.

0:28:22.8 Lee Jenkins: No, no, they're not.

0:28:25.5 Andrew Stotz: They understand right and wrong. They haven't gotten to the level where adults are, where we put a lot of gray area between right and wrong and politicians will lie about this and that to get in office or get money or whatever.

0:28:37.4 Lee Jenkins: Let me tell you a story about the cheating. There were three fourth grades in a row in a school. And in the middle between the other two fourth grades, they did cheat early in the year. They got a very high score. Then the teacher found out how they cheated and stopped it so they couldn't do it anymore. But they couldn't get classroom best because they had an artificial high score. So they're saying to her, "We cheated teacher, take it away that score that we cheated." She says, "No, you cheated." It took them till November before they could have a classroom best.

0:29:16.7 Andrew Stotz: Right.

0:29:17.4 Lee Jenkins: So they paid a price for it. Now, people have fun with random. This is out of the state of Delaware. It looks like a skeleton from Halloween and they spray painted lima beans, put them inside the skull, wrote numerals on them and you draw the numerals out and that's the concept you're going to quiz. So there's been fun with how you do random, fun with how we celebrate.

0:29:55.0 Andrew Stotz: Right.

0:29:55.2 Lee Jenkins: Fun with making the graph pretty with I get to put Google... I mean, I get to scribble on it and do different things that make it pretty. Yeah. And here is a... There is a styrofoam nose. I'd say it's a meter tall styrofoam nose. And the teacher had slips of paper with the concepts on them. And an eighth grader said, that is boring. Brought in a styrofoam nose and you put the slips up the nostril and that's where you pull out...

0:30:26.2 Andrew Stotz: Only kids are going to come up with that.

0:30:28.1 Lee Jenkins: Yes, I know. And this is a history teacher, world history. She has 65 concepts are going to learn during the year. She gave them the list, put the 65 on a tongue depressor, put them in a bucket. She pulls eight out each week and the kids have to put the eight in chronological order from memory.

0:30:52.3 Andrew Stotz: Right. That's interesting.

0:30:53.5 Lee Jenkins: But they can't do it in the beginning.

0:30:55.1 Andrew Stotz: Yep.

0:30:56.0 Lee Jenkins: But by the end of the year, you want every kid to be able to pull any eight you pull out and put them in chronological order, not because they know dates, but because they know history.

0:31:06.2 Andrew Stotz: Yeah. Yep.

0:31:10.1 Lee Jenkins: And then here from Saskatchewan is a teacher who hyperlinked the periodic table. It's up on the whiteboard. So in the bucket are the names of elements. So if a student pulls out the word potassium, they go up to the whiteboard and they click on the letter K. It's hyperlinked. When they click on it, up comes a question about biology. The question has nothing to do with potassium.

0:31:42.6 Andrew Stotz: Oh!

0:31:44.4 Lee Jenkins: It's just a clever way to do random.

0:31:48.6 Andrew Stotz: Yep. Yep.

0:31:50.1 Lee Jenkins: Okay. And then we celebrate as a whole class. This is a class that's celebrating doing the wave. They've been to athletic events. They've seen people do the wave at athletic events. When the class has an all-time best as a class, they do something quick to celebrate. They're doing the wave. This classroom, they have a spinner. And the kids chose 10 ways they wanted to celebrate. I said, "What's your favorite? " And they said, "Hamster ball." I said, "What's a hamster ball? " They said, "We've got a hamster in the room. We put it in a hamster ball, put it in the middle of the room and watch where it goes."

0:32:32.7 Andrew Stotz: Yep.

0:32:34.4 Lee Jenkins: Fun. This is the whole school again. Just celebrating. One principal, when the school had an all-time best, somebody came in and cut his tie off. And he had dads giving him all their old ties to cut off. Yeah. And then they like to do item analysis. That's kids doing that.

0:32:59.0 Andrew Stotz: Right.

0:33:00.1 Lee Jenkins: They like to tell you what it is we most need help with.

0:33:04.2 Andrew Stotz: So this is looking at errors to say what we're struggling with. What does that mean?

0:33:07.7 Lee Jenkins: Yeah, here's the most room, most missed item in the whole room, all the way to the right, the item that nobody in the room missed it.

0:33:15.8 Andrew Stotz: Yeah, like allusion. I'd miss that too.

0:33:21.1 Lee Jenkins: And then we made histograms. So it's taking the data from the scattered diagram and putting a different one together for each week. So the kids see an L-shaped curve in the beginning, a bell curve in the middle of the year, and a J-shape at the end of the year. And this was taken because they were so excited that they could see the J finally. They knew the J was coming, and there it was.

0:33:47.8 Andrew Stotz: Yep.

0:33:49.7 Lee Jenkins: Yeah. And then we used the information from the scatter diagram to calculate effect size and to see what's the effect of all of this compared to all the other things in the world that have been done. And we got six times the average of the effect size research from John Hattie. If you don't cram and forget, you actually just remember, of course, it's a lot higher. Duh, of course.

0:34:15.5 Andrew Stotz: Yep.

0:34:18.0 Lee Jenkins: And we did the scatter diagram that I showed earlier, we mentioned earlier, that's what we use. And when John Hattie saw the scatter diagram, he said, "That's what you need for effect size."

0:34:29.6 Andrew Stotz: Right.

0:34:30.3 Lee Jenkins: Because effect size is you increase the mean and you reduce the variation. I've been talking a lot about knowledge. I haven't been talking about skills. The same process works for skills. And this is the dichotomous rubric. It's on my website. It's blank. It's free. And we use the dichotomous rubric to measure skills.

0:34:53.4 Andrew Stotz: Yep.

0:34:56.6 Lee Jenkins: So this is my pastor. It was, school was starting, he called two kids up on the platform and he said, "What are you excited about school? School started. What are you excited about? " The girl says, "See my friends." And the boy said, "Quizzes."

0:35:09.3 Andrew Stotz: Yeah, making my charts, seeing the quiz, watching the progress.

0:35:13.3 Lee Jenkins: It's hard to believe, but that's exactly what happens. And there's the Jenkins curve, which is the loss of enthusiasm year by year through the grades. I would have never done this without Deming because he talked about graphs have to be long and skinny.

0:35:29.3 Andrew Stotz: Man, we just grind down the kids in a normal situation.

0:35:32.9 Lee Jenkins: Just grind them down.

0:35:34.4 Andrew Stotz: Yep.

0:35:34.7 Lee Jenkins: Every year, fewer and fewer kids love school.

0:35:37.3 Andrew Stotz: Yep.

0:35:39.9 Lee Jenkins: So podcast number three, when it comes up, will be the future. What can we do because of all this that we haven't done before? It'll be fun.

0:35:51.2 Andrew Stotz: Wow! That is a lot of stuff. If you were to take all that we just went through, which was really fun and exciting, what would be the one takeaway you want people to get from that?

0:36:04.2 Lee Jenkins: The takeaway is that we can keep the intrinsic motivation alive that children were born with. And when we keep it alive, the complaint in the staff room will be, I can't keep up with all these things that these kids want to learn.

0:36:22.3 Andrew Stotz: Yeah.

0:36:23.7 Lee Jenkins: Instead of complaining that they won't sit still, they won't do the work, we'd be saying, "I can't keep up. They want to learn so much. I'm overloaded with what they want to know."

0:36:32.7 Andrew Stotz: And the end result is they become lifelong learners.

0:36:38.0 Lee Jenkins: Yes, absolutely. Yeah.

0:36:38.2 Andrew Stotz: Yeah.

0:36:38.9 Lee Jenkins: Yeah.

0:36:39.2 Andrew Stotz: I'm going to wrap it up there. And Lee, on behalf of everyone at the Deming Institute, I want to thank you again for this discussion. It was fascinating and it was fun. So for listeners, remember to go to Deming.org to continue your journey. And this is your host, Andrew Stotz, and I'll leave you with one of my favorite quotes from Dr. Deming that ties directly in to what we've been talking about, and that is, people are entitled to joy in work. And I'm going to add in, learning.