The Cult of Pedagogy Podcast, Episode 236
Jennifer Gonzalez, Host
GONZALEZ: Any time we teach our students something, we need to check to see how well they learned it. The most common way to do this is with some kind of a test, which traditionally happens at the end of a unit — a final exam of some kind. But if we only do this check at the very end, after all the teaching is done, and we find that our students haven’t learned the material quite well enough, it’s too late to do anything about it. That’s why we really need to be checking as we go, so if there are problems, we can fix them.
We call these two kinds of testing summative assessment, the kind that happens at the end, and formative assessment, the kind that happens during the learning cycle. In today’s episode, we’re going to focus on formative assessment — specifically, five things we need to do to make sure we get it right, because if we do, we can dramatically improve our students’ learning.
Joining me are Kim Marshall and Jenn David-Lang, two people who have put in some serious time figuring out what works in the classroom. Kim Marshall is the force behind the highly regarded Marshall Memo, a weekly, subscription-only summary of 8 to 10 of the most impactful articles in education. Since 2003, he has published 50 issues of the Memo every year, and countless educators rely on his summaries to stay on top of what’s happening in education. Jenn David-Lang, meanwhile, started a similar project in 2007: She began publishing The Main Idea, a single, detailed summary every month of an important education book to help education leaders connect to the most important ideas on leadership, teaching, and learning. A few years ago, the two joined forces to publish The Best of the Marshall Memo, volume 1 and 2, a “super-curation” of the very best Memo summaries. Since then, supported by a foundation grant, they have made these summaries available for free on their website, bestofmarshallmemo.org. The collection is sorted into 24 categories, one of which is assessment. The five conditions for formative assessment that we talk about today come straight from that collection.
Before we get started, I’d like to thank Scholastic Magazines + for sponsoring this episode. What if we told you there’s a way to delight your students AND take the pressure out of lesson planning? Scholastic Magazines+ delivers with captivating content for kids and time-saving lesson plans for you. Every single story, video, and activity is crafted to engage your learners and complement your curriculum. Throughout the year, your class receives all-new stories in the format of your choice: print, digital, or both. Your students will love reading relevant texts, watching real-life videos, and connecting lesson topics to their own lives. You’ll love that every issue builds essential knowledge and skills for evidence-based instruction. Visit scholastic.com/magazines today to learn more, and use code 6789 to save 40% on the 2024-2025 subscription of your choice. Offer expires January 31st.
Support also comes from Listenwise and their new product: Lingolift. Designed for newcomers and beginning-level English learners in grades 2-12, Lingolift builds the foundational social and academic language that newcomers need. Their large collection of short educational videos teaches content and language together. The unique “Watch-Learn-Speak” instructional sequence and teacher’s guides offer a powerful yet flexible tool for building newcomer skills as quickly as possible. Now Listenwise has something for students at every level of English language acquisition! Try out Lingolift today when you start a free trial at listenwise.com.
Now here’s my conversation with Jenn David-Lang and Kim Marshall about the five conditions we need to put in place to get formative assessment right.
GONZALEZ: Kim and Jenn, welcome to the podcast.
DAVID-LANG: Thank you, good to be here.
MARSHALL: Great to be here.
GONZALEZ: So we are going to be talking about formative assessment, but before we do, I would just love it if the two of you could just take turns just giving us a quick bite-sized overview of what you do in education and also how you work together. So we will start with Kim.
MARSHALL: So former Boston principal, frustrated that I couldn’t keep up with the reading. Fifteen years of struggling to recap Ed Leadership and Ed Week, and all the good stuff and, and sometimes on weekends and vacations. And then when I left the principalship in 2002, I had this idea of maybe I could be the designated reader. Maybe now that I had some time, I could actually read stuff and find the best material and then put it out in a newsletter. And I got support from two people to get me started, Jon Saphier and Jon Schnur, and started off on a once-a-week routine. I was, you know, questioned how often should it be and how long should it be and how should I get the word out and gradually it’s grown from about 300 subscribers to this kind of international phenomenon where people, you know, find it helpful that I spend about 20 hours a week on Sunday and Monday reading and writing précis, and they can read it in 20 minutes. And then they’re empowered to send it out to their colleagues and get people thinking about mathematics ideas or, or literacy ideas or time management ideas or whatever. So that’s the sort of 21-year history, this week’s is 1,055, you know, Marshall Memo. And so it’s kind of a regular routine. My wife gives me grief for, you know, why 50 issues a year? Like, what’s up with that? But it’s a nice round number. And then, so then you know Jenn and I got together and had this idea of what if we could pluck out of the archive, you know, the thousands and thousands, overwhelming number of articles now in the archive, pull out and just in originally 22 topics. What if we could find the best of the best? And we did that. Took two years. It was sort of our pandemic project. And we published two books and they’re great except they’re big and clunky and not very practical for principals. And so then we had outreach from the Gates Foundation and we ultimately, after a lot of back and forth, they supported putting all of the content, all 900 pages on a website. And so The Best of Marshall Memo website with Jenn’s great tactical help with me and wisdom and pushing me and helping to refine the sequence, and the order, and the, it was really quite difficult to select, you know. Just the 12 or 13 or so articles in each of these topics like, you know, time management, differentiation, race and equity, and so forth to find the very best of the best and then organize them well. And then Jenn did a PD section on each one. So that’s all on this website, bestofmarshallmemo.org, and it’s, it’s quite heavily used. So that’s the —
GONZALEZ: Yes, it is.
MARSHALL: — the very brief summary. Meanwhile, I’m out in schools and giving talks and, you know, around the country and so Sunday, Monday is The Marshall Memo time but then I’m out keeping myself honest about what’s really going on in schools.
GONZALEZ: Awesome. A lot of that sounds really familiar. That last part too about keeping yourself honest, because I try to do the same thing to stay plugged into what’s actually going on in schools. And Jenn and I were talking before we got on with you about how much the three of us actually have in common. Because I had this same frustration as a teacher with never being able to keep up with best practices and research. And I kind of consider myself to be doing kind of the same thing, which is figuring out what works and trying to condense it so that teachers in the classroom can make good use of it. So I think it’s great that we’re all coming together now.
MARSHALL: And you do an amazing job.
GONZALEZ: Oh thank you.
MARSHALL: I told you in my, in my qualitative, qualitative review of all the memo articles, Cult of Pedagogy is No. 1.
GONZALEZ: That was, that was a really cool statistic to see.
MARSHALL: Pound for pound, you’ve been.
GONZALEZ: So Jenn, Jenn David-Lang. Tell us a little bit about what you do in education and how you’ve come into this picture with Kim.
DAVID-LANG: Yeah, it’s been great to work with Kim. I started out many years ago teaching in that special place we call middle school. Oddly, I taught both English and math. Oddly, I love the eighth graders.
GONZALEZ: Love me some eighth graders too, yes.
DAVID-LANG: You do? Okay. I love them. All right. I’m not alone. Jennifers Who Love Eighth Graders. And then years later, after getting my principal’s license, actually I interned with a principal who was a huge reader. And then all the other leaders I met just didn’t have time for professional learning.
GONZALEZ: Yeah.
DAVID-LANG: And I saw that in medicine and law, people had abstracts, but school leaders were not reading books, they didn’t have time. So I hadn’t heard of The Marshall Memo, but I started this idea, a subscription service called The Main Idea where I summarized one education or leadership book a month. And then at the end, I included PD ideas so the leader could introduce the ideas from the book to their school. So the book was about classroom management or formative assessment, then they could run PD in their schools. Because what’s the point of just reading to read. You want to read to make your school better.
GONZALEZ: Yes.
DAVID-LANG: So yeah, I found my superpower is more in supporting other leaders through The Main Idea. And then I also run online school master, school leader and district leader Masterminds where we meet for one hour every other week to do professional learning, coaching, resource sharing, strategy sharing, and that has been great too. And the partnership with Kim has been really great. Because yes, he summarizes the books, I summarize the articles, and so we try to provide a full service for people. And yeah and combing through his thousands and thousands of articles to find the best articles on the topics that leaders care most about, and teachers too, we’ve really been able to pull out what is important and what people should be reading. And now it’s free for everybody on the website too.
GONZALEZ: Yeah. So, and so that’s kind of how we all came together just because we’ve, you know, been in each other’s orbits for a while, but we thought, let’s have you on the podcast and pull something sort of from what you all have offered. So this is, you’ve condensed everything in The Marshall Memo down to this stuff on this site, and so we’re now going to just pick further condensation and just pick one topic. And we decided that we are going to talk about formative assessment today, because you cover a lot of different topics and what works best in all of these different areas. So what we decided to do, because I said, you know, it’s got to be something I haven’t already talked about on my site. And, so what, what you all are going to share with us today are five conditions that are really necessary for formative assessment to work really well, for it to go well. And just to make sure everybody listening is on the same page, formative assessment is the assessment during instruction. So it’s not the test at the end, it’s the stuff that’s happening while you’re teaching to make sure that the kids are getting it. So we’ve got five conditions, five things that, that will really help formative assessment go well. So let’s just get started. Let’s talk about what is the first thing teachers need to be doing to make formative assessment work well.
DAVID-LANG: I can jump in with the first one, if that’s okay, Kim.
MARSHALL: Yeah, yeah.
DAVID-LANG: Yeah? The first one I think is incredibly important and that is you need to be responsive when students haven’t learned and take next steps. So certainly in my time, you know, we would teach, test, move on. Now, at least teachers are more aware, you’re supposed to do a formative assessment, an exit ticket, a check for understanding. But then what, right? So this piece is incredibly important that you need to do the formative assessment and then respond. And, and I think it’s actually something you need to write into the lesson plan, right? So if you do a formative assessment and 30, 40, 50, 80 percent of your kids don’t understand the concept you are trying to teach, then what? You can’t just say, “I taught it and then it was learned.” It was not learned. So sure, you could be on your feet and figuring it out, but it’s better to have it planned ahead of time. So if the students don’t understand, will you have them turn and talk and convince each other, right? Will you reteach to a small section of the class while the rest are working independently? Will you have planned stations for all the different ways kids might have misunderstood, right? Imagine Station A for kids who misunderstood it this way, Station B for kids who misunderstood it that way. But the idea is that it’s, it’s a real-time instructional adaptation. It’s not just plowing through regardless of the results of what you find out from the formative assessment. Craig Barton, who is one of the authors in the chapter, he’s a teacher, he says, “Teaching without formative assessment is like painting with your eyes closed.” But I would amend it a little bit in saying teaching without — did I say with? — teaching without formative assessment. I would say teaching without responding to the results of formative assessment is like painting with your eyes closed, right? So it’s like, ignoring what is in front of you. So, so yeah, so the idea is to have a Plan B for if a certain segment of the, of your class does not understand.
MARSHALL: And that, in other words, certain segment there brings up a key equity idea. Grant Wiggins has a quote. I may not have it exactly correct here, which is, “The more you teach without seeing who understands, the more only already proficient students will be successful.” That, that really brings out the, you know, the gap widening quality of teaching, testing, and moving on. Or as Grant used to say, “Teach, test, and hope for the best.”
GONZALEZ: Yes, yeah. And you know, this is something I bring up a lot in professional development too. I, when I worked with a group of student teachers years ago, I remember one student who had a plan to give quizzes, and then he told me, this is partway through, he said, “Well I’m going to just file them, and I’ll wait, and I’ll grade everything at the end.” I said, no, you have to, that’s for something. You have to do something with that. And so it kind of surprised me that that wasn’t already obvious. So I think it’s a good idea to state that this is important.
MARSHALL: But I think also to talk about the difficulty of this. I mean if a teacher has done a really good job teaching something, they, and they have the curse of knowledge, you know, they know it and, and they don’t quite understand why the kids — I actually watched a teacher do a, use Plickers, which are pretty cool. You know, you, every kid has a QR code. The class responded. And a lot of kids got it wrong, and the teacher literally said, “What were you thinking?”
GONZALEZ: Oh.
DAVID-LANG: Yeah, I also want to add, there’s a misconception that I think all of us have that this is going to take more time, right? You have to plan it into your lesson. It’s taking time out of your lesson to do the formative assessment.
GONZALEZ: Yeah.
DAVID-LANG: But it’s so much more time later on if you moved forward and kids weren’t understanding. So yes, does this take time to think through your lesson plan and plan for a Plan B? Absolutely. Does it take time out of the actual lesson? Yes. But it’s, it’s, it’s going to be a much higher cost later on if you don’t.
GONZALEZ: Yeah.
MARSHALL: Well the biggest cost of all is the kids didn’t get it. They failed the final test or the final performance test. They don’t know how to do it, and you’re thinking, my gosh, what happened here?
GONZALEZ: Yeah.
MARSHALL: There were signals all along. But then I also, I was just in, in this school for two days, and watched eight classes, co-observed eight classes, processed them. And I saw, you know, some evidence of a teacher who would, who would teach something and say, “Any questions?” And that was the formative assessment. But it’s not, or even thumbs up, thumbs down. I actually did not see that in this school, but I often see that. And, you know, it’s just, it’s, it’s an imperfect formative assessment. And so the quality of the assessment as well as Jenn saying, the quality of the feedback, you know, the follow-up. Like, what do you do next?
GONZALEZ: Yes.
DAVID-LANG: But you’re providing the segue, Kim, to the second bullet point.
GONZALEZ: Yeah. I was going to say, we’re going to, we’re going to actually get into some, some, some ideas for how you actually check into it, because that question — “Are there any questions?” — that relies on your kids not only being confident enough to speak up when they don’t understand something, but even knowing what question to even ask if it’s on them. I did a whole episode with Connie Hamilton about the idea of kids saying, “I don’t know.” And when they say “I don’t know” they actually mean maybe one of 12 different things. They just don’t know how to articulate it. And if you can teach them even how to ask, but that still puts the onus on them. So let’s get to No. 2.
MARSHALL: Mhmm. So No. 2 is the system, and I’ve been fascinated with this ever since I first stumbled upon clickers. I got the, got the first generation of Turning Technologies’ physical clickers, which of course then became impossible in the pandemic, but used them for years. But, but the thing is you’ve got to know how, what everyone thinks.
GONZALEZ: Yes.
MARSHALL: And, you know, popsicle sticks, random calling, okay, that’s a step in the right direction, at least people are on their toes. But that’s not really a formative assessment of the whole class. So Dylan Wiliam is spelled out, and he’s our hero. In fact, the dilemma in putting together this chapter was not having too many articles by Dylan Wiliam because he’s had so much great writing about this. Especially, you know, the classic article “Inside the Black Box,” which, you know, the title does sort of capture that idea. There’s this black box in kids’ heads, and we’ve got to find out what’s going on in their heads. So what’s an efficient way of doing this? So the highest tech is Plickers, although that does limit you usually to a four-part response, you know.
GONZALEZ: Right.
MARSHALL: Of like four possible things or actually 10 now. The Turning Technologies things had 10. But a cell phone, if you’re texting, you can do even more, and you could do word splashes. You can have kids texting in, or excuse me, or yeah, but in this case kids. So but, but the criteria are how can you hear from every kid? How can you quickly see the results? How can the teacher quickly process that? And then how can you then have, as we said a minute ago, a wise and thoughtful response to that? Turn and talk, convince your neighbor, get into groups, you know, reteach it or whatever. Reteach it louder and slower, that’s the best technique, right? So, but I, I think it doesn’t have to be high tech. You know, I’ve used Plickers. During the pandemic, I shifted to texting, cell phone texting, and that’s worked well. But it can be very low tech. I once was in a webinar with, or excuse me, now actually an in-person thing with a high school chemistry teacher. And he described, okay, not thumbs up, thumbs down. That’s too public. But what about signaling on your chest, you know? Four responses and I signal, he can scan the class, and he can see, okay, three quarters of the kids have this. We can, we can move on with a quick explanation. Or, oh my gosh, you know, three quarters of them don’t have it.
GONZALEZ: Yeah.
MARSHALL: We need to, we need to fix that right now. And fixing it right now is the key thing. But I think those four criteria, I think this came from Dylan Wiliam of, of a good so-called hinge point question. You know, as Jenn said, in the lesson plan, it’s actually in the lesson plan, what’s the fraction between one-sixth and one-seventh? And, and then time for the kids to process it, time for them to all respond, and also, another criterion is it can’t be a question that they can get right for the wrong reasons. So, so it isn’t just on the fly. It’s, you know, the question is carefully planned, ideally, so that you really get information. You know, how long does it take the Earth to go around the sun? You know, and, and, but anyway, and all the possible misconceptions built into the possible answers. So, you know, I think, but that’s, that is the key thing is well-planned and high tech or low tech.
GONZALEZ: When you were talking about the, putting something on your chest instead of a thumbs up, thumbs down, and you had, you were holding on the video like a four or a three. So what you’re saying is actually ask a multiple-choice question.
MARSHALL: Right.
GONZALEZ: Have them answer privately. Because I think the thing about the thumbs up, thumbs down thing is that there will still be kids that will hide the fact that they don’t get it, and they might be like —
MARSHALL: Right.
GONZALEZ: — yep, I get it. Don’t pay attention to me. But with a forced choice answer —
MARSHALL: Right.
GONZALEZ: — you can actually get a diagnostic there.
MARSHALL: Also they, they might think they have the right answer, sincerely believe that they have the right answer and put their thumb up —
GONZALEZ: Yes, right.
MARSHALL: — but in fact they, they really don’t.
GONZALEZ: Okay.
DAVID-LANG: What I emphasize also because, you know, the idea is that we want to check all students’ understanding and not only is that good for formative assessment but you’re also getting all the students engaged in the lesson. And if you do use some type of technology, then it’s anonymous when kids get it wrong. And so that helps prevent students from embarrassment and embed the idea that it’s okay, right. Look at all these other kids who got it wrong too. So I think there’s two other elements about assessing all of the students as opposed to our typical, right, you know, any questions? Who’s got it?
GONZALEZ: Right.
MARSHALL: And speaking, speaking of technology, you’ve probably seen, excuse me, the classic video, My Favorite No by California middle school teacher teaching math. And she actually has a little slap at, at clickers. She says they’re so expensive. Our school couldn’t afford them. But she has kids write on a card the answer to it — what is a quadratic equation solution? — I think. Collects the cards, very quickly goes through them.
GONZALEZ: Right.
MARSHALL: Right in front of the kids, picks her favorite wrong answer.
GONZALEZ: Yes.
MARSHALL: But then very interestingly, speaking of an anonymity issue, she recopies it so that the kid’s handwriting or name is not visible to the rest of the kids.
DAVID-LANG: She projects it onto a screen, yeah.
MARSHALL: A document camera, which I think, I think Doug Lemov says document cameras is the single best piece of technology in the last 20 years. Because you can actually see student writing, or in this case the teacher’s writing. But then, you know, here’s, here’s the wrong answer, but what’s right about this, I mean this whole, Jennifer, what is her name, gosh, I’m blanking on her name. Anyway, My Favorite No, but it’s all over the internet, and I’ve used it frequently.
GONZALEZ: Yeah. And just reviewing that and really exploring why it was incorrect.
MARSHALL: Right, yeah. But first what’s right. You know, what do I like about this, she says.
GONZALEZ: Yes.
MARSHALL: You know, and calls on kids. And then finally she says what’s wrong, and you can see the class, and not every kid has got their hand up, but she says the kids who got it wrong, she says, oh that’s my mistake. And they can begin. But she does this at the beginning of every class, so it’s a regular routine, you know, in just first getting kids ready for the lesson but also checking in. What’s going on here? Do they understand this? It takes a lot of good classroom management to do that.
GONZALEZ: Yeah. It kind of normalizes that we’re all going to, you know, misunderstand things and have these little quirks where it’s like, if you just change that one thing, then now it’s breaking up the misunderstanding, and now you got it. And so it’s not this terrible thing to have gotten it wrong. It’s just part of the learning process.
DAVID-LANG: It’s a low stakes formative assessment, right.
GONZALEZ: Yeah.
DAVID-LANG: There’s no grade going into any grade book, right? It’s just for the purpose of learning. And I think by her picking that favorite no, she’s saying, actually, making errors is actually on the path toward learning.
GONZALEZ: Yes.
DAVID-LANG: And by seeing this error, helps us learn.
GONZALEZ: Yeah.
MARSHALL: When I show this video, and I use it frequently, you know, when I do workshops on formative assessment, you know, the immediate pushback is how can you spend 5, 10 minutes at the beginning of every class with this? How are you going to cover the curriculum? But back to Jenn’s point, you know, now even by slowing down, you’re speeding up and you’re greatly increasing the quality of instruction because you’re tuned in to what is actually going on, you know, that mystery of what’s going on in kids’ minds. In some of the lessons I was watching in the last two days, you know, just eight different classroom observations, I was looking at the kids’ faces, and I really just didn’t know whether they were getting it or not. And that’s the mystery. That’s the black box.
GONZALEZ: Interesting. I used to use facial expressions all the time as my formative assessment. I could just see on their faces whether they were kind of like, eh. I thought, you don’t look comfortable. There’s something off here. I don’t know what it is yet.
MARSHALL: Well you had the spidey sense, right?
GONZALEZ: I don’t know. And, you know, who knows what’s happened in the last few years. It could be that they’ve gotten better at hiding it.
DAVID-LANG: Sure. Or they’ve got masks, yeah.
GONZALEZ: Yeah. But that was actually the frustration of teaching. I taught a remote class at college where they’ll have the different campuses. And so their faces were super tiny. It wasn’t even like Zoom. And that was part of the frustration. I couldn’t read their facial expressions anymore, and it really got in the way of even the relationship building and everything.
DAVID-LANG: Interesting.
GONZALEZ: Yeah. All right. So we’ve got, I’m going to review what we’ve done so far. We want to make sure No. 1 is be responsive when students haven’t learned and take some steps. Two is choose a questioning system that checks all students’ understanding, and we talked about a couple of tech options. I’m a big fan too of just Google Forms, making up a quiz and having kids take it real quick on a Google Form. And you can get it right away. You know, that information too. There’s obviously paper exit tickets. There’s, some teachers have whiteboards or even made-up whiteboards with sheet covers and the piece of white paper and they can do dry erase markers and the whole thing.
DAVID-LANG: That’s good for when it’s not multiple choice, right?
GONZALEZ: Yes.
DAVID-LANG: You have to write an equation, steps.
GONZALEZ: Write an actual answer.
DAVID-LANG: Or write a sentence using a simile or something, right. You can’t do that with multiple choice.
GONZALEZ: Yeah. And then we’ve also got apps like Formative which allows teachers to look on their screen and see what every kid is doing on their own computer screen. And so, lots of systems.
MARSHALL: And one of the coolest moves, I have a slide of a high school class with individual dry erase boards. And the kids have done a graph, and they’re holding it up. And you can see that some of them, some answers are different from others. And the teacher’s next move is find someone who disagrees and discuss and reach a consensus. So that’s, that’s more than a turn and talk. That’s, that’s really.
GONZALEZ: Yeah.
MARSHALL: And you can imagine, you know, the chaos in the class as kids mill around and, you know, okay, then begin to — that’s also similar to Eric Mazur’s thing, convince your neighbor.
GONZALEZ: Yeah.
MARSHALL: You’ve got one answer, they’ve got another one. You know, you just taught this, and it would be, if a principal walked in at that moment who liked law and order, it might look a little chaotic, a lot of conversations and almost arguments. But that’s a brilliant instructional move.
GONZALEZ: Oh, yes, it is. Okay, so our third thing, and you had mentioned this a little bit asking the right question, about hinge point questions.
DAVID-LANG: Yes.
GONZALEZ: Let’s talk a little bit about what that looks like and how we do this well.
DAVID-LANG: Well they have different names, because there’s, some people call it a hinge point question, diagnostic, on-the-spot, check for understanding. But the idea is that, you know, the teacher thinks ahead of time, what’s the concept that I want to make sure students are understanding, and I’m going to ask that question, that hinge point question that my lesson, the rest of my lesson is going to hinge upon, right? And in, you know, one of the articles we summarized Craig Barton. He says we used to all, we learned in teacher school that open-ended questions are best, right? Why did the Civil War happen? How do you know which is bigger: Three-sevenths or four-elevenths? Right? That’s what we were all taught. But in fact, you actually, it actually helps to have a closed question if you really want to check for understanding. And so he says the ideal is a multiple-choice question where each incorrect answer represents a misunderstanding of the learning, you know. So a couple of simple examples like you could have: The apostrophe is used for a) to separate items in a list, b) when you want to introduce a quotation, c) when you want to shorten words like do not to don’t, d) when you want to show possession, that something belongs to someone or something, right. So let’s say 90 percent of your class says it’s to separate items in a list. Oh, I didn’t distinguish between commas and apostrophes.
GONZALEZ: Right.
DAVID-LANG: Like, then you can tell as the teacher what is the misunderstanding. And so, you know, he says, you don’t want like yes/no because you guess, right? But he really wants multiple choice where the wrong answers reveal what it is that kids don’t get, right.
GONZALEZ: Yes.
DAVID-LANG: You can imagine that also, right, in math, adding numbers with different denominators, two-thirds and four-sixths. So one of the answers would be six-ninths, oh, they’re adding the numerator, the denominator. That’s the misunderstanding.
GONZALEZ: Right. So instead of just random wrong answers, they need to actually be likely guesses under certain conditions.
DAVID-LANG: Exactly. And they’re actually really hard to write, you know.
GONZALEZ: Yeah.
DAVID-LANG: Several of the authors, a lot of them said — Dylan Wiliam, others — say it’s really hard to write them because you really have to, as the teacher, distill your own thinking. What’s the essential content and skills here? And that you really want to build that into the lesson along with that, you know, the Plan B that we’ve been talking about, whether it’s turn and talk or find someone who disagrees.
GONZALEZ: Yeah.
DAVID-LANG: So yeah. Hinge questions are, are quite difficult. That would be a great exercise for, to do at a professional learning session. Have teachers come together who are teaching the grade, same grade, same content, take out your lesson plan from tomorrow, and let’s talk about what would the hinge question be and how would we phrase that?
GONZALEZ: Yeah, yeah.
MARSHALL: And quality of questions is all important. I mean, I do hear a lot of simple, you know, just way, just low-level factual recall kind of questions, which might be important to check in on, but it’s really much more of a, you know, which of these fractions is the largest with different denominators. And the misconception about a larger denominator and so forth. So I think, and I think that planning it beforehand, making it part of the lesson plan. Now, sometimes you might come up with a great hinge point question right on the spot. By the way, I believe that’s a Dylan Wiliam term or at least a British term, because I know, I said let’s not give Dylan all the credit here, but he’s the one who first mentioned it in an article.
GONZALEZ: Okay.
MARSHALL: Yeah, the quality of questioning and then, of course, the follow-up, but we’ve talked about that. That’s the, that’s the deer in the headlights moment of oh my gosh, what do I do now?
GONZALEZ: Yes. Okay, so No. 4 you have is keep the check for understanding brief and frequent.
MARSHALL: Well, again, the misconception that you’re going to not cover the curriculum, and it’s going to slow you down, but, but the brevity and the frequency. I mean, so, so Eric Mazur is another hero in this whole thing, the Harvard physics professor who, you know, discovered that lecturing wasn’t working. And again, an equity dimension here with his Harvard physics classes of 180 kids discovering that No. 1, that they weren’t understanding the conceptual part. He gave a conceptual test and, you know, the kids who could plug and chug and be brilliant at the numbers, you know, didn’t understand the concept of the collision between a truck and a small car, something like that. So No. 1, they weren’t getting the concept, but No. 2, the female students in his Harvard classes were doing less well. And there was a whole sort of gender thing going on here where women were scoring lower even though they had the same entering, you know, power in their brains. And then they weren’t following up in other STEM classes at Harvard. And so he began to do this thing of frequent, several times during this 50-minute Harvard lecture, he would stop, ask a clicker questions with four possible, and very clever questions, you know, that really went to the concept, look at the responses and if between 30 and 70 percent got it wrong, then convince your neighbor, walk around, repoll and so forth, and his book on this is not called “Clickers in the Classroom.” It’s called “Peer Instruction” because he says, and he describes this epiphany, oh my gosh, in two minutes they were able to help each other understand because the ones who understood it did a better job than him because they only recently learned it, where he’s the expert who has the curse of knowledge.
GONZALEZ: Yes.
MARSHALL: Who understands it too well, and even though he did this brilliant explanation, it didn’t get through. So, but back to your point, frequent, so several times. I think, you know, three or four times during, you know, you’re having this interaction of question, poll results, okay, convince your neighbor, now we consolidate, now we move on. So frequent checking in, and he says, I mean his statement on this, it triples the learning gains. And he’s, as I’m sure you know, he is like an apostle around the world, he’s the anti-lecture guy. And, and not just for college, for high school and for middle schools too, I would imagine. Just constantly checking in, which I think, you know, a lot of teachers do this instinctively, and some really need some PD on this. You know, like it’s not rocket science, and yet it’s a mind shift.
GONZALEZ: Yes.
DAVID-LANG: Very teachable, very teachable, yeah. The research, I think Hattie says it doubles learning but if it’s done well, like Hattie says about a lot. So I think that’s what Kim and I are trying to say about these, these five main points is that a lot of people are, you know, you walk into a classroom, you see exit tickets, right, without a doubt. But is it, does it include these five main criteria? And is it being done well? Because that’s what doubles the learning is same curriculum, right, same teacher with and without formative assessment done well, it’s a difference in years’ worth of learning. That’s kind of incredible.
GONZALEZ: Yeah.
DAVID-LANG: But I also think that part of the brief and frequent, you know, this is a big thing of yours, Jenn, is, is making the workload manageable for teachers, right? So we’re not talking about, you know, collecting a huge essay from every kid, right. The reason why you want it brief and frequent is it’s like a dipstick. Kim and I were talking about, should we use that analogy? Because I don’t know if people are old enough, do you remember checking the level of your oil, but it’s quick. You stick it in, and oh, my oil’s low. So it’s a quick check so that it’s, it shouldn’t be burdensome to teachers. There’s no stack of papers to grade. It’s on the spot, right there, and if, and if you can make it more manageable for yourself, then you’re more likely to do it more regularly as well.
GONZALEZ: Right, right.
MARSHALL: But another, another aspect of this is, you know, that deer in the headlights moment of what do I do now? You know, I just taught this. I’m good at this. I understand this, and they’re not getting it. That’s where I think the teacher team can be helpful, the PLC. And, you know, PLC’s not just looking at the actual test they gave or the essays the kids wrote or the artwork they did but actually looking, here was my formative check for understanding, my hinge point question, you know, give me some ideas of what I do now, because I’ve taught it every which way and they’re not getting it.
GONZALEZ: Yeah.
MARSHALL: So I think hey, have you tried this? That would be the great question you would hear in a teacher team meeting. Have you tried this? Oh yeah, yeah, yeah.
GONZALEZ: Yeah, yeah.
MARSHALL: That was a good follow-up. I’ll do that tomorrow.
GONZALEZ: Yeah. Well and Jenn, actually, you kind of just off the cuff listed some of those in the first point. You talked about turn and talk, and we had mentioned convince your neighbor. You talked about having stations set up ahead of time that sort of represent all of the potential issues. That’s, I love that, because that can just be reused over and over. Reteaching a small group. Reteaching the whole class. Anything else?
DAVID-LANG: I think Kim’s point is it’s not always during the class also.
GONZALEZ: Okay.
DAVID-LANG: Like, you know, sometimes you do bring the misconceptions to your teacher team or to yourself when you have more time to think. Like, wow, you know, that’s a new, I didn’t expect the kids not to get that. But for reteaching, you know, I think a lot of independent work so that the teacher could meet with some of the kids who don’t understand.
GONZALEZ: Independent work for everyone else so that you have, yeah.
DAVID-LANG: Yeah, you could have different packets, right, like for kids who did get it, here’s the next level of this is what, and then this is a different packet. You don’t have to label them. Nobody knows.
GONZALEZ: Right.
DAVID-LANG: So you could actually have materials that you bring in that, that address the different misunderstandings.
MARSHALL: But there is a special power to what happens during the lesson, you know, which distinguishes this from the exit ticket. The exit ticket, okay, you look at that, and tomorrow, we’ll follow up.
GONZALEZ: Right.
MARSHALL: But the actual during the lesson, so one of the quotes, and again, I don’t have this exact, but I could email it to you if you want. Basically, if they don’t understand it by the end of the lesson, there’s no guarantee that they’ll understand it in two weeks’ time. But if they don’t understand it during the lesson, you can be sure that they won’t know it in two weeks.
GONZALEZ: Yeah.
MARSHALL: And so, and again, the research on this, I mean, I think this is some of the most powerful research in education. Whereas like PLC and benchmark assessments, the research on that is much weaker. I think partly because it’s screwed up so often, but during the lesson, on the spot, in the moment, dipstick, check for understanding, and following up effectively, that is some of the most rock-solid research in all of education. And not done nearly frequently enough.
GONZALEZ: Yeah.
DAVID-LANG: And I want to add one more idea, Jenn, which is you can, let’s say you’re seeing 80 percent of your kids are not getting, you can reteach it with different models, ideas, and examples, you know.
GONZALEZ: Yeah.
DAVID-LANG: And that’s where some of AI is great, you know. So, you know, you’re going to teach about the three branches of government. You can ask AI, you know, for what are three different ways, and you’re like, oh, this is what will work for my kids. And you try teaching it that way, you do a check for understanding. Oh, that wasn’t, and then you could actually just, I think the problem is when teachers just do it again or like Kim was joking, more slowly.
GONZALEZ: Yeah. Slower and louder, yeah.
DAVID-LANG: …what will you as the teacher, what are different examples or analogies in your teaching that could be different that maybe, and then you got to check again. That’s something else. I don’t know if we made that clear, right.
GONZALEZ: Right.
DAVID-LANG: You do a check for understanding, you teach a little bit, and then you have a different question. Obviously not the same one, to check for the understanding. And oh, oh now a large enough percentage got it that I can deal with the two stragglers.
GONZALEZ: Yeah.
DAVID-LANG: I can move forward, yeah.
GONZALEZ: Yeah.
MARSHALL: I can’t remember if this was a Cult of Pedagogy thing, but the idea of during the actual lesson going to ChatGPT.
GONZALEZ: Oh. No, that’s not from me, yeah.
MARSHALL: Okay. But I mean there’s no reason, I mean if you’ve got — this school I was in, they had 70-minute lessons. There would be time to go right on your computer and say, explain this concept in a way that a fifth grader who has the following misconception might understand. And I mean, I am, there’s all the reservations about ChatGPT too, but my, it is a marvel.
GONZALEZ: Yes.
MARSHALL: I mean, because it will come up with three or four different ways and, you know, that your teacher team tomorrow, it’s too late.
GONZALEZ: Yeah.
MARSHALL: Let’s find it now.
GONZALEZ: What’s nice about ChatGPT is it doesn’t seem to give up.
MARSHALL: Yeah.
GONZALEZ: You say, give me more, it’s like, okay, here, take some more.
DAVID-LANG: And it doesn’t get tired.
MARSHALL: Right.
GONZALEZ: Yeah.
MARSHALL: Yeah, it doesn’t give you an attitude.
GONZALEZ: Right.
DAVID-LANG: You can read the tone in, if you want.
MARSHALL: And you can ask for it in iambic pentameter too.
GONZALEZ: Oh my gosh.
MARSHALL: It’ll do that.
DAVID-LANG: If you’re feeling bored, yeah.
GONZALEZ: So we’ve got one more left, and this is a very important one, create a culture in which students are not afraid to make mistakes, and in fact, where mistakes are normalized. And we did get to this a little bit with My Favorite No, so let’s finish on this one.
DAVID-LANG: I think this is incredibly important, right, setting, setting the stage, right. So when you create a classroom that’s focused on the rewards, the gold stars, the grades, it’s very hard. When kids are just trying to look good and get the good grade, right. So what we want is a classroom that’s focused more on the learning than on, than on the grades. We want a classroom that is also focused on a growth mindset for the students and for the teacher, the idea that if you didn’t get it the first time, that’s okay. You can learn. You want a classroom where kids are not afraid to make mistakes. We talked about, you know, My Favorite No, but I think the language the teacher uses around mistakes. So let’s say a kid gives an answer and it represents a misconception, “Oh, I’m so glad that you shared that answer.” It represents, you know, I think other kids here probably didn’t fully understand.
GONZALEZ: Yeah.
DAVID-LANG: “Let me explain that,” right? Or “Thanks for bringing that up.” Just the way you talk about mistakes. “Oh, mistakes are actually how we learn. We don’t learn by getting everything right all the time.” So you want to create the kind of culture where formative assessments are not intimidating. You have to make it very clear these are low stakes, nothing’s going in the grade book. Be very explicit with kids. And then I think it’s almost cyclical. If you work to create a classroom where formative assessments thrive and can be successful, then formative assessments actually increase learning and improve the culture. Because when kids are like, oh, I’m not getting graded on this? I can be more honest, more vulnerable. And so then the, the learning improves, and it’s this whole cycle where the culture improves, the formative assessment, and the formative assessment improves the culture. Don’t we want a culture in our classroom where it’s okay to make mistakes, and we don’t, you know, we don’t have kids on a hierarchy of “these are the kids who get it right all the time, and these are the kids who get it wrong.” And I think that’s why you want to have it, going back to the, the earlier point, frequent and brief. So you know what? With summative assessments, it’s like, oh yeah, I’m the A kid. I always get it right. It’s not like that now. It’s like we work constantly seeing what you’re learning, what you’re not learning, and it’s so regular it’s, it’s literally part of the lesson, right.
GONZALEZ: Yes.
DAVID-LANG: I think that’s the idea is it’s not separate. Here’s the teaching and here’s the assessing. It’s part of the culture, it’s part of the lesson. It goes back to that, you know, the quote about painting with your eyes closed. Like, it’s not separate from, from the lesson. So I think that kind of culture is absolutely necessary to make formative assessment.
MARSHALL: So to make this point in my workshops, I found this cartoon where an elementary teacher is sort of pointing at this kid. And she’s saying, “You are wrong, Timothy. You are terribly, terribly wrong.” And sort of setting totally opposite tone. And, and that’s sort of, that’s a pivot into saying, you know, we gotta make it low stakes. You gotta make it, you set that climate up. So you know Jeff Howard’s work, don’t you, Jenn, the efficacy guy?
GONZALEZ: No.
MARSHALL: Okay, well.
GONZALEZ: I’m going to write that down.
MARSHALL: By the way, yeah, he’s, he’s amazing. He’s spent the last 55 years of his life, you know, working on achievement, motivation, and McClelland and sort of all that stuff. Anyway, for, you know, think you can, work hard, get smart as opposed to the opposite, focusing mostly on children of color. Anyway, he has a thing called FADAF: Failure and difficulty are feedback. And so, and he actually teaches this to kids, FADAF, you know, you got to FADAF this one. And then, and the third thing that popped into my head on this one is the Thinking Classrooms, the Liljedahl work.
GONZALEZ: Yes.
MARSHALL: Which, by the way, thank you, Jenn, for alerting me to that. And I, everywhere I go now, when I mention that, like somebody in the room just, like, lights up. “Oh my God. We’re doing this.” In fact, the school up in Maine, you know, they’re really into this. They read the book two years ago.
GONZALEZ: Yeah.
MARSHALL: And they’re spreading it to science and social studies. But anyway, what’s going on in there is kids, it’s standing up, you know, with the one worker in groups of three at whiteboards around the room, grappling with a difficult problem. But the teacher walking around and seeing the mistakes, you know, giving hints, prodding. I saw a high school trigonometry class in New York City where 30 kids, you know, working on whiteboards on a trig problem, and the teacher all of a sudden said, “Huddle up.” So all 30 kids came to one whiteboard. And said, “So what’s going on here?” But just, again, and not embarrassing those kids.
GONZALEZ: Yeah.
MARSHALL: You know, what’s the thought process here? Look at these steps here. Those kids explained it. No actually, he had the kids who were doing it stand aside. Someone else say what’s going on here. So creating this very low stakes environment where the mistakes were part of, you know, kids do a gallery walk. Have you seen this in action in classrooms? Have you seen Liljedahl’s stuff?
GONZALEZ: I’ve seen the, the videos of it. I haven’t seen it live. I was actually, went to a conference where someone else was doing kind of a version of it with teachers. So I participated in that. So I got to be in one. But I’ve not —
MARSHALL: So it is extraordinary. There’s a group of high schools in New York City that are really onto this, up in Queens.
GONZALEZ: Yeah.
MARSHALL: They’re visiting each other’s schools. I was a part of a tour of one. So it is, but I think this goes to the heart of how do you create this culture —
GONZALEZ: Yes.
MARSHALL: — where it’s low stakes, you know, a mistake, a mistake, not mistake-free, but, you know, mistake-okay. But they don’t FADAF: Failure and difficulty are feedback.
GONZALEZ: Well, this, okay, so remind us again. The website is the best, you say the URL so people can go and find it.
MARSHALL: Bestofmarshallmemo.org.
GONZALEZ: Okay, and that is where they are going to find just —
MARSHALL: And, and we made an early decision to no password, no login. It’s completely open. You just go there and then you see these, these actually now 24 because we added two more. But you’ll see the bubble on Assessment for Learning, and you click that, and then you see the list, table of contents, and then you can scroll through them. You can then get a PDF of, or you can listen to it. There are two different recordings of it, professional recordings.
GONZALEZ: Oh, fantastic.
MARSHALL: So it’s all there.
DAVID-LANG: The topics in particular, the teachers, because I know that’s your primary audience, might be interested in. There are, and each of these is a collection of 10 to 15 articles. So you, there’s a, there’s a collection of articles on making learning stick. There’s one on differentiation, one on literacy, one on mathematics, one on beliefs about students, social-emotional development, planning units and lessons, positive discipline. This one we’re doing today, assessment for learning or formative assessment, grading practices. So I think those are probably the topics that are most interesting to teachers that are on the website.
MARSHALL: And also race and equity. I think we’re very —
DAVID-LANG: Race and equity.
MARSHALL: Very proud work. So one, one way of doing this, you know, and Jenn has got these wonderful PD suggestions at the end of every one. But one way of doing it, I think you mentioned this in your, in at least one of yours is, I did this with a leadership team in a school in New York City, in Bedford-Stuyvesant. So we took 12 people in the room. I just went around and assigned, you know, they got it up on their device, so they’re looking on their cellphone or their laptop. Each assigned each article to one person. And they read it silently. So the summaries are, you know, can be read in five or so minutes. They read it silently. Then we went around in order. There was no, no empowerment here. It was assigned, you know. Jenn, you’re reading Article 1. And then each person summarized their article.
GONZALEZ: Yeah.
MARSHALL: Said who it was by, summarized it, and what struck them about it and how it resonated with them.
DAVID-LANG: A little jigsaw.
MARSHALL: Yeah.
GONZALEZ: Yeah.
MARSHALL: So in 35 minutes, you know, everyone read the whole chapter, which would take an hour.
GONZALEZ: Right.
MARSHALL: Everyone gave their personal stuff. This particular school we were talking about beliefs about students, and it was powerful.
GONZALEZ: Wow.
MARSHALL: Because every person in that room had something, you know, I was, I didn’t know English when I came into kindergarten. They thought I was stupid. All these sort of personal stories.
GONZALEZ: Yeah.
MARSHALL: But that’s just a very efficient way of using each one of these chapters or, you know, modules.
GONZALEZ: Yeah. That’s such a nice little protocol, and it’s free. And so this is like a great idea for PD. And this is years of work that you have done to comb through everything out there on education. It’s condensed and curated way down to just the stuff that is really going to be the most powerful. So people definitely need to go and check that out, because this is a fantastic free resource. Thank you both so much for coming on here and for the work that you do for teachers and school leaders. I really, it’s just an honor to have you on.
DAVID-LANG: Thank you. It’s been a great chat with you. Clearly you do such important work.
MARSHALL: Yeah, super-duper.
For a full transcript of this episode and links to all the resources mentioned in this episode, visit cultofpedagogy.com, click Podcast, and choose episode 236. To get a bimonthly email from me about my newest blog posts, podcast episodes, courses and products, sign up for my mailing list at cultofpedagogy.com/subscribe. Thanks so much for listening, and have a great day.