[MUSIC] Welcome to week three of this course on assessment. The theme for this week is reporting assessment information. And, we're first of all going to pick up from last week, on feedback by talking about how we can report feedback usefully. In looking at our curriculum map, what I've added now to this diagram is the extension to the right, in which we take all this information from the assessment process, and we bundle it up together in terms of a communication or reporting to external stakeholders. Clearly there's reporting to students, there's reporting to their parents, and school leaders, and even possibly depending on your country to government officials. And, there's a wide variety of things that we can and are often asked to report about. We're asked to report on the nature of student learning; parents often and especially want to know about behavior; almost everybody wants to be sure that children attend and aren't late; we like to know things like what their aptitudes or attitudes are, whether they're making an effort; are they involved in extra-circular activities. So, there's a wide variety of topics, that are expected in school reporting. Teachers and schools often want to know, what is the ability of this student? What is their success? And there's a sense in which assessment information also gets bundled up with the idea of school quality. The number of parents who ask me, "Is x a good school?" And I ask them, "What do you mean by good school?" And for many immigrant parents, a good school is where the children are well behaved, the children are dressed nicely and the children get high scores. And, that simple understanding is rather prevalent. A lot of parents think a good school is one where students get high scores. And so, we have this as a tension in education because we know that for students to improve, we have to first of all find out that they have bad scores or low performance, so that we can work on it to move them towards good performance. And reporting can be done both verbally, in conversations with parents, or with some students - and quite informally, in New Zealand, primary school practice is very strong on this verbal interaction between the parent and the teacher. Parents drop the kid off and say, "Oh, can I have a word with you" and the next thing you know, you've got an instant reporting session. But, almost all schools rely also on formal written reporting. In New Zealand, primary schools are required to report formerly against national standard, at least twice a year on every child to every parent. A lot of other schools would report at the end of every one of our four semesters, or sometimes just three semesters of the four. And then at least there's always some sort of formal reporting. Reporting can be seen as a form of feedback. Reports involve information from assessments, as well as observations and records of other aspects. But, there's three time perspectives that can be involved in reporting. There's the retrospective, the looking back. Here's where your child is now compared to where they used to be. The advantage of this kind of reporting is that even though the child may not be high on the scale, they may be 25th out of 30, or they may only be a low C, or below average for their year group - they've still had progress, and making the progress visible is powerfully useful. Certainly in athletics, people keep a record of their performances and get excited. "Yes, I came 300 out of 2,000, but I did a personal best." Knowing that you've done better than ever before is still a powerful and useful thing to do. There's the prospective view, looking ahead. What's next? Where is the child going? What are the targets and goals? It's based on where the child is now, and with what progress they've made, here's some reasonable, plausible, challenging but achievable targets. As the teacher states, this is what I will do as the teacher, and this is what your child should be doing. And then there's the more introspective view. Students are not just widgets being shaped and formed by teaching processes, children are willing, we hope; active, we hope, participants in teaching and learning. And so, it's useful to ask them, "So, what do you think? Are you better than the last time? Have you improved, and in what way have you improved? Are there any things that you've done that you'd like to change? Where would you like to go in your learning next? What do you think you should do next?" These kinds of conversations and questions, make for very interesting reporting. And these are a useful technique when you're going to having a parent, teacher and child interview meeting - ask the child what they think about their learning, as a start off would be a useful technique. One of the tragedies that we've discovered in researching the qualities of reports by schools and teachers to parents, is there's a strong tendency for teachers to focus on surface features about presentation and neatness, quantity of work and effort, and punctuality. These kinds of surface indicators, this is not to say that these things are unimportant, but they tend to draw attention to compliance rather than the deeper issue of what did they actually learn? Another tendency is to see praise in comments written by teachers, and especially the dangerous praise, "Doing well for his or her age", and "A pleasure to teach". Now, no parent wants their child to be considered a horrible brat. But, if that's the best the teacher can tell you, then that's not a very informative report about what the child needs to learn, where they are, and where they have to go next. Clearly, blaming reports that blame the learner - "You clearly didn't study, you need to pay more attention, you're not working to your capacity." While the teacher may feel frustrated about the student not working as hard, often teachers don't know how hard students are working. In a lot of developing nations, students do work long hours late into the night, trying to master material. So, blaming and praising learners, we've already seen in our feedback talks, don't really work. And yet, these are the kinds of things that appear in teacher reports. Another interesting habit is students who are very good often when they hand their work in, they'll often get told, "It could have been better. This wasn't quite right, this wasn't quite right", even though the general quality of the work is good. In contrast, students who normally hand in low quality work, when they hand in something considerably better than they have ever done before, teachers will often praise this student, explicitly and in front of the whole class. Whereas the student who has done excellent work, will often get this message back to say, "Your work wasn't good enough". So, I think there's a tendency for teacher reporting to be inconsistent, in relationship to the criteria and targets of the work that the children are doing. Roger Peddie and John Hattie did a survey study of school reports in New Zealand, and they concluded that on the whole more than 80% of children are a pleasure to teach and doing well for their age. And, we know that on average 50% of children are below average for their age, so what we are seeing is that a lot of teacher reporting just fails to meet standard expectations of "Tell me the truth." A long and important study in Mangere in Otara, Peter talked to the Pacifica people of New Zealand and in Mangere in Otara there are a lot of Pacifica families, and the constant message from those families to the schools was, "If my kid is behind, tell me the truth, so we can do something about it." And, this was a struggle for the teachers to find a way to tell this truth, while not somehow doing something wrong or dark. What should a good report tell us? Well, this is a report system that's actually being devised by psychometricians, statistical experts in analyzing test scores. And what they've suggested is that we need to break down performance on test items and test questions, according to whether the questions were hard or easy, and whether the student got them right or wrong. And once we've determined what the level of performance of the student is we can then say, anything the student got right, but was hard compared to their ability is a strength, and is shown in this diagram in the yellow box as a strength. In contrast, anything that they got right that was easy to do, is in the green mastery box. They've mastered these things. In contrast, when students make mistakes, sometimes they're easy things, that we would have expected them to get right. Now, that's always disappointing when good students make mistakes on easy material. But, they should be treated rather as gaps that the student might need to practice, which they could easily do at home. But more importantly, the take home message should be this is going to be easy for you to learn because overall, your performance is better than these questions. You're able to do much harder things than these questions. And then finally is the hard stuff, that students get wrong. Now, a good test should have some challenge in it. So, when there's challenging questions, they'll be questions that students get wrong because they don't know how to them. They're just too hard. And those are represented in the blue box in this diagram as their needs. Based on this kind of analysis of how hard the question was, and whether the student got it right or wrong, we can diagnose a teaching response. When students have strengths in the yellow box, we should congratulate them. We should have say, "You can move ahead now in this material to more advanced material. In this aspect of what I'm teaching you, you're ready to do the harder stuff. And you should start doing that." In the green box, the stuff that they mastered because it was easy and they got it right, the take home message is really simple: "Teachers, stop teaching me this stuff. I already know how to do it. I don't need to keep practicing it. I know how to do it." So, the teacher needs to say, "Well, I don't need to repeat that for this group." In the red box, easy things that the child got wrong, some coaching, a little bit of practice and work at home on it, probably is all that's needed. The stuff in the blue, that they got wrong and that was hard, then that's what the teacher needs to plan to teach. That's the material that the student probably won't get right without some direct teaching from the classroom teacher. And that's where, the school leader or the grade leader or even the parent could say, "So, what's your plan? When will you be teaching this stuff that my child finds hard and needs help with?" Unfortunately, teachers in most schools, don't teach single students. They teach classrooms of students. And, this chart shows us that we can take the same logic and aggregate it across multiple questions for multiple students, and what this chart shows us is everywhere there's a large blue zone, we know that's an area that was hard for the students, they got it wrong, and they need to be taught it. And so, the teacher could look down the list of this report and go, "Here's the areas where the kids need to be taught", and start planning to teach that in the next two to three weeks or two to three months. And certainly, the stuff in the green, where they got it right and it was easy for them, that's the stuff the teacher doesn't have to do anymore, take that out, and put in the blue stuff. And that's how this kind of assessment analysis can inform teaching and ultimately reporting. So, how can you put this approach into work in your class? If you remember back when we talked about the test template in week two and lecture two, if you can analyse the content and the objectives of your assessment tasks, and you have a sense of this is the easy stuff, this is the middle stuff, and this is hard stuff, then you could manually plot each student's performance, by content into one of those four squares. Did they get it right or wrong, and was it hard or easy? And then, you get a sense of how could this test guide me in identifying who needs to be taught what next. And having identified who needs to be taught what next, you begin to be ready to tell parents and school leaders and even governments, how you're doing and what you're doing with your educational progress. More of that in the next session. [MUSIC]
