Now I'm going to show you something about the new school and how things could be different. And again I'm going to use the example of Scholar, but it's our working environment, it's where we're experimenting about these things. And what I want to show you is the ways in which we tried to make assessment instruction indistinguishable. So, there can't be instruction without assessment, there can't be assessment without instruction. The one is embedded in the other, there is not a distinction. Now, I want to illustrate this idea that there need not be a distinction between instruction and assessment by going into the Creator space in Scholar. And what we do in Creator is we build texts which are what we call complex epistemic performance. They're representations of knowledge which are not just in writing but using all the resources of multimodality. I've got here quite a sophisticated example, an example of a clinical case study at the university level, of veterinary medicine student analyzing the case of Natalie, the cow. But in fact this space can be used from grade four and above, we've had students in here writing historical narratives, we've had students in here writing animals projects in the elementary or the primary school. So this is just, I'm really just going through this now to show an example of how instruction or assessment might be integrated. But firstly, how the knowledge is represented. We have here a relatively simple editor. This is not an italics, it's an emphasis. We can make dot points, we can put in tables, we can put in mathematical formulas, and of course we can put in links, because in this world where we represent knowledge, we might referencing out to something on the web. But also importantly, we can insert images, we can insert video, we can insert audio or we can insert any other digital object which might be a PDF file, something static or it might be something dynamic like a data file, it might be an Excel file, so we can look down behind the pie chart and see what the data is behind it. But also what we can do is we can embed external media in this space, so it might be a YouTube video, and see here in fact to illustrate Bovine Abdominal Paracentesis, we've had this particular student has embedded a YouTube file. As well as putting in tables and diagrams, and all sorts of other kinds of dynamic text for the purposes of their knowledge representation. So over here we have the structure of their thinking, the various sections that they've created. So what we've got at the start is they've done their work, okay? So I'm going to go to a new tab now and I'm going to show you what happens when they submit their work to peers to review. So the peers get a view that looks like this, which is a read-only view. And here is now the full text, including all sorts of data that's been inserted, including diagrams that have been inserted, and an analysis of the differentials, which is an important concept in medicine, the evidence. There's the video that we saw a minute ago in the editable version, and here is in all realm. And there's the references the person's used to the various websites and medical sources and books and journal articles that they have used to build their knowledge. On the right-hand side of the screen, while the student is writing or while they're reviewing a new work, they see a rubric. So, what are the problems that this animal has, what are the differentials, and what we can do is we can open up the rating levels here to get a clear idea of how this might be rated when we come to peer review. These are the differentials, this is evidence from case observation, this is understanding. So this is how this piece of work is going to be evaluated. And when the reviewer gets this piece of work to look at, what they do is they drag across the little slider, they double-check what a 2 is, and if we specify very clearly what a 2 is, we get good inter-rate reliability. They type in a comment and they go to the bottom, and when they're ready they press Submit. Now, as it turns out in this particular work, the results of the peer review are here. So after the peer review is finished, the author of the work gets results. This happens to be the second version of their work, and this is one particular student's review around problems, around differentials, around evidence from the case observation, around understanding. And what's possible to do here in the comment area is for the person who gets the comments to build a kind of a dialog and comment on those things. But the other thing it's possible to do as well is that they can give overall feedback on the feedback. So here, the criteria would be please give your level of writing. This is the writing levels, and if I close that up, this is the feedback that the person gave on the feedback. So what we've got is a kind of a recursive process here. Now, classical assessment processes are linear. You hand in your work and at the end of the project or the end of the semester or whenever it is and you get it back. The teacher hasn't had much time to look at it and they've given it an answer which is B+, which just says you're not a very good person or you could have been better and be better next time. But it doesn't give you a feedback, which is really helpful. Whereas here you're getting constructive feedback but also we're enlisting the students to give each other feedback. And see here that's one review, this is another review which is a little less substantial. Here's another review which is a little less substantial, and also what we do is in terms of the quality of the reviews, this business of rating the reviewers is a way to give the reviewer feedback on their feedback, if you like, along the way. So, there's those kind of cycles of recursiveness. But hopefully what we're doing is we're enlisting the peers to give constructive feedback to each other. So not only is this series of assessments exactly how the professor or the teacher is going to assess the work, but we're enlisting the students into the assessment process. Not only reviews, we also have detailed in text annotation as you can see here, we can mouse over annotation and it takes us to that bit of the text and we can open that up and I think this should be defined as a little earlier as opposed to a few sentences that's one comment. This person said midsize dairy herd or farm, this sentence is a little confusing. The students are working with each other to be involved in the refinement, by the way these were anonymous peer reviews, I can see the names because I'm the admin. There's a setting where they could have been named but in fact they're anonymous. And what we can do is even between anonymous peer reviewers, we can say, what did you mean by that and how can I, there could be a kind of a dialogue that goes on here in this particular space. What's also possible in the reviews area is it's possible for a student to do a self review. So when they get to the point of doing their final version of their work, they can say, look, I took on board this comment, I didn't take on board that comment, and then I can sort of be reflective about the dialogical process. So this is classical peer review that happens in journals, but also in a way it's in this particular case which is the clinical case, it's like a second opinions, it's building a collaborative knowledge discourse along the way. So now, what we've got here is a kind of an architecture where we've got this work about Natalie, the Cow, this analysis of this unwell cow on the left. Something empirical, but also here when were thinking about things like differentials, we're thinking theoretically about medicine on this side of the screen. Here we have the products of student work which is the stuff that happens during instruction. And on the right here, we have forms of assessment. And multiple, and formality of assessment, assessment that goes through a number of cycles. So we've got this left right architecture where I do my work, I look at the rubric. I get however many, two or three, other people's work, and I write reviews. I get my work back and I look at the comments, the results on this side. I finalize a new version of my work, the latest version of my work, and I write a self-review on this side. So we've got a kind of a dialectic between the two sides of the screen, and a kind of an architecture where in fact, as you can see, assessment is deeply embedded into instruction. There is not a distinction between assessment and instruction in a way that might have historically been the case.
