The New School: Embedded Assessment in Scholar

Loading...

Ubiquitous Learning and Instructional Technologies

University of Illinois at Urbana-Champaign

This course will analyze currently available technologies for learning. Areas addressed include: learning management systems, intelligent tutors, computer adaptive testing, gamification, simulations, learning in and through social media and peer interaction, universal design for learning, differentiated instruction systems, big data and learning analytics, attention monitoring, and affect-aware systems. Participants will explore the processes for selection and implementation of suitable technologies, the design of electronic learning resources, design and application of digital media in teaching and learning, familiarization with web usability and accessibility, and critical analysis of the benefits of technologies in education.

From the lesson

Ubiquitous Learning

This module examines the implications of ubiquitous learning for assessment and instruction, and technology as a mediator of social relations.

Assessment in the Old School5:14

The Not-So-New School2:26

The New School: Embedded Assessment in Scholar9:47

Learning As Distributed Knowledge Creation3:22

Assessment As Recursive Feedback6:20

Taught By

Dr William Cope
Professor
Dr Mary Kalantzis
Professor

Transcript

Now I'm going to show you something about the new school and

how things could be different.

And again I'm going to use the example of Scholar, but it's our working environment,

it's where we're experimenting about these things.

And what I want to show you is the ways in which we tried to make

assessment instruction indistinguishable.

So, there can't be instruction without assessment,

there can't be assessment without instruction.

The one is embedded in the other, there is not a distinction.

Now, I want to illustrate this idea that there need not be a distinction between

instruction and assessment by going into the Creator space in Scholar.

And what we do in Creator is we

build texts which are what we call complex epistemic performance.

They're representations of knowledge which are not just in writing but

using all the resources of multimodality.

I've got here quite a sophisticated example,

an example of a clinical case study at the university level,

of veterinary medicine student analyzing the case of Natalie, the cow.

But in fact this space can be used from grade four and above,

we've had students in here writing historical narratives, we've had students

in here writing animals projects in the elementary or the primary school.

So this is just, I'm really just going through this now to

show an example of how instruction or assessment might be integrated.

But firstly, how the knowledge is represented.

We have here a relatively simple editor.

This is not an italics, it's an emphasis.

We can make dot points, we can put in tables, we can put in mathematical

formulas, and of course we can put in links, because in this world where we

represent knowledge, we might referencing out to something on the web.

But also importantly, we can insert images,

we can insert video, we can insert audio or we can insert any

other digital object which might be a PDF file, something static or

it might be something dynamic like a data file, it might be an Excel file, so

we can look down behind the pie chart and see what the data is behind it.

But also what we can do is we can embed external media in this space,

so it might be a YouTube video, and

see here in fact to illustrate Bovine Abdominal Paracentesis,

we've had this particular student has embedded a YouTube file.

As well as putting in tables and diagrams, and

all sorts of other kinds of dynamic text for

the purposes of their knowledge representation.

So over here we have the structure of their thinking,

the various sections that they've created.

So what we've got at the start is they've done their work, okay?

So I'm going to go to a new tab now and I'm going to show you what

happens when they submit their work to peers to review.

So the peers get a view that looks like this, which is a read-only view.

And here is now the full text, including all sorts of data that's been inserted,

including diagrams that have been inserted, and an analysis of

the differentials, which is an important concept in medicine, the evidence.

There's the video that we saw a minute ago in the editable version, and

here is in all realm.

And there's the references the person's used to the various websites and

medical sources and books and

journal articles that they have used to build their knowledge.

On the right-hand side of the screen, while the student is writing or

while they're reviewing a new work, they see a rubric.

So, what are the problems that this animal has, what

are the differentials, and what we can do is we can open up the rating levels

here to get a clear idea of how this might be rated when we come to peer review.

These are the differentials, this is evidence from case observation,

this is understanding.

So this is how this piece of work is going to be evaluated.

And when the reviewer gets this piece of work to look at, what they do is they

drag across the little slider, they double-check what a 2 is, and

if we specify very clearly what a 2 is, we get good inter-rate reliability.

They type in a comment and they go to the bottom,

and when they're ready they press Submit.

Now, as it turns out in this particular work,

the results of the peer review are here.

So after the peer review is finished, the author of the work gets results.

This happens to be the second version of their work, and

this is one particular student's review around problems, around differentials,

around evidence from the case observation, around understanding.

And what's possible to do here in the comment area is for

the person who gets the comments to build a kind of a dialog and

comment on those things.

But the other thing it's possible to do as well

is that they can give overall feedback on the feedback.

So here, the criteria would be please give your level of writing.

This is the writing levels, and if I close that up,

this is the feedback that the person gave on the feedback.

So what we've got is a kind of a recursive process here.

Now, classical assessment processes are linear.

You hand in your work and at the end of the project or

the end of the semester or whenever it is and you get it back.

The teacher hasn't had much time to look at it and they've given it an answer which

is B+, which just says you're not a very good person or

you could have been better and be better next time.

But it doesn't give you a feedback, which is really helpful.

Whereas here you're getting constructive feedback but

also we're enlisting the students to give each other feedback.

And see here that's one review,

this is another review which is a little less substantial.

Here's another review which is a little less substantial, and

also what we do is in terms of the quality of the reviews,

this business of rating the reviewers is a way to give the reviewer

feedback on their feedback, if you like, along the way.

So, there's those kind of cycles of recursiveness.

But hopefully what we're doing is we're enlisting the peers to give constructive

feedback to each other.

So not only is this series of assessments exactly how the professor or

the teacher is going to assess the work, but

we're enlisting the students into the assessment process.

Not only reviews, we also have detailed in text annotation as you can see here,

we can mouse over annotation and it takes us to that bit of the text and

we can open that up and I think this should be defined as

a little earlier as opposed to a few sentences that's one comment.

This person said midsize dairy herd or farm, this sentence is a little confusing.

The students are working with each other to be involved in the refinement,

by the way these were anonymous peer reviews,

I can see the names because I'm the admin.

There's a setting where they could have been named but in fact they're anonymous.

And what we can do is even between anonymous peer reviewers, we can say,

what did you mean by that and how can I, there could be a kind of

a dialogue that goes on here in this particular space.

What's also possible in the reviews area is it's possible for

a student to do a self review.

So when they get to the point of doing their final version of their work,

they can say, look, I took on board this comment,

I didn't take on board that comment, and then I

can sort of be reflective about the dialogical process.

So this is classical peer review that happens in journals, but also in a way

it's in this particular case which is the clinical case, it's like a second

opinions, it's building a collaborative knowledge discourse along the way.

So now, what we've got here is a kind of an architecture where we've got this work

about Natalie, the Cow, this analysis of this unwell cow on the left.

Something empirical, but

also here when were thinking about things like differentials,

we're thinking theoretically about medicine on this side of the screen.

Here we have

the products of student work which is the stuff that happens during instruction.

And on the right here, we have forms of assessment.

And multiple, and formality of assessment,

assessment that goes through a number of cycles.

So we've got this left right architecture where I do my work, I look at the rubric.

I get however many, two or three, other people's work, and I write reviews.

I get my work back and I look at the comments,

the results on this side.

I finalize a new version of my work, the latest version of my work,

and I write a self-review on this side.

So we've got a kind of a dialectic between the two sides of the screen, and

a kind of an architecture where in fact,

as you can see, assessment is deeply embedded into instruction.

There is not a distinction between assessment and

instruction in a way that might have historically been the case.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play