Machine grading - PoC

Loading...

From the course by Rice University

Algorithmic Thinking (Part 1)

207 ratings

Rice University

Algorithmic Thinking (Part 1)

207 ratings

Course 5 of 7 in the Specialization Fundamentals of Computing

Experienced Computer Scientists analyze and solve computational problems at a level of abstraction that is beyond that of any particular programming language. This two-part course builds on the principles that you learned in our Principles of Computing course and is designed to train students in the mathematical concepts and process of "Algorithmic Thinking", allowing them to build simpler, more efficient solutions to real-world computational problems. In part 1 of this course, we will study the notion of algorithmic efficiency and consider its application to several problems from graph theory. As the central part of the course, students will implement several important graph algorithms in Python and then use these algorithms to analyze two large real-world data sets. The main focus of these tasks is to understand interaction between the algorithms and the structure of the data sets being analyzed by these algorithms. Recommended Background - Students should be comfortable writing intermediate size (300+ line) programs in Python and have a basic understanding of searching, sorting, and recursion. Students should also have a solid math background that includes algebra, pre-calculus and a familiarity with the math concepts covered in "Principles of Computing".

From the lesson

Module 1 - Core Materials

What is Algorithmic Thinking?, class structure, graphs, brute-force algorithms

Coding styles and standards - PoC11:53

Machine grading - PoC10:31

Plotting data - PoC14:22

Peer assessment - "We want a shrubbery!" - IIPP15:46

Meet the Instructors

Luay Nakhleh
Associate Professor
Computer Science; Biochemistry and Cell Biology
Scott Rixner
Professor
Computer Science
Joe Warren
Professor
Computer Science

[MUSIC]

One critically important aspect of software development is testing.

Now in another video,

we're going to talk a little bit more about testing on your own.

But in this video,

I'm going to talk about the testing that we're going to do for you.

We are going to be using a machine grader in this class and

this machine grader will actually be the thing that gives you a grade.

But hopefully, you'll also find it valuable

in that it will basically test your code and give you feed back.

Now I don't expect you to be using our machine grader as you

only form of testing, but it definitely is sort of

a sanity check that you are headed in the right direction.

It's going to give you information about what test you're getting correct,

what you're getting incorrect and

it is going to do some checking of the style on your code.

So you should be running our machine graders as you develop things, so

that you can get this feedback.

So let's talk about how they're working, how you use them.

In this class, we're going to be using a machine

grader that we developed called OwlTest in order to evaluate your code.

So for each mini project, you're going to go to the mini project description page.

And under the Testing Your Code section, there will always be an OwlTest link.

And you have to get OwlTest through this link,

because this link actually passes parameters to OwlTest

that configures it to properly evaluate this particular mini project.

So once I've clicked it, here's the OwlTest page that you can see.

A couple things to note here.

The first is the background is sort of a pale, yellow color.

And the second thing to notice is up in the upper left-hand corner,

you should see the word OwlTest there.

And this is how you're going to recognize that you're on the OwlTest page.

A little bit more about that later.

So, what's actually here?

Well, we have a text box here where we can input a CodeSkulptor URL.

So when you are ready to evaluate your code, you come over here and

you paste your CodeSkulptor URL directly into that text box and

we click the Submit to OwlTest button.

And it gives us a message, waiting and then it creates my code.

Now you can see I got a whopping 14 out of 100 here, pretty good for me.

And it tells me why?

I lost 80 points, because my merge function failed the test here.

And I lost 6 points, because I have style warnings.

Now if we go back up to the description here of the test,

it tells me that merge is worth 80 points.

So I lost them all, I got nothing correct here and style is worth 20 points.

So I did get some of my style points, I only lost six.

First, I want to walk through here.

There are other tabs.

My style warnings are put on particular tabs based on the category they fall into

in Phyton.

So here is some general code warnings that says, I have an unused variable named

xyz and that's not allowed as per Python in our style guide.

There're some coding convention warnings where I have an invalid name for

a global, I shouldn't even have global variables.

If you read the style guide and I have an invalid name, a.

This does not follow our conventions.

And you can go navigate to a page on our website that gives much more

information that helps you decipher some of these Python errors.

You even can see your code here, which you shouldn't have to, but

you can see your code to see what you actually submitted.

Now back to the real important tab here, the Unit Yest Failures.

And one thing I want you to notice here is what basically,

this is saying is this is what the tester did.

It called merge with an input list of 2 and you can actually copy that code right

there, paste it into CodeSkulptor and run it.

This is the exact test that was run in OwlTest and

you can run it in CodeSkulptor directly.

Now I will point out that as the class moves on, there will be some more

complicated tests where the test is not just a single function call.

There will be multiple lines of code that go on here.

Those lines will each be separated by commas in this listing, so

that it doesn't get too lengthy.

And so when you paste it in, you will have to replace those commas with line breaks,

so that you can actually execute the multiple statements in sequence.

But then it says, expected 2.

What this means is when I call merge with a list of 2, OwlTest expects the answer or

the return value from merge to be a list with the number 2 in it.

But it actually received a list with the numbers 2, 2 and 3 in it.

So my merge function is returning the list with 2, 2 and 3 in it.

When OwlTest says, it should have been returning lists with 2 in it.

I'll let you figure out who is correct here.

Basically, the idea is you look at these tests.

And as things go on,

there will be more than one kind of test in addition to the style test.

And so each one of them will give you some feedback,

tell you how many points you've lost.

And as you pass more and more tests, hopefully, you'll get more and

more points or rather lose less and less points.

And eventually, maybe you'll get a perfect score.

Now, its really important to understand that OwlTest does not actually submit

anything to Coursera.

OwlTest is really just for you to get feedback, as you develop your code not for

you to get any sort of grade.

The way to think of this is that OwlTest is an interface to our machine grading

infrastructure that allows you to submit your code, get it graded and get feedback.

Now, we take that and we just throw it away.

It's just for you.

When you're happy in what you have, now you have to come back to the Coursera sign

and you have actually to submit the code to, of course, to Coursera.

The way to do that is to find the actual assignment page here.

So if I want to submit 2048 (Merge),

I find this assignment the 2048 (Merge) assignment.

I click on that link and it takes me to a page that looks like this.

You would have to set up your identity to actually submit something,

I'm not going to do that.

I don't want to verify right not for the purposes of this video.

So I'll continue without verification, you obviously would do that.

This takes me to a page here where there is a blue button that says, Open Tool.

This is going to take us to our machine grading infrastructure once again.

This take us to something that I am going to call Coursera LTITest.

Now, what is that?

What is it? It is a different interface to exactly

the same machine grading infrastructure as OwlTest, but

you should notice there are two things about this page that are different.

The first is that the background is white.

So, that's different.

That's an indication that you're on the actual submission page instead of

the practice OwlTest page.

The second is if you look up at here in the upper left-hand corner, it says,

Coursera LTITest.

So when you're trying to submit a grade to Coursera,

you should notice these two things.

That the upper left says the right thing, Coursera LTITest and

the background is actually white.

Otherwise, everything is exactly the same as before and you submit in the same way.

You paste your URL, just as you would've before and you submit to Coursera LTITest.

This is going to do exactly the same thing, submit it to exactly the same

machine grader and I should get the same grade in the feedback.

Let's see.

Yes, I do.

I've still got my stunning 14 here and I'm getting the same feedback.

The difference now is that once I see this,

this has now been submitted to Coursera.

So, my grade of 14 has been submitted.

Now don't worry, it records your best grade.

If I've done better than 14 before, it's not a big deal.

This isn't going to hurt anything and you can submit here as many times as you want

and like I said, it's going to take the best grade.

Now if we go back here, you'll notice that it says, Passed 100%.

At the time that I'm recording this video,

that 100% number wasn't always completely accurate.

This does indicate that I did submit it before at some point.

If you wanted to accurately see what your grade was though,

you go to this Assignment link.

And this is going to tell me for 2048 (Merge),

I had previously submitted at 100%.

Notice it did not downgrade me to 14% even though I just submitted

something with a lower grade.

If this number is not what you expect like it doesn't have any number here

whatsoever, that's a strong indicator that you didn't submit to Coursera LTITest,

instead you accidentally just submitted to OwlTest.

And so go back through this process again and

make sure that you're clicking the button and make sure the page says,

CourseraTest not OwlTest and that the background is white and

then you should be able to get your grade submitted to Coursera.

This video has introduced both OwlTest and CourseraTest,

which are two different interfaces to exactly the same machine grader.

They will run exactly the same test, but they're doing so for different purposes.

So first, let's talk about OwlTest.

The idea behind OwlTest is to help give you feedback, as you develop your code.

So as you develop your code, you should run things through OwlTest.

And remember that you have to get through to OwlTest through the mini project

description page, because that passes the parameters to OwlTest that tells it which

tests to actually run.

But when you get that feedback, it's really just for you.

It gets thrown away.

It doesn't really get recorded anywhere, but you can use it.

You can see, hey,

this is going to be my score if I actually do some at the Coursera.

Further, you can see some of the test that you fail.

And remember, you can actually cut and

paste those test directly in the quotes called turn and run them yourself.

That way, you can run them.

You can add debugging information, you can sort of see what's going on and

compare to what you should be getting according to OwlTest and

think about why you're not getting that.

Hopefully, this helps you fix your code and then you can resubmit.

And hopefully, you get a better grade.

So, that's OwlTest.

Now once you've got what you're happy with an OwlTest,

you need to go over to CourseraTest.

CourseraTest is what actually submits things to Coursera and

you have to go through the mini project page to get there.

If you're on CourseraTest and you submit,

you should get the same grade you would have gotten from OwlTest,

but the big difference now is that it's going to post that grade to Coursera.

Now I want to caution you though, this posting is not instantaneous.

What I will tell you is that if you get a grade and

feedback back from CourseraTest and you actually are in CourseraTest,

that grade is eventually going to be posted to Coursera.

However, this is a large class.

There's lots of things going on.

There are many sources of delay between the machine grader and

Coursera getting the grade posted.

So, it could take several hours for

that grade to actually post once you get that feedback.

But if you do get the feedback, it is going to post and

you're going to get the timestamp that you got the feedback on

CourseraTest not the time at which it finally posts to Coursera.

We hope that you find these machine graders useful and helpful and

that they make things a little bit easier, as you develop your code.

Explore our Catalog

Join for free and get personalized recommendations, updates and offers.

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2018 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play