Loading...

11 - Derivatives, Laplacian & Unsharp masking - Duration 14:24 - Optional breaks at 05:21 and 11:33

Course video 26 of 79

Some of the most basic tools in image processing, like median filtering and histogram equalization, are still among the most powerful. We will describe these and provide a modern interpretation of these basic tools. Students will then become familiar with simple and still popular approaches. We will also include non-local means, a more modern technique that still uses classical tools.

In this course, you will learn the science behind how digital images and video are made, altered, stored, and used. We will look at the vast world of digital imaging, from how computers and digital cameras form images to how digital special effects are used in Hollywood movies to how the Mars Rover was able to send photographs across millions of miles of space. The course starts by looking at how the human visual system works and then teaches you about the engineering, mathematics, and computer science that makes digital images work. You will learn the basic algorithms used for adjusting images, explore JPEG and MPEG standards for encoding and compressing video images, and go on to learn about image segmentation, noise removal and filtering. Finally, we will end with image processing techniques used in medicine. This course consists of 7 basic modules and 2 bonus (non-graded) modules. There are optional MATLAB exercises; learners will have access to MATLAB Online for the course duration. Each module is independent, so you can follow your interests.

Skills You'll Learn

Image Processing, Image Compression, Image Restoration, Image Segmentation

Taught By

Guillermo Sapiro
Professor

Transcript

Hello. And welcome back to our Image and Video

Processing Class. We have seen very interesting effects

that we can obtain in images just by doing these very, very simple local

neighborhood operations. Let's use this video to see a couple of

additional ones. In particular, we're going to talk about

derivatives. How we implement those derivatives with

local neighborhoods, and what important are, how important are those derivatives

to do very, very interesting effects in images.

So let's dive into that. Here we have a picture.

It's a one dimensional image that has flat, rump, flat and a very, very sharp

jump and flat again. And let's see what happens when we take

the derivatives of this. I have drawn this here.

But before that you might ask yourself, derivatives, that's a continuous

operation. How can we implement that in our

computer? There are many, many ways of doing that.

But let me illustrate a very simple one. If I want to take the derivative of an

image, in the x direction, I can basically write that in discrete

fashion as f, add the pixel x1-fx.

+ 1 - f(x). Very simple.

I just subtract two consecutive pixels. If I want to write the neighborhood

operation. Let's say in a 3x3 that would mean f(x +

1), I have a 1 here, - f(x). I have a minus one here, and I replace as

a zero. So that's basically how I represent that.

I get, a one dimensional derivative in the x direction.

Similarly, I can get a one dimensional derivative in the y direction.

So we see here that, simulated values of this rump, flat, jump and flat.

So we start with flat 6 we have a, decrease in rump,

flat 1 and then a big jump. The first derivative is this first row

that we see here. Very easy to compute following this

operation. We have actually a couple of 0s because

there's no derivative. The values are constant.

Then basically we have a -1, -1 that means that the pixel values are

decreasing in jumps and constant jumps of one.

It's negative, so they are decreasing. And then once again, we have flat.

And then there's a big jump here when we do 6 - 1,

we're going to get 5 and then it flattens out again.

So we already see that considering the first derivative, we can understand that

for example there is a rump, a smooth decline of pixel values and then

there's a big jump. Very interesting with a derivative we can

detect jumps as we expected. Then we can take actually the second

derivative. And once again we can write an explicit

formula and a mask for this second derivative.

So we are going to do this second derivative of our image, once again in

the x direction. And once again there are many ways of

implementing the second derivative but one of them is to take f at x + 1 + f(x -

1) and subtract 2 times f at the actual pixel.

And that's, once again, very simple to do with our, ubiquitous 3x3.

Now we have a -2 here. A 1 and a 1 on each one of the sides

representing these two. And then we have a 0 everyplace else.

if we do that, so that's kind of an addition derivative

to the first derivative. It's a very simple exercise.

We are, have to take derivatives of these.

We could directly apply this to the original one if we want.

But it's easier just to take, you know, continuous pixel subtraction to the next

one. And we see jumps when there are big

changes. So, 0 - 0 is 0.

For example -1 - -1, once again, 0. But when there are changes, we see a

jump. We see 1 here.

We see another here. So the second derivative very clearly

marks when there are big changes in our image.

And that's extremely powerful. We are detecting big changes by doing

derivatives which are, at least in their simplest fashion, implemented with masks

like this one. So let's just continue with this, and see

some of the examples in a second. Here are different masks that compute

different types of derivatives. So I'm going to ask you, look at this

mask, for example, and tell me what is this mask actually computing?

Is it computing the third derivative for example in the x direction?

Is it computing the third derivative in the y direction?

Or is it computing the sum of the derivatives in the first the first

derivatives in the x and y directions? Or is it computing the sum of the second

derivatives in the x and the second derivative in the y direction.

An interesting property. So what is this mask implementing?

I thought that, that wasn't too hard for you, so I gave you that as an exercise,

but I just write it down. This is actually computing the sum of the

second derivative in the x direction and the second derivative, in the y

direction. So this is computing, the second

derivative of f in the x direction plus the second derivative of y.

Of f, I apologize, in the y direction.

And that's basically we know, either from the previous slide,

because we can compute the second derivative in x and the second derivative

in y, and add them. And we actually know that that's

basically f(x + 1) + f(x - 1) - 2f(x). And I am not writing the ys, because they

haven't changed. It should be at y, y and y and plus.

So this was x and now we are going to do y.

That's f(y + 1) + f(y - 1) - 2f(y). And once again, all these are for a

constant x. So all these are for a constant y.

And all these are for a constant x. When we add them, we get this.

And this is actually what's called the Laplacian of f.

And we briefly talked about this when we were talking about the relationships

between averaging and the Guassian filtering, and the heat equation.

So this is what's called the Laplacian of the image,

is the sum of the second derivatives in the x and in the y direction.

These are different implementations of the laplacian.

Here for example we make the difference even stronger.

We stretch it. And we're going to see the effects of

that. The basic idea is that in the

surrounding, basically we have one sign and in the pixel we have a different

sign. And we are basic computing in that way

second derivatives. Just different implementations of this

type of second derivative, derivatives. So what are the effects of this in an

image? Very interesting.

Here we see an image of the moon. What we see next in this image is the

laplacian It's the second derivative in the x direction, plus second derivative

in the y direction. And we actually see it implemented with

this. Here, we basically just stretch it.

Remember, that we know how to modify histograms so we are stretching it.

And what we are doing next, is we are adding this to this.

So we basically take an image, takes the laplacian.

We stretch it, and then we add them. And we get this image, which you can

observe, is a bit sharper. Basically, some of the details are much

more clear in this image than they were in this image.

Now, we can actually make those details even more clear by computing a laplacian

that takes in contrast with only a -4, a -8.

So it actually stretches the differences even more.

The process would be the same. We take the image,

we compute the laplacian with this, at every pixel.

So every pixel is replaced by minus its own value, and plus the values of the

surrounding pixels and then that we owe to the original image and look, we have a

very sharp rendering of these image, you see a lot of details here that were very

blurry here. Very simple operation.

Image plus its reflection gives us a very, very nice effect.

Many, many of these combinations can be done.

And here we see another example for a different image.

Again an image that looks a bit blurry. We see once again it's a laplacian.

It might be hard to notice. All what we have done from here to here

is once again to take second derivatives, and then we add them.

And look how much sharper this image is than this one.

Much, much sharper image. We can see a lot of the details that were

not very clear here. By a very simple operation of adding the

image to it's laplacian. And here, we see basically what's called

a Sobel edges that are basically a different way of computing these

derivatives. So, very interesting effects in images by

combining derivatives with the image in different directions.

Additions of derivatives, very, very interesting effects.

One of those effects is what's called, Unsharp masking.

It's a very, very famous concept. Very, very simple.

It's call Unsharp masking. We have it here.

The idea is very simple. You take the image and you smooth it out.

For example, how can we smooth an image? We already know many ways of doing that.

One of them is by doing local averages. So that's the response to a mask which is

all 1s. And, you know, 1 / 9 just to basically

normalize. We do that, and then we subtract the

original, minus the blurry image. Now, what's the mask correspondent to the

original image? Very simple.

A mask that does nothing, has a one here and a zero.

And replace X. So originally match is this mask, blurry

matches this mask. We take the differences, so it's this

mask minus this. Would produce this.

Remember signs. So, if I subtract from this, this, I will

get a zero in the middle. And I will get, basically I won't get a

zero if I normalize. But I will basically get eight ninth, if

I do the normalization by the positive number, and every place else I will get

-1/9 a negative number. That's kind of a derivative, as we saw

before, kind of an implementation of derivatives, in the, horizontal and

vertical direction combined. We take that result and we add it to the

original image. And look what we have, we basically are

enhancing the bandwidth as we have seen in the previous examples that basically

things become sharper because what we managed to do with the blurring was

reduce the boundaries and then when we add that back we had sharper boundaries

are very sharp image and that's what that's called Unsharp masking, a very

simple operation and very powerful as well.

So I'm going to conclude this video. In actually the next video by showing you

once again inside math lab how we play with these type of operations of

smoothing added images very simple operations that produce very interesting

effects in our images. So see you in the next video with our

math lab demonstration.

About Coursera

Courses, Specializations, and Online Degrees taught by top instructors from the world's best universities and educational institutions.

Join a community of 40 million learners from around the world

Earn a skill-based course certificate to apply your knowledge

Gain confidence in your skills and further your career

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. All rights reserved.

Download on the App Store

Get it on Google Play