0:00
Hello. And welcome back to our Image and Video
Processing Class. We have seen very interesting effects
that we can obtain in images just by doing these very, very simple local
neighborhood operations. Let's use this video to see a couple of
additional ones. In particular, we're going to talk about
derivatives. How we implement those derivatives with
local neighborhoods, and what important are, how important are those derivatives
to do very, very interesting effects in images.
So let's dive into that. Here we have a picture.
It's a one dimensional image that has flat, rump, flat and a very, very sharp
jump and flat again. And let's see what happens when we take
the derivatives of this. I have drawn this here.
But before that you might ask yourself, derivatives, that's a continuous
operation. How can we implement that in our
computer? There are many, many ways of doing that.
But let me illustrate a very simple one. If I want to take the derivative of an
image, in the x direction, I can basically write that in discrete
fashion as f, add the pixel x1-fx.
+ 1 - f(x). Very simple.
I just subtract two consecutive pixels. If I want to write the neighborhood
operation. Let's say in a 3x3 that would mean f(x +
1), I have a 1 here, - f(x). I have a minus one here, and I replace as
a zero. So that's basically how I represent that.
I get, a one dimensional derivative in the x direction.
Similarly, I can get a one dimensional derivative in the y direction.
So we see here that, simulated values of this rump, flat, jump and flat.
So we start with flat 6 we have a, decrease in rump,
flat 1 and then a big jump. The first derivative is this first row
that we see here. Very easy to compute following this
operation. We have actually a couple of 0s because
there's no derivative. The values are constant.
Then basically we have a -1, -1 that means that the pixel values are
decreasing in jumps and constant jumps of one.
It's negative, so they are decreasing. And then once again, we have flat.
And then there's a big jump here when we do 6 - 1,
we're going to get 5 and then it flattens out again.
So we already see that considering the first derivative, we can understand that
for example there is a rump, a smooth decline of pixel values and then
there's a big jump. Very interesting with a derivative we can
detect jumps as we expected. Then we can take actually the second
derivative. And once again we can write an explicit
formula and a mask for this second derivative.
So we are going to do this second derivative of our image, once again in
the x direction. And once again there are many ways of
implementing the second derivative but one of them is to take f at x + 1 + f(x -
1) and subtract 2 times f at the actual pixel.
And that's, once again, very simple to do with our, ubiquitous 3x3.
Now we have a -2 here. A 1 and a 1 on each one of the sides
representing these two. And then we have a 0 everyplace else.
if we do that, so that's kind of an addition derivative
to the first derivative. It's a very simple exercise.
We are, have to take derivatives of these.
We could directly apply this to the original one if we want.
But it's easier just to take, you know, continuous pixel subtraction to the next
one. And we see jumps when there are big
changes. So, 0 - 0 is 0.
For example -1 - -1, once again, 0. But when there are changes, we see a
jump. We see 1 here.
We see another here. So the second derivative very clearly
marks when there are big changes in our image.
And that's extremely powerful. We are detecting big changes by doing
derivatives which are, at least in their simplest fashion, implemented with masks
like this one. So let's just continue with this, and see
some of the examples in a second. Here are different masks that compute
different types of derivatives. So I'm going to ask you, look at this
mask, for example, and tell me what is this mask actually computing?
Is it computing the third derivative for example in the x direction?
Is it computing the third derivative in the y direction?
Or is it computing the sum of the derivatives in the first the first
derivatives in the x and y directions? Or is it computing the sum of the second
derivatives in the x and the second derivative in the y direction.
An interesting property. So what is this mask implementing?
6:00
I thought that, that wasn't too hard for you, so I gave you that as an exercise,
but I just write it down. This is actually computing the sum of the
second derivative in the x direction and the second derivative, in the y
direction. So this is computing, the second
derivative of f in the x direction plus the second derivative of y.
Of f, I apologize, in the y direction.
And that's basically we know, either from the previous slide,
because we can compute the second derivative in x and the second derivative
in y, and add them. And we actually know that that's
basically f(x + 1) + f(x - 1) - 2f(x). And I am not writing the ys, because they
haven't changed. It should be at y, y and y and plus.
So this was x and now we are going to do y.
That's f(y + 1) + f(y - 1) - 2f(y). And once again, all these are for a
constant x. So all these are for a constant y.
And all these are for a constant x. When we add them, we get this.
And this is actually what's called the Laplacian of f.
And we briefly talked about this when we were talking about the relationships
between averaging and the Guassian filtering, and the heat equation.
So this is what's called the Laplacian of the image,
is the sum of the second derivatives in the x and in the y direction.
These are different implementations of the laplacian.
Here for example we make the difference even stronger.
We stretch it. And we're going to see the effects of
that. The basic idea is that in the
surrounding, basically we have one sign and in the pixel we have a different
sign. And we are basic computing in that way
second derivatives. Just different implementations of this
type of second derivative, derivatives. So what are the effects of this in an
image? Very interesting.
Here we see an image of the moon. What we see next in this image is the
laplacian It's the second derivative in the x direction, plus second derivative
in the y direction. And we actually see it implemented with
this. Here, we basically just stretch it.
Remember, that we know how to modify histograms so we are stretching it.
And what we are doing next, is we are adding this to this.
So we basically take an image, takes the laplacian.
We stretch it, and then we add them. And we get this image, which you can
observe, is a bit sharper. Basically, some of the details are much
more clear in this image than they were in this image.
Now, we can actually make those details even more clear by computing a laplacian
that takes in contrast with only a -4, a -8.
9:47
So it actually stretches the differences even more.
The process would be the same. We take the image,
we compute the laplacian with this, at every pixel.
So every pixel is replaced by minus its own value, and plus the values of the
surrounding pixels and then that we owe to the original image and look, we have a
very sharp rendering of these image, you see a lot of details here that were very
blurry here. Very simple operation.
Image plus its reflection gives us a very, very nice effect.
Many, many of these combinations can be done.
And here we see another example for a different image.
Again an image that looks a bit blurry. We see once again it's a laplacian.
It might be hard to notice. All what we have done from here to here
is once again to take second derivatives, and then we add them.
And look how much sharper this image is than this one.
Much, much sharper image. We can see a lot of the details that were
not very clear here. By a very simple operation of adding the
image to it's laplacian. And here, we see basically what's called
a Sobel edges that are basically a different way of computing these
derivatives. So, very interesting effects in images by
combining derivatives with the image in different directions.
Additions of derivatives, very, very interesting effects.
One of those effects is what's called, Unsharp masking.
It's a very, very famous concept. Very, very simple.
It's call Unsharp masking. We have it here.
The idea is very simple. You take the image and you smooth it out.
For example, how can we smooth an image? We already know many ways of doing that.
One of them is by doing local averages. So that's the response to a mask which is
all 1s. And, you know, 1 / 9 just to basically
normalize. We do that, and then we subtract the
original, minus the blurry image. Now, what's the mask correspondent to the
original image? Very simple.
A mask that does nothing, has a one here and a zero.
And replace X. So originally match is this mask, blurry
matches this mask. We take the differences, so it's this
mask minus this. Would produce this.
Remember signs. So, if I subtract from this, this, I will
get a zero in the middle. And I will get, basically I won't get a
zero if I normalize. But I will basically get eight ninth, if
I do the normalization by the positive number, and every place else I will get
-1/9 a negative number. That's kind of a derivative, as we saw
before, kind of an implementation of derivatives, in the, horizontal and
vertical direction combined. We take that result and we add it to the
original image. And look what we have, we basically are
enhancing the bandwidth as we have seen in the previous examples that basically
things become sharper because what we managed to do with the blurring was
reduce the boundaries and then when we add that back we had sharper boundaries
are very sharp image and that's what that's called Unsharp masking, a very
simple operation and very powerful as well.
So I'm going to conclude this video. In actually the next video by showing you
once again inside math lab how we play with these type of operations of
smoothing added images very simple operations that produce very interesting
effects in our images. So see you in the next video with our
math lab demonstration.
Guillermo SapiroProfessor
