Hello. And welcome back to our Image and Video Processing Class. We have seen very interesting effects that we can obtain in images just by doing these very, very simple local neighborhood operations. Let's use this video to see a couple of additional ones. In particular, we're going to talk about derivatives. How we implement those derivatives with local neighborhoods, and what important are, how important are those derivatives to do very, very interesting effects in images. So let's dive into that. Here we have a picture. It's a one dimensional image that has flat, rump, flat and a very, very sharp jump and flat again. And let's see what happens when we take the derivatives of this. I have drawn this here. But before that you might ask yourself, derivatives, that's a continuous operation. How can we implement that in our computer? There are many, many ways of doing that. But let me illustrate a very simple one. If I want to take the derivative of an image, in the x direction, I can basically write that in discrete fashion as f, add the pixel x1-fx. + 1 - f(x). Very simple. I just subtract two consecutive pixels. If I want to write the neighborhood operation. Let's say in a 3x3 that would mean f(x + 1), I have a 1 here, - f(x). I have a minus one here, and I replace as a zero. So that's basically how I represent that. I get, a one dimensional derivative in the x direction. Similarly, I can get a one dimensional derivative in the y direction. So we see here that, simulated values of this rump, flat, jump and flat. So we start with flat 6 we have a, decrease in rump, flat 1 and then a big jump. The first derivative is this first row that we see here. Very easy to compute following this operation. We have actually a couple of 0s because there's no derivative. The values are constant. Then basically we have a -1, -1 that means that the pixel values are decreasing in jumps and constant jumps of one. It's negative, so they are decreasing. And then once again, we have flat. And then there's a big jump here when we do 6 - 1, we're going to get 5 and then it flattens out again. So we already see that considering the first derivative, we can understand that for example there is a rump, a smooth decline of pixel values and then there's a big jump. Very interesting with a derivative we can detect jumps as we expected. Then we can take actually the second derivative. And once again we can write an explicit formula and a mask for this second derivative. So we are going to do this second derivative of our image, once again in the x direction. And once again there are many ways of implementing the second derivative but one of them is to take f at x + 1 + f(x - 1) and subtract 2 times f at the actual pixel. And that's, once again, very simple to do with our, ubiquitous 3x3. Now we have a -2 here. A 1 and a 1 on each one of the sides representing these two. And then we have a 0 everyplace else. if we do that, so that's kind of an addition derivative to the first derivative. It's a very simple exercise. We are, have to take derivatives of these. We could directly apply this to the original one if we want. But it's easier just to take, you know, continuous pixel subtraction to the next one. And we see jumps when there are big changes. So, 0 - 0 is 0. For example -1 - -1, once again, 0. But when there are changes, we see a jump. We see 1 here. We see another here. So the second derivative very clearly marks when there are big changes in our image. And that's extremely powerful. We are detecting big changes by doing derivatives which are, at least in their simplest fashion, implemented with masks like this one. So let's just continue with this, and see some of the examples in a second. Here are different masks that compute different types of derivatives. So I'm going to ask you, look at this mask, for example, and tell me what is this mask actually computing? Is it computing the third derivative for example in the x direction? Is it computing the third derivative in the y direction? Or is it computing the sum of the derivatives in the first the first derivatives in the x and y directions? Or is it computing the sum of the second derivatives in the x and the second derivative in the y direction. An interesting property. So what is this mask implementing? I thought that, that wasn't too hard for you, so I gave you that as an exercise, but I just write it down. This is actually computing the sum of the second derivative in the x direction and the second derivative, in the y direction. So this is computing, the second derivative of f in the x direction plus the second derivative of y. Of f, I apologize, in the y direction. And that's basically we know, either from the previous slide, because we can compute the second derivative in x and the second derivative in y, and add them. And we actually know that that's basically f(x + 1) + f(x - 1) - 2f(x). And I am not writing the ys, because they haven't changed. It should be at y, y and y and plus. So this was x and now we are going to do y. That's f(y + 1) + f(y - 1) - 2f(y). And once again, all these are for a constant x. So all these are for a constant y. And all these are for a constant x. When we add them, we get this. And this is actually what's called the Laplacian of f. And we briefly talked about this when we were talking about the relationships between averaging and the Guassian filtering, and the heat equation. So this is what's called the Laplacian of the image, is the sum of the second derivatives in the x and in the y direction. These are different implementations of the laplacian. Here for example we make the difference even stronger. We stretch it. And we're going to see the effects of that. The basic idea is that in the surrounding, basically we have one sign and in the pixel we have a different sign. And we are basic computing in that way second derivatives. Just different implementations of this type of second derivative, derivatives. So what are the effects of this in an image? Very interesting. Here we see an image of the moon. What we see next in this image is the laplacian It's the second derivative in the x direction, plus second derivative in the y direction. And we actually see it implemented with this. Here, we basically just stretch it. Remember, that we know how to modify histograms so we are stretching it. And what we are doing next, is we are adding this to this. So we basically take an image, takes the laplacian. We stretch it, and then we add them. And we get this image, which you can observe, is a bit sharper. Basically, some of the details are much more clear in this image than they were in this image. Now, we can actually make those details even more clear by computing a laplacian that takes in contrast with only a -4, a -8. So it actually stretches the differences even more. The process would be the same. We take the image, we compute the laplacian with this, at every pixel. So every pixel is replaced by minus its own value, and plus the values of the surrounding pixels and then that we owe to the original image and look, we have a very sharp rendering of these image, you see a lot of details here that were very blurry here. Very simple operation. Image plus its reflection gives us a very, very nice effect. Many, many of these combinations can be done. And here we see another example for a different image. Again an image that looks a bit blurry. We see once again it's a laplacian. It might be hard to notice. All what we have done from here to here is once again to take second derivatives, and then we add them. And look how much sharper this image is than this one. Much, much sharper image. We can see a lot of the details that were not very clear here. By a very simple operation of adding the image to it's laplacian. And here, we see basically what's called a Sobel edges that are basically a different way of computing these derivatives. So, very interesting effects in images by combining derivatives with the image in different directions. Additions of derivatives, very, very interesting effects. One of those effects is what's called, Unsharp masking. It's a very, very famous concept. Very, very simple. It's call Unsharp masking. We have it here. The idea is very simple. You take the image and you smooth it out. For example, how can we smooth an image? We already know many ways of doing that. One of them is by doing local averages. So that's the response to a mask which is all 1s. And, you know, 1 / 9 just to basically normalize. We do that, and then we subtract the original, minus the blurry image. Now, what's the mask correspondent to the original image? Very simple. A mask that does nothing, has a one here and a zero. And replace X. So originally match is this mask, blurry matches this mask. We take the differences, so it's this mask minus this. Would produce this. Remember signs. So, if I subtract from this, this, I will get a zero in the middle. And I will get, basically I won't get a zero if I normalize. But I will basically get eight ninth, if I do the normalization by the positive number, and every place else I will get -1/9 a negative number. That's kind of a derivative, as we saw before, kind of an implementation of derivatives, in the, horizontal and vertical direction combined. We take that result and we add it to the original image. And look what we have, we basically are enhancing the bandwidth as we have seen in the previous examples that basically things become sharper because what we managed to do with the blurring was reduce the boundaries and then when we add that back we had sharper boundaries are very sharp image and that's what that's called Unsharp masking, a very simple operation and very powerful as well. So I'm going to conclude this video. In actually the next video by showing you once again inside math lab how we play with these type of operations of smoothing added images very simple operations that produce very interesting effects in our images. So see you in the next video with our math lab demonstration.
