Hello, and welcome to week three of our class on image and video processing. What you're seeing right now is me in a relatively dark room. You can probably see the Duke Chapel behind me but you can't see details, for example, the color of the shirt I'm wearing. Or, if you were to look at my desk, it's basically very dark, and you can't see much. What happen when we take an image like this? What happen when we have a video that is very dark? That happens us all the time, we take a picture at night. Doesn't have the lighting that we want. It's too dark or too bright, sometimes too noisy. What we are going to do during this week is to learn how to improve those images. How to enhance those images. We're going to start with very simple manipulations. That will basically help us to improve the contrast, remove the noise. And we're going to go towards the end of the week to more advanced techniques. Which are basically leading to state of the art approaches today. And we're going to come back to some these towards the end of the class to basically week seven, week eight, to really, really advance techniques for image enhancement. During this week and during the following weeks we're also going to do online and real time demos so I'm going to basically teach you some of the material and then we're going to run some different software platforms to basically do some of these operations. Right now on my PC, and you are going to see them, in practice, how they are working. We're also going to introduce some online video quizzes so, basically I'm going to pause for a second and you're going to be able to think about a question that I will post to you. That won't basically affect your, your certificate at the end but it's just to have you think for a while before I give you the answer to the question, like in any regular class that I will basically pause to have the students think for a minute. Before we continue let me turn on the lights so I can continue this and write. And explain to you the basic principles of image enhancement and contrast enhancement and image you're now seeing. So bear with me for a second. I'm going to turn on my lights and we will be right back. So now I'm turning the lights on. So you can see. A bit of the details. you can't see much in the background anymore but you can see what's in the room. And what I'm going to show you is how we turn the lights on, digitally, in the computer. So you have taken your image. You can now go back to SIM. I want to turn the lights on with image processing and video processing software so let's start doing that. And we basically are going to start from very simple operations, which are histogram modifications, or pointwise operations. So, the basic idea is illustrated in this figure. Every pixel of the image, it's going to change it's own value, following certain function. So. The pixel here will move to a new image where basically it changed its value but it will change its value only based on its own value right now. It's not going to be influenced by anything that's happening in any other part of the image. That's the first part of this week three. We're going to see later how we basically influence the pixel value by other things that are happening in the image. But to start, very simple, what's called point-wise operations. There's a certain pixel value in, in the input, a certain pixel value in the output. Similarly here. For example, what's happening here is that everybody up to pixel value K. Becomes zero, everybody above K becomes one or the maximum value. In the image. So this is basically a thresh holding operation. So, what we need to specify, is it one dimensional function, that translates the input to the output. And, let's just see, a few examples of that. So here we see a few illustrations of this. Several of these operations, inverse log. Of course lets start from the identity. The input is identical to the output. So basically this is the identity. If we basically mark the output as S. An entity is S = R. The output is identical to the input. We have done nothing to the image. We also have a log operation. For example the output is some constant times the log of the input, and we normally are going to add let's say 1. Because if the input is 0, we cannot take the logarithmic of zero. So we do we this. So that's an hour operation. Whatever is the grade value of the input we transform it to the output. There is also the inverse, which is written here. And I want to, I want for you to think for a second, what will be basically the function that will represent the inverse. Considering that our image goes from 0 to L. So the minimal value is 0, the maximum value is L. While would be the function, basically I want to write S. As a function of r. Why would be that function that will basically do the negation of the image. The maximal value of the input will become zero. And the minimal value of the input meaning zero would become L. So let's just think for a second and it's a very simple exercise just to illustrate how simple. And as we are going to see in a second, how powerful at the same time, these type of operations are. So, we know that, basically, I ask you how to, basically, construct this function? And it's very simple. We want the output to be the inverse. of the input, so S equal L, the maximum value minus R. So when R is L, S becomes zero because its L minus L. When R is zero, then basically S become L, which is the maximum. So, a very, very simple operation that just involves our great values. And note, that all what occur in all of these operations, is the actual great value. I'm not asking where's the pixel located in the image? I'm just asking what's the pixel value? What's the great value of that image? So, here is one example of this. Basically this function that we just wrote S equal L minus R is basically taking every pixel and doing the inversion so it's the negative image and some of the things that we see is that is basically easier for us to. See just visually some of the things in the negative image than it is. In that original image. And why is this? Remember early on we talk about the properties of the visual system that there are certain contrast that we understand better. That we perceive better at different levels of the background. And we saw that there are some contrasts that are perceived better when we are in the dark regions. Some contrasts are perceived differently when we are in the light regions. So, this inversion, basically, might help us to perceive things that are difficult to notice in the original image. Very simple illustration. One other important thing of these type of operations is, I can go back. So, if I start from here, I have not destroyed the information. I can simply go back and reverse this operation, that's not always possible as we're going to see later this week. But, for this particular operation that is doable. Another extremely important operation is basically an exponential function, and I'm going to write it here S, the output is going to be a constant, the input, to some exponent, gamma, and the basic idea here is that depending on, gamma in particular we get different functions, that transforms the input, to the output, and this is very often called gamma correction. And this actually is very popular. In monitors, because of some properties of the monitor, if you don't do a gamma correction to your image, your image is going to look relatively dark. So, if this is. My image its going to look dark, and it's going be hard to perceive exactly the details. And that's because of the properties of the monitor. So sometimes before you send the image to the display to the monitor you do what's called a gamma correction. So you gamma correct the image, and then you basically send it to the display. And what's basically the gamma correction is doing is inverting this. Darkening operation that happens due to some properties in the monitor. So it's a very useful, very common operation. Basically shows up. In virtually every monitor there is a gamma correction before the image is basically display. But gamma correction or basically this exponential function is very useful in general to basically kind of stretch if you need the pixel values. Remember we have instagrams and. Which are the distributions of the pixels. And, for example, if we look at this image it looks like most of the pixels are concentrated on the bright values. So, if we were to draw, and we're going to do. This. This event. Very often, basically starting, in the next in the next video, but allowed in this way. So, probably, if we were to plug the distribution of the pixel values of this image, there would occupy a lot, this region. Because we look like the image is basically too much of the high values, very bright. So, with gamma corrections, we can manage to actually stretch this histogram and this are just different values, but I think the simplest is to just look at this one. And we see that we have basically a nicer image it looks better than this image first of all. But it also helps us to better understand some of the details and I hope that you can still in your own monitor that it's doing its own gamma correction. Then you can actually perceive better, details here, details here, and in different regions in the image. And sometimes, it might become too much, so you might do a different type of gamma. So you might say oh, this region actually, I'm not. ... Perceiving a lot of the details. So maybe I should a different value of gamma as illustrated here. This are researches of different values of gamma. And you could actually do some of these operations not in the whole image but in some regions of the image. So you can invert only some region of the image. You can do a gamma correction or an exponential function in only some regions of the image. So you don't have to do them every place. You can do them locally. So that's. A very simple operation, that immediately gives us a much nicer perception of the image. Remember the video, as we started this video, was very dark. That means that the distribution, basically peaks of values. Were around here. If this is the histogram distribution and this kind of operation help us to stretch to basically use better the resources that we have in the display to use the whole spectrum of values from 0 to 255 instead of just using the dark part of the values or only the bright part of the values. Again a very, very simple operation. here there is a lot of art in designing of these operations but with experience we understand what we need to do. Here is yet in our example of the stretching of the pixel values. So once again this is the input, this is R, this is S, this is the output. So if your pixel values are between this point and this point not a lot is happening. Okay. So this is going to be the original image. Pixels values that are relatively dark, not a lot is going to happen to it, then. Pixels that are relatively bright. Not a lot is going to happen to them but pickus value in between are going to be stretch basically I'm going to, this has slope much larger than one. If the slope was one, nothing will happen to them. If the slope is less than one, it would be basic contracted. If the slope is larger than one, it would be stretched. Okay? So if we have slope larger than one, zero stays zero. But one basically goes up to the slope and gets stretched. And then we see what happened. This is the retaliament because the mid-range of the pixel values gets stretched. All of a sudden, I can see details that I couldn't see before. Look at the points here are extremely difficult to notice, but here I can see that actually there were spots in this image. And then if you wish you can threshold that image and that's whats happened here so we can distinguish the different regions. But notice the difference of going from here to here, very hard to understand. What's happening here? Much nicer looking image here, not only much nicer looking, but I can also see the details in the image, and that's very important for image analysis, and I can do simple things like thresholding. So everything that's happened here is extremely, extremely simple operations. By the way, this is this. Going from here to here, basically, is also. A very simple operation. We actually saw it before. It's an operation of this tie. Okay? So, this is S, this is R. The sum of the pixel values go to zero if they are below this particular value. And if they are above, they go to the maximal value that they can display in the image. So very simple operation can do drastic transformations in the image. And very useful transformations in the image. A couple of additional examples, here. We have, it's kind of a thresh hold, but in the mid range. So, basically we say, okay, anybody that goes from zero to A, I want it to be very low. Everybody that is above B, I wand it to be very low, and everybody in between, I want it to be very high. And, this is another example. Basically everybody between zero and eight, don't touch. Everybody above B, don't touch. Everybody between A and B make it very, very bright, or I could say, make it very, very dark, okay? So let's assume that you have information that the pixel values of your region of interest are between A and B and you want them to be highlighted in the image, then you do this type of operation or this type of operation. Let's just see the effects of that in the next slide. This is the original image here. And this is the first operation, where we basically take a wrench of the image, and that wrench of the image we'll make very brighter. Very bright, everything else, we'll make very dark. And this is another example. This is the second example where we take a region of the image. And we only change that region of the image. Everything else basically stays the same. Of course, you need information. What's the region that I care of. And that comes, as I say, from experience. Offer information about the sin that you're capturing. In this case. Basically, information about the biology and what kind of pixel values. This region of the image appears. And that's a region that I'm going to basically say that I need to make it brighter or I to make it darker and similarly here. The, the goal of this is to illustrate how very, very simple operations can get us basically what we want. Can get us to improve the visual appearance of the images, going from very dark or very bright to very uniform and nice looking images or can help us to basically highlight the region of interest. If you want to do the analysis of this region in the image, maybe it's nice to basically make everything as dark. So I can concentrate my attention here. But maybe you say, yes. But I lost details, because you made everything white. You say, okay. Then let's do something else. Let's just take the whole. Background, basically, and make it zero. But let's not change this range of values. And this is the type of operation that appears here. We make everything that we don't like just very dark or very bright. But everything we care, we leave untouched. And then we can actually. Concentrate our visual observation with concentrate only on that region. So these are very simple operations. In the next video we're going to start talking about histogram equalization but before that, let's just make a short video that basically run some of these operations in real time so we can see how simple they are and we can actually observe how they are working on our images. So see you on the next video. Thank you.
