Welcome back. What do we learn in this segment, is how to find the appropriate function or mapping that will take each and every intensity value of the input image, and map it onto an appropriate intensity value in the output image at the same spatial location. So that our objective of an approximate image is fulfilled. We will discuss, for example, how to take the negative of an image or for what objectives you might need to apply the logarithmic transformation to an image or its discrete fourier transform, or apply the power law transformation. The application of the latter is gamma correction which you will discuss in some detail. We'll also discuss how to stretch the contrast, or increase the dynamic range of an image. Finally, we will discuss how to slice, or extract a range of intensity by using the image. Which, if done appropriately will better highlight important features in the image. We actually perform this slicing not on the intensity values directly but on the bit plane's of the binary representation of each intensity value. The material of this segment represents simple tools which however, can prove to be very powerful in visualizing an image. Or providing the enhanced image as input to another processing step such as button matching or recognition. So let us then precede with the coverage of this fun material. So let us look first at Point-Wise Intensity Transformations. We are operating directly on the intensity values of an image, but we are processing each pixel separately and independently of the values of the neighboring pixels. So the general expression for this transformation is shown here. x(n1,n2) is the intensity of a pixel of the input image, and y(n1,n2) is the intensity of the output image at exactly the same pixel location. For emphasis, we've put the subscript point here, just as information to again indicate this fact that I only need one Intensity value on pixel location is the input to find and test the value at the corresponding pixel location at the output. More general case, you might say, is when now x and y are not scalars, but they're vectors. So this is the case for example, one idea with a [INAUDIBLE] so X is a vector, three by one vector, for example the LGB channels of the input image, and the LGB channels of the output image. Actually here, we might have cases where let's say, X is a vector but Y is a scalar So I transform one color image into a grayscale image, and the other way around, X is a scalar and Y is a vector, so I'm doing pseudo-coloring, I'm turning a black and white image into a color image. The generalization you might say of all these expressions is shown here. So first of all, N here is a vector 3 by 1 or 4 by 1 vector, if I'm dealing with a three-dimensional, four-dimensional image, and also X and Y are shown here as vectors with understanding, as already explained, that not both of them need to be vectors. One of them can be vector and the other a scalar to cover the cases I just described. Now these point-wise intensity transformations can be linear or piecewise linear. Or in general can be non-linear. And you're going to see example of all of these cases in what follows. Let's look at some examples of point-wise transformations here. First of all, on the horizontal axis, we have the intensity values of the input image. They range from 0 to 255. Assuming that we're using 8 beats per pixel. This, however, without lack of generality, if more beats are used then the range changes. On the vertical axis, similarly, we have the intensity values of the output image also ranging from 0 to 255. So the first transformation we see here is the identity transformation indicated by this diagonal. It does absolutely nothing to the intensity values of the input image. They are mapped exactly the same into the output image. The second linear transformation is the negative transformation. It maps the value 0 to 255 and the value 255 of the input of the input image to 0. So it gives us the analogy between a negative film and the print for those of you who are old enough to remember films. Then I have the log transformation in red. And the nth root transformation in this solid blue. What these four transformations that lie above the diagonal are doing, is they expand the low intensity values of the image, while they compress the high intensity values. So, if we look at the log transformation, for example, then clearly this range of input values, low intensity values, are mapped into much larger range of output values, therefore we have this expansion. The remaining of the values here, the higher values are mapped into my smaller range of values of the output, therefore, we have this compression. Let's look now at the two transformations that are below the diagonal. We have the inverse log here. And the nth power transformation indicated by this curve. So the transformations below the diagonal do the reverse of the transformations above the diagonal. In other words, they compress the low intensity values while they expand the higher intensity values. And this can be seen, let's say with the inverse log here, this range of values of the input image is mapped into a much smaller range of values of the output image. While these small range values, high intensity values of the input, are mapped into a much larger range of values at the output image. So low intensity values are compressed, high intensity values are expanded. Let us now look at some examples of these transformations. Here's an example of the negative transformation of the intensity of an image. Here is the positive image, and here is the negative image. This is a CT scan, of the ruptured abdominal aortic aneurysm, so it's ruptured AAA, as it typically referred to. So the aneurism is this ballooning, this dilation of the aorta. If it exceeds a certain diameter it's called an aneurism. So here it's a ruptured aorta, shown here, while the second arrow here indicates the free flow of blood. So for the specialist, the negative image might provide a better visualization of the condition, or be more useful in the sense that certain conditions can be easier to be diagnosed. Let us look now at an example of the application of the logarithmic transformation to an image. Here is the mathematical expression of the transformation. I take the log of the input intensity. And y(n) is the output in intensity. We are adding this value of 1 here, so that the log is defined, since log of zero is minus infinity, by adding 1 for that case when x is zero, log of 1 is equal to zero. In addition, clearly now the output of the log is an non-negative number. We have also this multiplicative factor here, c, to kind of adjust this transformation. One of the important applications of this log transformation is when we want to display Fourier Spectra. Here is an endoscopic image. And here is the magnitude of its discrete Fourier transform. We can hardly see anything. There's only a non-zero point here. This is actually the center of the spectrum. So this is the 0, 0 point. So what happens, clearly, is that there is a large value, or a few large values, around 0, 0, while the rest of the values of the spectrum are very small. So when I map this range of intensities linearly into the 0 to 255 range, so that they can display it here, the small values become very, very small, very close to 0. And the large values map to 255, so it's the white spot I see here in the center. So instead of displaying the magnitude in a linear fashion as I explained, if we take the log of the magnitude of the DFD we see the image here. So, clearly there's considerably more information. Of course, I see the 0, 0 frequency here in the center is still displayed. But due to the fact that the low intensities were expanded, they're now visible. And the high intensities were compressed, so they can all now, all these intensities can fit into this 0 to 255 range I used at the output image for displaying purposes. And clearly, visually there is considerably more information in this image than in this one. Medically speaking, the same values exist in both images, but again, for displaying or visualization purposes, this log transformation improves the quality of the image considerably. The power-law transformation is depicted here. Again on the horizontal axis are the intensity values of the input image. They range from 0 to L- 1, so if K bits per pixel are used to represent intensity values, then 2 to the K = L. Similarly, on the vertical axis, we show the intensity values of the output image. The mathematical expression for the power-law transformation is shown here. So the output value equals a constant times the input value raised to a power. The exponent there is denoted by gamma. The curves shown here are for c = 1. So therefore, gamma equals one, I have a straight line here along the diagonal. So this is the identity transformation, it maps the intensity values of the input image to the output image with absolutely no change. For gamma less than one we have the curves above the diagonal, gamma greater than one the curves below the diagonal. For gamma less than one, we have the similar behavior we had, let's say, with the logarithmic function. That is, small intensity values of the input are expanded, While high intensity values are compressed, in going from the input to the output image. The reverse takes place for gamma greater than 1. A variety of devices used for image capture, printing, and display respond according to this power-law. Since, by convention, the exponent of the power-law is denoted by a gamma, the process used to correct the power-law phenomena is referred to as the gamma correction. So for display devices, the relationship between the input voltage or current and the intensity follows this power-law. So we can have intensity that's a display. Intensity equals the input voltage raised to the gamma power. Since the gamma is, in general, different for this device, this is the reason that the intensity and the color of an image might look different on different monitors or when it is projected on a screen. We show here an example of gamma correction. Here is an original image. It's an intensity ramp, so the intensity increases as you go from left to the right, that you would like to display on a monitor. The gamma of the monitor is 2.5, and therefore the displayed image looks like this. Raising each value to the power of 2.5 compresses the range of small values and expands the higher values, so the end result is this darker looking image. Our objective is that the displayed image on the monitor looks very similar, exactly the same ideally, to the original one. So if before sending it to the monitor we collect the image with the gamma, which is equal to 1 over 2.5, which is equal to 0.4. Then this image going into the monitor will appear as shown here. This is the corrected image and in this particular case, since we know exactly the gamma of the monitor, we're able to perfectly correct for this gamma, and therefore the original here and corrected images are identical. The challenge, clearly, in practice, is to find the exact value of the gamma of the monitor, or the display device in general, or the projection device. Here's another example of the application of the Power-Law Transformation to an image. This is an image obtained by the Hubble Space Telescope of the Ring Nebula. This image is rather dark and in some sense it should remind you a bit of the spectra that we talked about earlier. So, we would like to transform it so that more information is visible. And if we apply the Power-Law Transformation with a gamma equal to 0.5, then as we know, the low intensity values will be expanded and the high will be compressed. And here is the result of this transformation. So, clearly a lot of the small intensity values are now visible, and this again should remind you of the log transformation we perform to the spectrum of an image. This is yet another example of the application of the Power-Law Transformation to an image. We see here an image that has this appearance of being too bright, or too light. Therefore, we would like to transform it and make it look darker. This is, by the way, the engineering building here at North Western back in the 40s when Lake Michigan out here was coming all the way to the front of the building. We know, therefore, that to make this image look darker, the gamma we should apply should be greater than one. And if we use gamma = 2, then the transformed image looks like this. So this is an image that probably is more pleasing, or it balances the intensities better than the original one. But again, this is an enhancement problem, we have no objective criterion that we are optimizing and obtaining image. We are giving an image like this and we would like to enhance it. What's wrong with this image? It's too dark, it is hard to see bright spots, it seems like details are hidden under this darkness. So we're going to stretch it's contrast and we gonna do that in the linear way. I just want to use a linear function, a straight line to do so. So, what one can do is just, Take this input image and find the minimum and maximum value, so XMIN and XMAX. And then map this to the full range of the output image. So the output image will have intensities from 0, let's write it in general, 2 to the B-1. So, B bits per pixel are used. So eight bits, this is 0 to 55. And then we would like to map the maximum value of the input image to B-1 and the minimum value to 0. So therefore, we would like to map according to this straight line here. The equation of this straight line is given here. It's straight forward to find this equation. And if I use this linear mapping, on the input image, what we obtained, is this output image. So clearly, the x marks of the input is now mapped to 255, it's an 8 bit image, so the highest possible value, so you can see clear whites now. And you have stretched, you've mapped a smaller region of the input here to a larger range of output values, so you have stretched the dynamic range or the contrast of this image. So, certainly visually, there's considerably more information that can be observed in the processed image over here, the enhanced image. Here is yet another example of image enhancement. It comes under the name of Dynamic Range Expansion. We called the previous one Contrast Stretching. They all express the same idea. So again, what's wrong with this image? It's dark, it's hard to see details, it's hard to see changes, differences between black and white. So in this case, I'm allowed to use a transformation that it is piece-wise linear. So, if I use this transformation of this image, so it has three pieces, this transformation linear pieces. And we see that I compress the low intensity values as well as the high intensity values while I expand the values in the middle. Right, so the range 95 to 159 was mapped to the range of intensity values 31 to 223 in the output image. So, if I apply this transformation, I obtain this image here. So, it's the clear that the definition of the clouds in the sky is much more pronounced and clear now. The waves here and the white waves, and so on. It's a much more informative image, as well as more pleasing to see. So, a piece-wise linear transformation like this, I have two control points, this one and this one. And, by adjusting them appropriately, I can observe the effect of this transformation on any input image. Ideally we would like to have an interface here that will allow you to dynamically adjust again these two control points, and for each position see the output image and of course, keep the resolve that it is the most informative, the most pleasing for the viewer. Another piece-wise linear transformation is Intensity-Level Slicing. In this case, we are interested in highlighting or enhancing a certain range of intensity values of the input image. Applications may include enhancing features in satellite imagery, such as masses of water, lakes, rivers, or enhancing the flaws in an x-ray image. Two approaches to do so are shown here. So here on the left, we map a range of values of the input image into a specific level. Level, let's say, A here which could be equal to 255 while the rest of the input intensities are mapped to this value B which could be 0 as well. So, clearly, in this case, a binary image is generated. With the second approach shown here, a similar approach is taken when it comes to the range of values of interest. This is the range of values we want to enhance, or highlight. However, for the rest of the values, we keep them unchanged. So this is the identity transformation that is applied to them. We show an example of the Intensity-Level Slicing enhancement technique on this angiogram image. So with the transformation as shown here in how the image is normalized between zero and one and then range of value between point 0.2 and 0.55. Let's say this is 0.2, this is 0.55. These values are going to beam up to one, and the rest will be going to zero. So this is the transformation that was applied to this and this on this one. It can be argued that some of the blood vessels are easier to pull out to pick up in this intensity-level sliced image. We can also [INAUDIBLE] in this image, by applying the same transformation but now the background values remain unchanged. So [INAUDIBLE], is the 11 point, then we pull out the values between 0.2 again and 0.55, but the remaining of the values remain unchanged. And again, the same comment can be made here that for the specialist possibly additional information can be perceived by looking at either this or this image than at the original one. Instead of highlighting intensity-level ranges, we can highlight the contribution made to the total image appearance by specific bits. So if we consider an eight bit image, it can be considered as being composed of eight one bit planes. Bit plane 1 here formed by the least significant bit, all the way to bit plane 8 formed by the most significant bit. Clearly each of these planes being in one bit plane is binary plane, it only has two values. Let us now see an example of how these different bit planes manifest themselves. We show here an 8-bit grayscale image of a one dollar bill. The eight binary planes that can be derived from this are shown here. So this is bit plane one, bit plane two, three, four, five, six, seven, and eight. It should be clear that the two or three higher order bit planes contain a significant amount of the visually important information. On the other hand, the lower order bit planes contribute to more subtle intensity details in the image. We can actually obtain this bit plane eight here, by binarizing directly the original image, by mapping the intensity values from zero to 127 to zero. And the values from 128 to 255 to one. If we do so, this binary image will result. The composition also ties to compression that we will be discussing later in the sense that it demonstrates which of the bits are important towards preserving the original appearance of the image. Or, which plays an important in reconstructing the original image. So, for example, let's look at the next slide. So, we show here reconstruction by using the bit planes eight and seven. The construction is done by multiplying the big [INAUDIBLE] of the nth plane. So nth plane is multiplied by two to the n minus one. So the pixels in bit plane 8 are multiplied by 128. Bit plane 7 multiplied by 64, and then the results are added up to obtain this image shown here. Let's compare actually to the original image. So it's clear that the main features of the original image are restored, however the constructed image looks rather flat. Especially in the background. And also there's this false contouring appearance in it. This is not surprising because the two planes that were used can only produce four different intensity values. So if we add one more plane, so now we're using planes eight, seven, and six, we see that now the reconstructed image looks considerably closer to the original one. And by adding also bit plane 5, we can say that these two images are almost indistinguishable. Actually, adding more planes will not help and from that we can conclude that four bits in the image convey all the useful information about the original image, and then, since you're using four out of eight bits, we are achieving a 50% compression.
