In the previous slides, we considered so-called linear operations which can be expressed by using convolution. But for sure, there are other local or special image filtering operations which are not linear and the most famous one is called median filter. In this filter, we compute the median of the pixel intensities in the local neighborhood. In order to compute the median of some sequence of numbers, you have to sort them and you have to choose the value in the middle of the sorted sequence. When we speak about two dimensional case, again, it's the same. You see here are nine different values of pixels in the local neighborhood of pixel with value of 150. If we consider this neighborhood values you see nine values, we just present them in a sorted way. You see that the median value here is equal to 124. In this particular case, what does it mean? It actually means that the input pixel is equal to 150 and the output value of the pixel at this position is equal to 124. We decrease the brightness of the output pixel. Why do we decrease it? Because the brightness in the neighborhood of this pixels are much lower when compared to the brightness of their original pixel. In this particular case, we assume that this value of 150 is some noise and we just remove this noise completely. But median is a nonlinear operation. Here is a really nice example and really simple. You see that if we compute the median of the sum of two sequences with three elements, then the value of median is equal to 6, but if we compute the sum of two medians the sum is equal to 7. It's obviously non-linear operations and it can be computed using convolution if we're using Fourier transform. Anyway, it's a really nice operation and let me show in detail how it works and it is typically used for so-called Salt-and-Pepper noise. When the input image like on the left part of this slide contains some just random noise, which doesn't depend on the values near this pixel. You see in the bottom part some chart for one row of this image and you see that its intensity is changed from 0-155 and vice versa. You see that there are a lot of changes. It's assumed that in good image, this changes are rather limited in pixel intensities of neighborhood pixels. You see that the output of median filter looks much better because we practically removed all the salt and pepper noise here and you see the output for the same role. You see that the brightness values is in this row are more or less identical and they changed smoothly from left to right. Again, there are other kinds of non-linear filters and let me show you the main disadvantage of Gaussian smoothing and let me introduce another filter which gives much better quality. Gaussian filter for sure, smooths noise but its main disadvantage here is that it also smooths the boundaries and the borders of objects. We see here an example in the input image you see the two parts, black and white, and you see a lot of noise, something like, again, Salt-and-Pepper noise but it doesn't matter. Here you see that the output of Gaussian filter is rather smooth. It doesn't contain noise, but it also mixes pixels in the border of these black and white rectangles. How to deal with that kind of issue: We can use slightly more difficult filter. Again, the weights are really similar to what is done in the Gaussian filter, but it contains an additional part. You see the weight definition here and the part which is marked by red, it's domain kernel. Actually it's the Gaussian filter here but in addition to these Gaussian filter or to this domain filter, we add some other multiplier on this exponent. You see, it's also called range kernel. The range kernel depends on the difference between the center pixel and the current pixel vaule of the brightness of the central pixel and the brightness of the current pixel. If these two values are rather different, then this range kernel part with this distance will be very high and the exponent of minus distance will be really low. In this case, the weight will be low and such pixel doesn't have much influence on the central pixel. In this case, you have two kernels, domain kernel and the range kernel and one of them depends on the position of pixel only and the position of the neighborhood pixel and the central pixel. Another one depends on the difference in brightness of the input and of the neighborhood pixel and of the neighbor pixel and the central pixel. For sure if we want to implement the smoothing operation, then again we need to show that the sum of weights will be equal to one so we'll have to divide the output of the filters by the sum of this Omegas. In this equation G is the value for output function of so-called bilateral filter. I will show you some examples of how this filter is implemented for real images but I should say that the running time of this filter is much worse than compared to the Gaussian because we can't use convolution here. Even we don't have the same kernel because this kernel and this Omegas depend on the values of pixels inside the center of this neighborhood and in this particular case, we have to understand that we should compute these weights for each pixels and for each pair of pixels in the neighborhood. It's really time consuming operation and then practice bilateral filter is used with rather small neighborhood, something like seven by seven or 11*11 is the maximal values for the neighborhood and even in this particular case, the running time will be much much slower when compared to Gaussian, to box filter to any filters which we discussed before and even with median filter. The examples are really nice. You see some kind of noisy image here, noisy input image. If we use the bilateral filter with the neighborhood size seven by seven, then you see that practically all noise was removed, but the borders between objects remain the same. Speaking about image processing, it's really nice quality. Let me show some examples of image processing with non-linear filters for binary image. Let me recall that we obtained binary images, for example, when we used image binarization or image thresholding in the previous week. Again, we should understand that after such binarization, the binary image can contain a lot of noise and how to improve the quality of such kind of images. We will use so-called mathematical morphology here. The fundamental part here is so-called structural element. It's absolutely the same as kernel or in the neighborhood in the convolution but these structural elements can be anywhere. It's not only a square neighborhood. You see that we can use square neighborhood in the top left part, but we can use only some cross in the top bottom part. We can use some circle in the center and so on and so forth. Also it can have values equal to one and the some values equal to zero. In general you can use any kind of kernel here but again, it's a binary kernel so it can contain only values one and zero. Again structural element defines a neighborhood of each pixel which will be analyzed by mathematical morphology operations. Let me show you some examples of mathematical morphology and we will use the simplest structural element. It's a square, a five by five pixels. All of these pixels are equal to one. Will consider five by five neighborhood similar to what we consider before for linear operations and for convolution for non-linear operations and so on and so forth. The most important parts of mathematical morphological operators here are so-called erosion and dilation. Let me show you some examples here. I have to highlight that you see that the background of this image is black and foreground of the object is represented by white color. It's not typical in some way. Many other images use white for background and black for foreground, but anyway, these operations can be used for each kind of binary images. Speaking about erosion operation, the output of this operation will be equal to one or to white color only if all pixels defined in the neighborhood defined by the structural element are equal to one. You see that in this particular case, the area of white color is decreased by using the erosion operation and the area of black pixels increased, significantly increased here. Again, if you will use the same operation for other images in which you have black foreground for object and white color for background, then erosion will be implemented in the same way but in this particular case, this area of the object will be increased. Let me show you an example of another operation. It's controversial to erosion, so-called dilation. You see that the output pixel is equal to one only if any pixel in that neighborhood defined by structural element is equal to one. In this particular case, we increase the area of white pixels and decrease the area of black pixels. These kind of operations are only the most simple ones. They cannot deal with the presence of noise in the input image. They have to be combined with each other to remove some noise in input image. Let me show these kind of combinations. There are two operations like opening and closing. When you speak about opening its erosion followed by a dilation. At first you remove noise in the background. If background is black, so you remove all these low white points in the left image. Also for sure, when you use erosion you will decrease the area of the white color of the white part of this image. In order to increase it, to reduce it back to the original size, you will use dilation. At first you remove these small white points and then you return back to the original image and you see pretty nice results. The close operation is implemented vice versa. You first use dilation and then you use erosion and in this particular case, you see that at first you remove all black holes inside the object. Then after this operation, you reduce the space and the area of white part of this image. This operations are implemented in OpenCV functional morphology. Actually, you can have some loop of this operation so we can repeat erosion and dilation, for example, three or five times. It's really typical to remove more complex noise and binary images. Anyway, it's some magic to create a good sequence which will work for many input images. Anyway, it's a really powerful operation which can be used to process any kind of binary images. Let me show you how to produce the binary images using the so-called edge detection in the future lecture.