Hello, and welcome back. This week we are going to be talking about image and video inpainting. In contrast with the topics that we described last week, where we performed a description of fundamental tools to solve a number of problems, like an isotropic diffusion, active contours, and even image inpainting. This week is about a particular problem. It's not about tools for multiple problems, it's about solving one particular problem in image and video processing. Of course, some of the tools that we are going to be using for this might be used for other techniques, and we often learn from the solutions of a problem towards another problem. But it's a very important difference between providing for the mental tools for solving multiple problems, and solving one particular problem. So what is image and video inpainting? And let's start with image inpainting. Image inpainting is basically the art of changing an image in a non-detectable form. So here we see a painting that has a lot of degradation, and here is the restoration of that painting. This was of course done by professionals manually. It's a painting, it's not a picture, was not done in the computer, although the computer can help as well, as we're going to see it later on. When you go and see this picture in the museum, you'll believe it's the original picture. Some experts actually might be able to notice the difference, and be able to see the regions that were actually inpainted or filled in with information from different regions. But the basic idea is that in inpainting, we show you an image or a video, and we make you believe that's the original one. In this case, it's to solve degradations. Inpainting, it's sometimes also called fill in, because we're filling in the regions of problems, the regions of deterioration with informations from this surrounding. And we are going to see that, sometimes, with fill in with information from the near areas sometimes with information from far away areas from the whole image. There's even techniques that do image in painting using database of images where you look for similar things. So, this is one example of what image in painting needs to be done. Remember, it's modifying an image in a non-detectable form. And when we say non-detectable, we say at least by the non-experts. Here is another example. A lot has been done to this image, basically to transform it into this one. Of course it has been inpainted, you see these regions have been filled in. But also there were color corrections and other things, but this illustrates yet another example of image inpainting. Look at all these regions that have missing information and that were basically restored, inpainted, filled in with nice information that looks like natural in the surrounding region of the image. Of course, it's natural to extend these to regular photography, in particular when we were talking about pictures, and not digital pictures, but analog pictures, and here we see examples. Some of examples like here have a few additional restoration techniques, but we see these. For example, the picture was turned into two pieces, then was put together and to repair that, you basically see that it was inpainted. A region of missing information or undesirable information was filled in with other types of information. We see here the same. There is a scratch. Now the scratch is gone. And here there is a similar example like this. Basically, the picture was folded, and when you unfold it, you see this kind of scratch in the middle and then here its gone. So, these are all examples of image impending to restore basically the image to a much nicer looking image. Of course, this is an example that we saw in the first week, where image inpainting is done with a completely different target. In this case, and this illustrates the idea of modifying an image in an undetectable form. In this case, we see nothing, nothing wrong with this image, that we know, if you remember from the first week of class, that basically this was the original image. So what was inpainted now is the object that the user wants to remove from the image. So basically, this region becomes the region of missing information that we are going to inpaint in, and we need to recover the water, and we need to recover the column. So we need to recover texture, we need to recover structure, straight lines and water that is basically non-straight lines. So very different types of informations need to be filled in, inpainted, in order to solve this problem of removing objects. And we discuss in the first week, this is used all the times in the movie industry, I show you a few examples of objects that need to be removed in the movie industry. Here is yet another example from this era, where basically we see here, a picture of Lenin, and a picture, and next to him there is Trotsky. And here, basically, Trotsky has been removed from the picture. So once again an object has been removed from the picture. And, yeah, you can have multiple examples of these. For example in this website, or the website that was linked in the previous slide. So again inpainting in order to remove objects that from the image or from the video. I want to explain something that inpainting, in its basic form, cannot do. I don't want to, you have the wrong understanding that inpainting is an extremely smart technique that can fill in with whatever you want. So let me use this example to illustrate that. Here, this person was removed, and then it was inpainted. So when you look at this image, it holds everything that inpainting is about, modifying the image in an undetectable form. But look what happened here. When the person was removed, two chairs were brought in. This is done by an expert. It's not done by the computer. There's no way for the computer to understand that what you want here are chairs. Maybe in a very, very advanced technique, when you basically provide a lot of side information, you could, in the computer, with programs like Photoshop, cut chairs from here and put them here. And this is the kind of stuff that was done here. So it's a cut and paste, and we are going to see how to do that. But basically, inpainting, does not have that high level knowledge of what needs to be there. The computer and the user can interact to basically bring that into the scene, as we're going to see later on. But if you don't interact, if you just say, do something with this region, the computer does not know what is that you want, why is that you want chairs there, and not, for example, green, not, for example, some of the background. But with interaction with the user, we might be able to do this, kind of a cut and paste. So, this is about image inpainting. In the next video, I'm going to show you that we are actually familiar, in nature, with image and video inpainting, and that's actually very interesting. See you in the next video. Thank you very much.
