Welcome back to the Image and Video Processing class. In the previous video, we saw a number of examples that show what image and video processing can do for us. Now we are going to see a few additional examples. We're going to start again from consumer images, images of the type that you are familar with, and then we're going to move to a couple of examples of medical imaging, an area where image and video processing has been extremely successful and a very important area by itself. So once again, a couple of additional examples of consumer images and videos. So this is a very interesting example of what video processing can do for us. Let us watch this movies for a second. So here we have two kids in one video and two kids in another video doing exactly the same activity. We want the computer to automatically identify that, although, these are very different people, very different backgrounds, in completely different jumping gear, and a house in one, and no house in the other in the background. We want to make sure the computer understands that in spite of all those different things, the activity is the same activity. Kids keeps jumping. We want the computer to actually be able to say that that activity is very different than this one, which is actually a person playing with a ball, but this particular one should be identified to be exactly the same at this other one, which is another person playing with a football. So, we want the computer to automatically classify these videos for us based on what activity is happening in the videos. Let's just see another very interesting example. I'm going to play this video in a second. If you see different frames surrounding the person, so green and red, that would mean that the computer has identified that basically the person is doing a different activity. If you see exactly the same color of the frame, so maybe both green or both red, means that the computer has automatically identified that the person is actually doing the same activity or the same type of activity. So let's watch this movie. So different colors, that means different activities. And now, same colors, meaning same activities. And one more example. In this case, we actually have three different persons at, at three different rows and we have in the left, the grand truth. That happens very often in this type of [INAUDIBLE] in image and video processing where you know what you expect, so you can verify if your algorithms are working properly. On the right, we actually have the actual recite of the algorithm. And once again, if you see frames of the same color, mean, means that in spite of the fact people are the different people, that computer has identified that they are doing exactly the same activity. If you see different frames, different colors, means the computer has identified they're doing different activities. So let's just watch the movie. So look that three of them are green and now this move to yellow which means that the computer understood that the person changed his or her own activity, move from running to jumping. So that's another thing that we want the computer to do for us, to identify what's happening in the images. Sometimes we actually want much simpler things. For example, here, we might just want these very poorly acquire image to become this very nice image. This is something that happens to us all the time. We take an image that is not, it doesn't look as good as we want it to look. For example, this, this is an extreme case, We're going to understand what happened actually in this image and we want to automatically, after we have taken the image, we don't want to go back and take again that image. We actually want the computer do automatically for us give us a very nice image. These aren't just examples of things we are familiar with. A more complicated example is here, yet another movie, but basically what we want is to be able to identify everything that is happening in the movie. This is sometimes called computer vision and not image processing, although, the two areas are close really, really, really close related and we are going to discuss that during the next coming weeks. All this has been about consumer imaging. Image and video processing, in particular, image processing, but also a bit more in these recent days and recent years, video processing is extremely important in medical and biological images. And let me give you a couple of examples that are very exciting applications, where image and video processing can actually make tremendous impact in society and health. So this is a particular application. We're going to talk about this application, actually, the last week of the class. And, this is a particular application that is related to neurosurgery. I'm going to describe this application again, as I said, in the last week of this class. But the basic idea is that the neurosurgeon implants in the brain an electrode and sends some excitation to that electrode, and this is extremely important for many, many problems, like Parkinson, tremor, depression or or, or other problems. But a fundamental problem is that a neurosurgeon needs to see inside the brain, needs to know where he or she is going inside the brain. And in order to do that, we actually, in order to know where to implant the electrode, and where exactly to go. The neurosurgeon will take images of our inside of the brain through MRI, CT, or other disciplines, and try to get a map, try to get basically what we see from Google Maps or from other images. They want to know, have a map from inside our brain. So they want to be able to know everything that is happening in our brain here. So one of areas that they want to implant electrodes, we find it here. So, this is the, this fancy and beautiful in color images that you see here is a combination of multiple modalities that are used to look inside our brain. They are put together with techniques that we are going to discuss, and then, you can create these beautiful images that localize and also tell you how different parts of the brain are actually connected. Once you do that, you can actually map, as I mention, you can actually map the brain and understand these are actually electrodes that were implanted and you can localize where exactly is the electrodes are touching in the brain. So it's really seeing inside our head and a lot of image processing needs to go behind this to make this possible. And, as I say, this is a very exciting area, because the contribution of new image processing here is tremendous. Similarly, we can look HIV research. Again, we're going to discuss this more, at the end, the last week of class. We're going to have entire lecture of this particular problem and the image processing challenges. So this is a, basically a virus and this is a schematic of an HIV virus. And for reasons that we're going to discuss later, we need to identify the actual shape of this, this is schematic. You might say, why is this so difficult? I'm going to show you that in just a second. The basic, idea is that we need to be able to identify that what I've just marked to you looks like this. This is a three-dimensional rendering of exactly that what's marked in the picture as the gp120 and the gp41. So that's what called the envelope and that's a very important part of the virus that a virus uses to basically latch into the next cell and identifying that shape is, is very, very important. What's behind the scenes? So why is this so difficult? Because actually what you get is images like this. basically this is what the schematic of what I showed you before, and these are, these small envelopes I showed you before. Okay? And from images like this one, you need to be able to do reconstructions like, like the ones I'm showing you now. Okay? So this is basic science that has a tremendous potential contribution into the development of vaccines or the understanding of different viruses. Finally, sometimes image processing, as we saw that, that it can help neurosurgery can help others kinds of surgery to make them safer. This is a particular application where it's important to understand where the esophagus is. So, cardiac ablation which is a very common surgery, when it's done, it avoids basically touching it and maybe damaging it. So once again, in our application of image processing, which we're going to discuss during this week. We are going to discuss different type of applications. We're going to touch on some of them. We won't be able to touch on all the medical imaging applications out there, but we're going to touch enough to give you an idea of why image processing is so important in the area of medical imaging and biological data. So to conclude, we have, in these two videos, we have seen a large number of examples going from scientific data from Mars to consumer videos and images like in the movies or like images that you, we take with our own cameras all the way to medical imaging. And we're going to discuss during the next, nine weeks starting with this week, the basic techniques that we need to be able to achieve all what, we've just seen that we want or we need to do with images. So, see you in the next video.
