In this segment, we introduce the classical restoration problem. According to it, there is a system between the original and the acquired image, which introduces some type of degradation. The system can represent the Hubble space telescope, or the turbulent atmosphere. Or be a combination of the motion of an object in the scene, and the motion of the camera. When the degradation system is linear and spatially invariant. The both the modeling of the degradation and its removal, are, relatively speaking simplified problems. We show the mathematical models for the systems, introducing some of the common encountered LSI degradations, of course there's almost always the ever present noise in the data as part of the degradation. We show how is the severity of noise typically measured in image restoration. As well as an objective metric used for estimating the improvement in the restored image. When a synthetic experiment is conducted. Of the recovery problems we described, such as concealment and painting, resolution. They have a common structure, they're all inverse problems. So, we assume there is a system, which you denote here by H, that takes an image at the input F, and produces an image G at it's output. And it could be a single image or multiple images or a video. When we observe G, and or solving for F, it's a recovery and inverse problem. If only G is known, and still the objective is to recover F, and of course H along the way, then becomes a blind recovery problem. And the in between situation is when G is observed, and H is partially known. So the shape may be of age is known, and but the, the [INAUDIBLE] but the width of the [INAUDIBLE] is not known, so there's a parameter I need to estimate. And in that case, it's referred to a semi blind recovery problem. Although the main objective is again, to estimate F as we'll see in a number of examples. There are a number of parameters that are introduced and they need to be estimated, as well. If f and g are known, and the objective is to find h. Then this becomes a system identification problem and if f and h are known, and. The motion estimation problem is an inverse problem. The disparate estimation, I just mentioned it in passing at some point, it's also an inverse problem, and the bounded detection through differentiation can be formulated as an inverse problem. So inverse problems are encountered are all over the place. It's a topic in Applied Math that he has seen a lot of activity for the long past and it's a very rich literature. There are numerous techniques that can be utilized to solve such inverse problems. The adaptivity on restoration was a result of the space race between the US and former Soviet Union in the 50s and 60s. So for example, the 22 images produced during the Mariner IV flight to Mars in 64, cost them million dollars so this were is the case with most area the activity comes in waves. It's driven, for example, by mishaps, such as the Hubble space telescope mishap, new applications, such as the super resolution of images and videos for these huge ultra high definition displays. And, by new mathematical developments, and sparsity of course the one that stands out and we'll talk about it at a later point. Restoration has seen some not alright in the media. There's a brother field of the assassination of JFK, underwent a number of restorations. And then in a number of movies such as no way out and rising sun, the whole plot relies on the successful restoration of some surveillance video. Let us look now at the components of a degradation restoration system. F is the original input, image. Is input to the degradation system H. Noise is added to the output of a system, and this gives rise to the observation Y. So, here's how the input looks, it's an aerial shot, and here's the blurred version of this by H, plus the addition of noise. So the objective of restoration is to design a system r, which will operate on the observation y, and give us an estimate, of the original image f hat. That will be as close as possible to the original one, subject to the optimal criterion. In carrying out this restoration step, we assume that we have perfect knowledge of h this is the nonblind version. Then we also need to know the noise statistics, and here there's a double sided arrow, or arrows going in and out from our. To indicate that these statistics can be updated based on the partially restored image, for example. And another component which is extremely critical when it comes to restoration, is prior knowledge on f, on the original image. And this prior knowledge can be expressed in various ways and we'll see examples of that as we proceed here. But it's also important that we are able to incorporate such, such prior knowledge in a rather straight forward way into the Restoration Filter. Wnd again there are two arrows,. Going, towards and away from this prior knowledge box to indicate again that this prior knowledge can be updated as the iteration progresses. Now, the noise statistics can be given to us which is idealistic or in most cases we measure them, let's say the noise variance under a Gaussian assumption from the available data y. Similarly if h is not known, then it needs to be estimated from the available data y. This is referred with a blur identification problem and in carrying out such a step, it's also important that we have some prior knowledge on the properties of h. So you do see here all the components that are involved in designing a Restoration system. Again, prior knowledge is of the utmost importance since it restrains the solution set. And produces an estimate here of that is as close as possible to the original image f. As it became clear from the previously slide in carrying out the striation, that blur is needed and knowledge estimate is needed if they're not known, then. There the steps of blur identification, noise estimation, and restoration that that involve. And these three steps can be done independently, using any technique of choice. However, at least conceptually, it should be clear that if their done at the same time, simultaneously. And by that I mean that an error in identifying the blur is taken into account when the noise is estimated, and this error is taken into account when the restoration is carried out. Then one should expect improved results. Stochastic methods, we'll talk about them a little bit of their, an advantage of that nature, that all these steps can be. Taken, simultaneously. And again, repeating here, prior knowledge on all the unknowns. H, f, and n is of the out most importance in providing a solution here, f hat, that is, as, as close as possible to the original image we're after. So translating the operations shown by the block diagram into an equation here we have the observed image y i j is the result of the application of this operator of this system h onto the original image plus the noise that has been added. So this is a very generic, general model that can be found in very many different applications. So the problem at hand is, given y, and given knowledge of H, we try to find an estimate of f and, of course, knowing the statistics about the noise. So this is the Restoration problem. [SOUND] Now, in many applications, this system can be well approximated by linear and Spatial Invariant system and noise of course additive, and signal dependent. And then as we know by now, in this case, the operation of H on f is given by the familiar two dimensional convolution. So the output, the observed image is the convolution of the input with the impulse response of the LSI system plus noise, and here is the expression for the two dimensional convolution. So in this particular incarnation, solving for F, given the impulse response of the system, and the available data, becomes now a deconvolution. Deconstruction problem. The system convolves, and the inverse problem we're solving is to deconvolve the operation of the system, undo what the system did. So restoration is a more general term, applies to this general equation, if the system again introducing degradation is LSI, then I'm solving de convulsion problem. We show here that impulses sponsor some of the most commonly encountered degradation systems. So if the degradation is due to one dimensional motion between the camera and the scene, so the object in the scene move on a plane that is parallel to the camera plane this is a D plane. And then they move in a horizontal direction, then here is the impulse response of this degradation system. So here's this shape. [BLANK_AUDIO] And here we assume that l is an even number so l over two is an integer. The extension of this is if the motion is at an angle, not along the horizontal direction but at an arbitrary angle theta, and that's also something that can be handled with not major difficulty. If the distortion is due to atmospheric turbulence, then the long term exposure through the atmosphere is modelled like this. This impulse is responsible for the Degradation system, so this, this ocean here controlled by the variants of the ocean. If it's an out of focus Degradation, then this disc, in two dimensions, i, j, is, represents the impulse response of the system, so it's a function of the severity of a, the focusing is a function of the radius of this disc. And finally the Pill-box distortion is kind of the extension of the one demotion so it square here from minus element two clone. Two in both dimensions and again l in this particular case is assumed to be an even number so that l over two is an integer. So well be making use in some examples so some of these Degradations. But there again can be found in the literature there rather widely used. In the image restoration community, in assessing the quality of the degraded image, we utilize not the signal to noise ration, but the blurred, signal to noise ratio instead. So here's the model we've been using, the Degradation model, the observed image is the convolution of F with impulse response of the system plus noise. So if we called the Blurred Image here GIJ, then in the num, numerator you see the variance of G bar is the mean or expected value of b. We assume the image has m times n pixels. Therefore, the numerator is the variance of the blurred signal and then the denominator is the variance of the noise. In assessing the quality of the restored image for simulation situations where the original image is available we use this ISNR matrix improvement in signal to noise ratio. So in the b term log base 10 in the denominator of this fraction we'll put the difference the square there between the original f is the original. Which, again, in a simulation environment is available and f hat is the restored, so the distance between the original they restored. So, that's the error in the estimation and in the numerator is the quantity f, the original minus y, the observation. So this arrow squared. So these are two quantities that we will be utilizing for the rest of this presentation restoration, Blurred Signal-to-Noise Ratio and improvement in Signal-to-Noise Ratio.
