Fundamentals of Digital Image and Video Processing

481 ratings

In this class you will learn the basic principles and tools used to process images and videos, and how to apply them in solving practical problems of commercial and scientific interests.
Digital images and videos are everywhere these days – in thousands of scientific (e.g., astronomical, bio-medical), consumer, industrial, and artistic applications. Moreover they come in a wide range of the electromagnetic spectrum - from visible light and infrared to gamma rays and beyond. The ability to process image and video signals is therefore an incredibly important skill to master for engineering/science students, software developers, and practicing scientists. Digital image and video processing continues to enable the multimedia technology revolution we are experiencing today. Some important examples of image and video processing include the removal of degradations images suffer during acquisition (e.g., removing blur from a picture of a fast moving car), and the compression and transmission of images and videos (if you watch videos online, or share photos via a social media website, you use this everyday!), for economical storage and efficient transmission.
This course will cover the fundamentals of image and video processing. We will provide a mathematical framework to describe and analyze images and videos as two- and three-dimensional signals in the spatial, spatio-temporal, and frequency domains. In this class not only will you learn the theory behind fundamental processing tasks including image/video enhancement, recovery, and compression - but you will also learn how to perform these key processing tasks in practice using state-of-the-art techniques and tools. We will introduce and use a wide variety of such tools – from optimization toolboxes to statistical techniques. Emphasis on the special role sparsity plays in modern image and video processing will also be given. In all cases, example images and videos pertaining to specific application domains will be utilized.

From the lesson

Signals and Systems

In this module we introduce the fundamentals of 2D signals and systems. Topics include complex exponential signals, linear space-invariant systems, 2D convolution, and filtering in the spatial domain.

So, in summary, what we have seen in the last two slides is that unlike

the continuous time complex exponential which are always periodic

in the time or spatial domain and not periodic in the frequency domain.

The discrete time complex exponentials are periodic in the frequency domain and may

or may not be periodic in the spatial or time domain as we just saw right here.

To illustrate the previous 2 properties of a discrete cosine we show here an

1 dimensional cosine function cosine omega n for values, values of a frequency omega.

Now, omega, in all cases, involves pi therefore,

this cosine is periodic in the time domain.

Time is the horizontal axis.

So discrete time, m, right?

So, for example, here, for omega equals pi over 8, the period is

2pi over omega, which is equal to 16.

The period here is equal to 8, equal to 4, equal to 2 and so on.

So the period keeps decreasing as you move

to the right, therefore the frequency keeps increasing.

So for omega equals 0.

Cosine of 0 is equal to 1.

This is the signal that does not have any other frequency other than the 0

frequency, the busy signal where omega equals

pi over 8 we that the frequency increases.

We see, right from here to here, is one period of the signal.

5 over 4 keep increasing and omega equals pi this is the

highest possible frequency of the discrete cosine.

And as a matter of fact cosine pi m equals to minus 1 to the n.

So the values of the signal keep alternating.

It switches from 1 to minus 1 and back to 1 and so on.

So this is the highest possible variation of the signal.

Now as the frequency keeps incre, increasing

from omega plus pi to 2 pi, right?

We see that the frequency of the variation of the cosine keeps decreasing.

As a matter of fact this and this signal is identical,

because 3 pi over 2, plus pi over 2, equals 2 pi.

So these are two complementary angles and cosine of

pi over 2 equals cosine of 2 pi minus pi over 2 which equals 3 pi over 2, right?

So generally I have cosine a equals cosine of 2 pi minus a.

Alright?

And similarly, these two are the same

signals and these two are the same signals.

So the, this particular discrete cosine is periodic the

time domain because we chose the frequency omega carefully.

And then the other properties that this discrete cosine is periodic in the

frequency domain with, with its, with frequency periodic with period 2pi.

And therefore the range of frequencies that this

cosine can change are from zero to pi.

Zero is the, the lowest frequency pi is

the highest frequency, I move to two pi, keep

decreasing the frequency, and then omega equals 2 pi,

cosine of 2 pi is also equal to 1.

This is the constant signal equals the cosine at zero here.

I've completed the full a full period that way in the frequency domain.

Similarly to the previous slide, we show

here the values of the three-dimensional cosine, cosine

omega1 and 1 plus omega2 and 2, for

various values of the frequencies omega1 and omega2.

So we can see that the frequency's 0 pi over 8, pi over 4, pi over 2 and pi.

And since pi is involved, the resulting cosine is periodic in the spatial domain.

Instead of showing it as a 3D plot, we show this

cosine as a two dimensional gray scale image where white

corresponds to the value one, black to the value

minus 1 and grey to the value 0.

The axis are, should have

this orientation shown here and each of these blocks is an eight by eight block.

Okay?

So if we look at this image for example, then, this shows cosine 0 and 1 plus

0 and 2, so cosine 0 is 1, so this a constant D merged with the value of 1.

If we look at the first row here of images, then they all have

omega one zero, [SOUND] so therefore they show

cosine omega two n two for various values of omega two.

If I look, for example at this image here, then,

this is an image of cosine pi over 2 and 2, right?

For this particular round the, the period is 2 pi

over pi over 2 equals 4 pixels or 4 samples.

Right?

So if I take one line of this image.

And see how it looks.

Then, we see that there's a value of one, followed by value of

zero, followed by the value of minus 1, followed by the value of zero.

So this is one period of the cosine pi over 2 n 2.

And since this cosine is independent of the value of n1 it means that for all

n1s the same value of this cosine will

be through therefore you see this vertical stripes right?

It's the one pixel white so all these values for example here are equal to 1.

And if I look also at this cosine, this represents cosine

pi n2, which is, which as we saw, is equal to minus one to the

n2, so this is the highest cosine in the n2 direction.

I have one and minus one out of [UNKNOWN].

And if I finally look at this cosine here, this is cosine pi n1 plus pi n2.

And you can easily verify that this is equal

to minus 1 to the n1, minus1 to the n2.

So clearly the pixels ultimate between minus 1 and 1,

and this is the highest two-dimensional cosine that uh,we can have.

We are going to encounter these images later

on in the course going to talk about compression.

This will be the basis function signals of the discrete cosine drafts form which

we will use to correlate data in

JPG as well as in media compression.00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000