2:43

In electronics, for instanc,

you have a lot of equations that describe the inner workings of electronic circuits.

Here you have an RC network, and

the voltage at the output is described by this law of charge or

discharge of the capacitor that would be something like this.

So what makes digital signal processing interesting?

In these two examples you can see that the mathematical model that we used to

describe the signals, is that of a function of a real variable time.

This is a standard model for what we call analog signals.

But if we go back to the motion of a project dial we see that this equation

has only one degree of freedom.

The only thing you can choose is the initial velocity, but

then the shape of the signal will always be the same.

And similarly in an RC network, you can choose the value for the resistor and for

the capacitor, but the signal will always have the same exponential decaying shape.

Now if I show you a wave form like this, you probably have seen this before and

you can guess that it is some sort of representation of a sound or speech.

And if we were to apply this mathematical model to this kind of information.

The question is, what is the function that describes the sound?

Well there's no really an easy answer for that.

What I can try to do is record this information, and

people have invented extremely sophisticated devices to do so.

So for sound, for instance, I could come up with a record player, I could come up

with a tape recorder, and then if I want to measure a temperature signal.

Then I would have to come up with a mechanical system that

drags a pencil on a piece of paper to record the evolution of temperature.

And to capture photographs, I will have to invent a camera.

But you see every device is specific to a certain signal.

It will record any information but

it will not let me manipulate this information easily and in a generic way.

In other words, the recording device will give me something like a picture,

something like this.

But it will not answer the question,

what is the function that describes the phenomenon?

This is the problem with analogue signals.

The big paradigm shift and the power inherent to digital signal processing,

is that we're moving away from an analog model for signals like this.

So we're not asking, what is the function that represents the signal anymore,

we're just moving to a recording of the values of this function.

And represent the phenomenon just as a series of numbers.

And in particular, for digital signals, these numbers are integers.

The so called digital paradigm is composed of two fundamental ingredients,

discrete time and discrete amplitude.

And let's start with discrete time because this is really the paradigm shift in

the way we perceive the world.

When you were in elementary school,

you were probably asked to run a little experiment.

You recorded the temperature every morning, and

then plotted it on a piece of graph paper.

And you obtained a plot that probably looked like this, so

you look at a little point on the mercury scale.

And plot it in the right position on the graph paper.

And when you do this for a number of days, the resulting representation,

it seems pretty reasonable.

It makes sense to measuring the temperature once, or at most, twice a day,

is a pretty descriptive representation of the weather.

But can we really do that, can we really splice up

time into a series of discrete instance, and not lose information?

The question is really tied to the very nature of time.

And this is something that philosophers have been grappling with

since the beginning of intellectual speculation.

One of the most famous analysis of the problem of time was conducted by

Augustine of Hippo in the 5th century.

And his analysis led to, fundamentally, a negation of the existence of time.

Because the past has already happened, the future is unknown, and the present instant

cannot be pinpointed the moment I mention it, it's already become part of the past.

And so Augustine said time does not exist.

Centuries later, one of the last philosophers to try and

systematize the world in philosophical terms, Immanuel Kant.

Solved the problem of time, but says that time and

space are fundamental categories of the spirit.

Fundamentally we are hardwired to project a notion of time and

space on everything that we observe.

But perhaps the best known philosophical reflection on the concept of time,

was performed by Zeno of Elea in 5th century BC.

7:13

Zeno came up with the famous paradoxes and

a dichotomy paradox is one of the most celebrated.

It states that motion can not really take place and the reasoning goes like so.

If you want to move from point A to point B, so cover some distance.

Well in order to reach point B, you will have to go through a midpoint,

let's call this midpoint C.

And even if you go through that midpoint then you will have another midpoint

between C and B to go through, and then another midpoint, and then another, and

then another, and so on and so forth.

Now, Zeno said, no matter how small this subsegment that you can divide A and

B into, you will need a finite amount of time to cross each one of them.

So now you have an infinite number of segments.

And the sum of an infinite number of times,

however small, cannot be finite and therefore you will never reach point B.

Of course, today any student with a little bit of a Calculus

will tell you no because, if you formalize this, you can see that

you're actually taking an infinite sum of a geometric series with ratio one-half.

We know that this sum is equal to 1.

This is not the solution to Zeno's paradox.

The solution to Zeno's paradox is represented

by the last four centuries of mathematical research that have produced

a mathematical model where infinite sums do not lead to contradictions.

Zeno's intuition was very profound.

If we apply a simplistic mathematical model to reality, like for instance,

the possibility of dividing a segment into an infinite number of subsegments,

then we run into contradictions.

So we have to be careful to avoid these pitfalls.

Back to our signal processing problem.

If we start with a model of reality while we have, say for, the projectile motion.

An ideal trajectory that can be model as a function of a real variable T, we are in

a situation similar to the Infinite divisibility of the segment we have a real

number time that can be split into an infinite number of smaller intervals.

So how do we go from this to a model in which we have a finite

number of, we can now call them with their name, samples of the original trajectory?

A set of measurements which is finite in nature.

Incidentally, having the discrete model of reality, would be extremely practical for

a lot of applications.

So for instance, consider this temperature signal between a and

b, if we use a continuous model of reality an analog model and

we want to compute the average temperature,

then we need to know the function f(t) that represents the temperature.

And then, we need calculus to compute the integral of this function over

the interval interest.

And then, divide by the length of the interval.

Conversely, if we only have temperatures measurements on a discrete set of times,

like so for instance, well the average is very simple.

We just sum these values and divide by the number of samples.

This we know how to do intuitively.

And this is valid in general.

Discrete models are extremely easy to use computationally speaking.

So we want to move away from this plutonic ideal functions of a real variable time

11:14

Are we burning bridges?

Are we losing some power of representation when we move to this discrete model

of reality?

So what happens inside an RC circuit?

Can we still describe this physical reality Using a discrete time sequence?

The question is a fundamental importance and the answer was given to use at

the beginning of last century by Harry Nyquist and Claude Shannon.

The answer is positive and it states that under very mild conditions, the continuous

time representation and the discrete time representation are equivalent.

Mathematically, the result is known as a sampling theorem, and

it has a very simple statement.

The relationship between the continuous time a presentation of a signal, and

its discreet time counterpart is given by this formula.

And you can see that we can build the continuous stamp representation as

a linear combination of copies of a typical function or building block

called the sinc shifted and scaled by the values of the discrete time sequence.

The sinc look like so, you probably have seen it

in countless signal processing logos it's actually an infinite support function

that keeps oscillating from minus infinity to plus infinity, and

the sampling theorem graphically looks like so.

You start with a continuous times signal and then you take measurements.

So you convert this to a discreet time sequence.

You take regular measurements of this function and then you

throw away all the rest and you're just left with the discreet time sequence.

To go back to continuous time,

all you need to do is take copies of the sync function and

place them at each sample location scaled by the amplitude of the sample.

And when you do that, and then you sum all these copies of the sync together,

you get back exactly the original function.

The conditions under which you can do this are given by at the statement of

the sampling theorem, and will require a tool called Fourier analysis.

The Fourier transform will give us a quantitative measure

of how fast a signal moves.

And once we know this speed, we will always be able to choose a sampling

interval, namely a space between measurements when we convert a function

to a sequence that will satisfy the hypothesis of the sampling theory.

So we have said that digital signals are composed of two ingredients.

And we have talked, at length, about discreditization of time.

Now, let's look at the other aspect of discreditization of amplitude.

Take for instance this sign wave.

The first discreditization happens in time,

and we get a discrete set of samples.

And then, the second discretization happens in amplitude.

Where each sample can take values now

only amongst a predetermined set of possible levels.

And the very important consequence of discretization

is that independently on the number of levels, the set of levels is countable.

So we can always map the level of the sample onto an integer.

15:07

As far as storage is concerned, just consider the difference between attempting

to store an analog signal, which required a medium

that was dependent on each kind of signal and on each kind of application.

And compare that to what we do today,

which is using general purpose computer memory for all kinds of signal.

So here in this picture, for instance, you can see what we used to do for sound.

We started with phonograph, perhaps wax cylinders and records and then moved on

to tape records and each median was incompatible with the previous one.

Same goes with temperature no way of transmitting the temperature out of them

physically delivering cylinder of paper to weather station and so on and so forth.

Today, all we do is stores 0s and 1s.

And this is a consequence of the fact that the possible values of discreet time

sequence are mapped onto a countable set of integers and

therefore onto binary digits.

There is a famous picture going around the Internet that shows how much

information you can store today on a MicroSD card

compared to the storage abilities of just a few years back.

And this fast evolution of technology is what you get when you can pool resources.

You don't have to specialize into different devices for

different purposes anymore.

And all the research goes into perfecting

a single technology that is shared across a variety of applications.

Processing was also a problem in the past.

With analog signals, you had to devise specific devices that would be able

to react to the physical phenomena that had to interact with.

So for instance, here you see a thermostat that had to had special coil that expanded

and triggered switches in order to regulate the temperature.

Mechanical systems require the design of very complex gears and

sound equalization system required discrete electronics.

Today with digital signals, all you need to do is write a piece of computer code

that will run on a general purpose architecture and

perform the same tasks that required hardware in the past.

Finally, let's consider the problem of data transmission

which is probably the domain where digital signal processing has made the most

difference in our day to day life.

So if you have a communication channel, and you try to send information from

a transmitter to a receiver, you're faced with the fundamental problem of noise.

So let's see what happens inside the channel.

You have a signal that will be put into the channel.

The channel will introduce an attenuation.

It will lower the volume of the signal so to speak, but

it will also introduce some noise, indicated here as sigma t.

18:05

So how do you recover the original information?

Well, you try to undo the effects introduced by the channel but

the only thing you can undo is the attenuation.

So you can try and multiply the received signal by a gain factor that is

the reciprocal of the attenuation introduced by the channel.

So if you do that, you introduce a gain here active receiver and

what you get is start a gain with your original signal, attenuated copy,

some noise added and then let's undo the attenuation.

What will happens, unsurprisingly is that the gain factor

has also amplified the noise that was introduced by the channel.

So you get a copy of the signal that is yes of a comparable amplitude

to the original signal, but in which the noise is much larger as well.

This is a typical situation that you get in second generation or third generation

of say, a tape or if you're trying to do a photo-copy of a photo-copy.

Just to give you an idea of what happens with this noise amplification problem.

19:12

Now why is this very important?

This is important because if you have a very long cable, so for

instance if you have a cable that goes from Europe to the United States, and

you tried to send a telephone conversation over there.

What happens is that you have to split the channel into several chunks and

try to undo the attenuation of a chunk in sequence.

So you actually put what are called repeaters along the line that regenerate

the signal to the original level every say, 10 kilometers of cable or so.

But unfortunately, the cumulative effect of this chain of receiver is that some

noise gets introduced at each stage and gets amplified over and over again.

So for instance, if this is our original signal which again,

gets attenuated and gets corrupted by noise in the first segment of the cable.

After amplification, you would get this, we just did that before.

And this signal is injected into the second section of the cable.

It gets attenuated, new noise gets added to it and

when you amplify it you get double the amplified noise.

And after N sections of the cable you have N times the amplified noise.

This can lead very quickly to complete loss of intelligibility

in a phone conversation.

Let's now consider the problem of transmitting a digital signal

over the same transoseanic cable.

21:20

What happens on the channel is the same as before.

We will have an attenuation, we will have the addition of noise, and

we will have an amplifier at each repeater then will try and undo the attenuation.

But on top of it all we will have what's called a threshold

operator then we'll try to reconstitute the original signal as best as possible.

Let's see how that works.

If this is what we transmit, say an alternation of zero and

one mapped to this two voltage levels, the attenuation and

the noise will reduce the signal to this state,

the amplification will regenerate the levels and will amplify the noise.

So the noise is much larger than before.

But now we can just threshold and say, if the signal value is above zero,

we just output 5 volts and vice versa.

If it's below zero, we will output minus 5 volts.

So the thresholding operator will reconstruct a signal like so.

So you can see that at the end of the first repeater, we actually have an exact

copy of the transmitted signal and not a noise corrupted copy.

23:37

With the digital revolution in the 90's, digital modems started to appear and

very quickly reached basically,

the ultimate limit of data transmission over the voiceband channel,

which was 56 kilobits per second at the end of the 90's.

The transition to ADSL pushed that limit up to over 24 megabits per second in 2008.

Now this evolution is, of course, partly due to improvements in electronics and

to better phone lines.

But fundamentally, its success and

its affordability are due to the use of digital signal processing.

We can use small, yet very powerful and

cheap general purpose processors to bring the power of error correcting codes and

data recovery even in small home consumer devices.

In the next few weeks we will study signal processing starting from the ground up.

And by the end of the class,

we will have enough tricks in our bag to fully understand how an ADSL modem works.

For now let's resume the key ideas that we have seen in this perhaps,

slightly rambling introduction.