>> Welcome to module one of Digital Signal Processing.
In this module we're going to see what signals actually are.
We're going to through history, see the earliest of examples
of discreet time signals, actually it goes back to Egyptian times.
Then through this history see how digital signals for
example, with the telegraph signals, became important in communications.
And today, how signals are pervasive in many applications
in every day life objects.
For this we're going to see what the signal is,
what a continuous time analog signal is, what the discreet
time continuous amplitude signal is and how these signals
relate to each other and are used in communication devices.
We are not going to have any math in this first module,
it's more illustrative and the mathematics will come later in the class.
>> Hi, welcome to our Digital Signal Processing class.
In this introduction we would like to give you
an overview of what digital signal processing is all about.
And perhaps the best way to do that is to consider in turn what we mean
when we use the word signal, when we use the word processing or the word digital.
And you will see that digital signal processing is
really an intermediate point in a reflection about physics,
about math and about the reality around us that started
a very long time ago and continues to this day.
So let's consider the concept of signal to begin with.
In general a signal is a description of the evolution of a physical phenomenon.
This is best understood by example.
Take the weather for instance, the weather is a
physical phenomenon that we usually measure in terms of temperature.
So temperature becomes
a signal that evolves overtime and that
represents a measurement of the underlying physical phenomenon.
We could've chosen another variable.
For instance, we could've chosen rainfall.
that would constitute another signal related
to the same underlying physical phenomenon.
Another example, easy to understand is sound.
Sound can have very many origins take for
instance some musical instrument or a person singing.
Now when you measure sound
with a microphone for instance, what you're measuring is
the pressure, the air pressure at the point of measurement.
The microphone translates the air pressure into
an electrical signal that represents the sound.
Now if you want to record the sound on a magnetic tape for instance, you will have
to convert this electrical signal to a magnetic
deviation that can be impressed over the magnetic tape.
And again, these are different
representations of the same underlying physical phenomenon.
Taking a photograph is a very similar operation.
In this case, we're mapping the light intensity of a scene onto gray levels, in
the case of a black and white
photograph, that can be recorded by photographic paper.
The only difference is that in this case we're
mapping the signal over space rather than over time.
To make things more tangible let's go back to an
experiment that most likely you carried out in elementary school when you
first learned about experimental procedures and
analysis of the world around you.
You were probably asked to map the daily temperature for say,
a period of a month, and to chart it over graph paper.
And so you dutifully looked at the thermometer every morning and then at
the end of the month you probably ended up with a graph like this.
So here we have two concepts
that are fundamental to digital signal processing.
The first concept is that the temperature measurements are taken at
discrete moments in time and they constitute a finite countable set.
And the second similar observation is that
the range of temperature is actually subdivided into
a finite number of possible values which are
determined by the resolution of the graduating scale
on the thermometer.
So, we look at the height of the mercury column
and we chose the tick that is closest to that level.
But, nonetheless, the number of ticks that we can chose from is finite.
So the two fundamental concepts here are the
discretization of time due to the fact that we
take observations regularly but not continuously and the discretization
of amplitudes due to the fact that our measuring
device has a finite resolution.
Now the discretization of amplitude is usually treated as a precision problem.
We can use more sophisticated instruments and achieve a better precision.
But the discretization of time is a veritable
paradigm shift in the way we think about reality.
So much so that the problem appeared for the first time over 2500
years ago when the great Greek philosophers started to think about
why is it that we perceive reality the way we do.
And the first character in the story here
is Pythagoras, who in 500 BC, maintained that
most of reality, if not all of reality,
could be described in terms of numbers and measurements.
Think for instance of the Pythagorean theorem,
if one draws a right triangle, one can
verify experimentally with the ruler that the square
built on the hypotenuse is equal to the sum of the squares built on the sides.
But what Pythagoras said, is that this is a universal
property that applies to the abstract class of all right triangles.
And this was really a major change in
the way people started to think about abstract concepts.
Pretty much the same time Parmenides,
another character in our story, brought this
line of reasoning more into the waters of metaphysics by planting the seed of
a fundamental dichotomy, a fundamental difference between
the reality that we can experience with
our senses and an ideal reality that we will never be able to know.
This of course was later developed by Plato into
a full fledged philosophical theory of the ideals.
So that even today when we talk about the Platonic ideal, we refer
to some form of perfect reality that lies beyond the veil of appearances.
But more interestingly for us is the fact
that right when this idea started to appear in
the global consciousness of the time, there were philosophers
that were ready to point out the potential pitfalls
of this new abstract models of reality. And the leader of the pack, so to speak,
was Zeno of Elea whose paradoxes have survived to this day.
One of Zeno's most famous paradoxes is the paradox of the arrow, which states that if
you shoot an arrow from point A to
point B, the arrow will never reach its destination.
And the reasoning goes like so,
well if we modeled reality with the concepts of geometry then
we know that any segment can be divided into smaller segments.
So what Zeno said was that the arrow, after leaving point a, and before
reaching point b, will have to travel through the mid-point between a and b.
Let's call this point c.
But now after it has reached the midpoint between A and B
it will also have to pass through the midpoint between C and B.
Let's call this point D.
And so on so forth, for every interval you can always find an extra midpoint
and the arrow will have to cross
all of this midpoints before reaching it's destination.
But because of the geometric modelling
of reality there is an infinite number of midpoints and so Zeno said, well in order
to cross an infinite number of points, you will need an infinite amount of time.
Of course today we rebut such an argumentation
by saying simply that we can express the length
of the segment as the sum of all the
sub segments and that this sum converges to one.
But this is a false answer to the problem
because the problem was never with computing the sum.
The problem was with a model of reality in
which the infinite and the finite are at odds.
And it took over 2,000 years of mathematical and philosophical
research to amend that model and come to today's model.
A model in which the sum of an infinite number
of terms can indeed converge to a finite quantity without contradictions.
Now you see the relevance of this problem
to digital signal processing where we are measuring physical quantities at regular
intervals in time while assuming that
the underlying physical quantity is actually continuous.
One of the reasons why arriving at this better
model of reality took over 2,000 years is that in
the Middle Ages, as you can see from this picture,
people were concerned with much more mundane tasks than refining
a mathematical model of reality. But progress did come in the end and the
two towering figures of the 17th century, in this sense, are Galileo and Descartes.
Descartes, the inventor of the Cartesian plane, started by put a name to think.
So if you have a point on the plane like so, Descartes said, well,
if I use a coordinate system around this point I can give a name
to this point and I can use algebraic formulas
to describe geometrical entities and perform operation on them.
So for instance, a line would map to a first degree equation.
This allowed Descartes to solve algebraically, geometric problems that had
baffled Greeks, such as for instance the trisection of the angle.
Much more importantly for us is the fact that the Cartesian Plane is the grand
daddy of all vector spaces and you will see how
useful vector spaces are in the context of digital signal processing.
Well, Europe in the 17th century was flush with
money and Europeans had two things on their minds.
Finding new markets and winning the war of
conquest that came with the appropriation of new markets.
Calculus, that was invented in those years, purported to
provide a new answer to both problems, in the sense that you could use calculus
to find optimal ship routes around the
globe and to find optimal trajectory for cannonballs.
Galileo, in particular, worked on the cannonball problem.
And operated by running a series of experiments in which
the trajectory of balls thrown by a cannon was experimentally determined.
And then working backwards to derive an
ideal Platonic model of the balls trajectory.
That is given by this equation where the initial velocity, expressed as a vector in
the Cartesian plane, is coupled with the pull of gravity to give a parabolic shape.
So the way science proceeded was by starting from set of
experimental data points and then work backwards to find the description
of the underlying phenomenon in the form of a perfect algebraic equation.
This usually worked very well for astronomy, which was a main concern
in those days, because the trajectories of the planets are perfect conic curves.
The invention of calculus and the availability of models for reality
based on functions of real variables, led naturally to what we call
continuous time signal processing.
So if you have a function like this which is
for instance, a temperature function, you can compute the average,
in continuous time, by taking the integral of the function
over its support and dividing by the length of the support.
Without calculus what you would have to do
is take daily measurements say of the temperature.
And to compute the average you would just sum this values together and then divide
by the number of days.
Now the question is what is the relation between these two averages?
What is the error I incur if I use experimental data rather
than finding the ideal function behind the data and then computing the integral.
And even if I can do that for certain signals that appeared to be smooth
and slow like this one, can I do the same if the signal is fast?
In other words, if I have a set
of measurements for something that appears to move quite
quickly, do I have any chance even at
recovering the ideal function that lies underneath the data.
Well it took a long time since the
17th century to answer this question because one
of the missing pieces of the puzzle was how to measure this speed of the signal.
The answer came from Joseph Fourier,
the inventor of Fourier analysis.
That showed us how to decompose any physical phenomenon, any
description of a physical phenomenon, into a series of sinusoidal components.
A sinusoidal component is like a wave, and a wave is parameterized by its
frequency, which is really a way of measuring how fast the wave oscillates.
You can see an example of this sinusoidal decomposition of
the signal in the spectral analyzer of your MP3 player.
By splitting a signal into frequency components you
can see where the energy of the signal is.
And if it is in the high frequencies then the signal will be moving very fast.
Whereas if it is in the low frequencies it will be moving very slow.
The next piece of the puzzle that completes the path from continuous
reality to discreet reality was given by Nyquist and
Shannon, two researchers at Bell Labs in the 50s.
Their sampling theorem is really the bridge that
connects the analog world to the digital world.
The formula is like so, and it looks pretty complicated right now but
you will be very familiar with it by the end of the class.
If you just have a tiny look at it, you will see that
the formula relates a continuous time function, the Platonic ideal
we were talking about, to a set of discrete time measurements.
And this sum really is a weighted sum where
for each discrete sample we associate a special shape.
Graphically it looks like so.
If this is our continuous time function, the ideal function, and we have a set
of measurements that we indicate with these red
dots, we can reconstruct the original functions starting
from the samples just by associating what we
call a sinc function to each of the points.
So you scale copies of the same function at each measured interval and
then when you sum them all together you obtain the original function back.
All you need for this magic trick to happen is
that the original function is not too fast, in
the sense that it doesn't contain too many high frequencies.
We will see this in more detail during the class.
If we now go back to the beginning of our overview, you
remember the second fundamental ingredient in
digital signals is the discrete amplitude.
What that means is that we have two forms of discretization of an ideal function.
Take for example this
sine wave.
The first discretization happens in time and we get a discrete set of samples.
And then the second discretization happens in amplitude, where each sample
can take values only amongst a predetermined set of possible levels.
The very important consequence of the discretization independently of the
number of levels is that the set of levels is countable.
So we can always map the level of a sample to an integer.
If our data is just a set of integers now, it
means that its representation is
completely abstract and completely general purpose.
This has some very important consequences in three domains.
Storage becomes very easy because any memory
support that can store integers can store signals.
And computer memory comes to mind as a first candidate.
Processing becomes completely independent on the nature of the signal because
all we need is a processors that can deal with the integers.
And again, CPUs are general purpose processors
that can deal with integers very, very well.
And finally transmission, with digital signal we
will be able to deploy very effective
ways to combat noise and transmission errors as we will see in a second.
As far as storage
is concerned, just consider the difference between attempting to store
an analog signal, which requires a medium dependent support for
each kind of application, and the task of storing a
digital signal, which requires just a piece of computer memory.
In analog storage the medium evolved as technology evolved.
And for instance when it came to sound we had wax
cylinders in the beginning, and then you had vinyl, and then
reel-to-reel tapes, and then compact cassettes, and so on and so forth.
Each medium was incompatible with its predecessor
and required specialized hardware to be reproduced.
Today everything is stored in general purpose support systems, like memory card
or a hard drive, and is completely independent on the type of
content that is recorded.
If you consider for instance the evolution of memory
supports, this is a famous picture from the internet.
Just one microSD card will contain all the information
that was contained in countless floppy discs and CDs from just a few years back.
But what does not change, although the support changes and the capacity improves
with time, what does not change is the format
of the data which will remain the same across media.
When it comes to processing again, the
fact that the representation of the data is
completely decoupled from the origin of the data
will allow us to use general purpose machines.
Here on the left you have three three analog processing devices.
A thermostat on top with its temperature sensitive coil,
you have a set of gears that for instance can be
used to measure movement or time, and a discrete electronics amplifier.
Each of these devices had to be designed and
built to process just one type of analog signal.
Conversely, on the right, you see just a
piece of C code that implements a digital filter.
Now this filter can be used to process a temperature signal or a sound signal and
its structure or its implementation will not change.
Finally, let's consider the problem of data
transmission which is probably the domain where
digital signal processing has made the most difference in our day to day life.
So if you have a communication channel and you try to send information from
a transmitter to a receiver you are faced with a fundamental problem of noise.
So let's see what happens inside the channel.
You have a signal
that will be put into the channel. The channel will introduce an attenuation.
It will lower the volume of the signal, so to speak.
But it will also introduce some noise, indicated here as sigma of t.
And what you will receive at the end is
an attenuated copy of your original signal, plus noise.
This is just facts of nature that you cannot escape.
So, if this is your original signal what you will get at the end is an attenuated
copy scaled by a factor of G, plus noise.
So how do you recover the original information?
Well, you try to undo the effects introduced by the
channel but the only thing you can undo is the attenuation.
So you can try and multiply the received signal by a gain
factor that is the reciprocal of the attenuation introduced by the channel.
So if you do that you introduce a gain here at the receiver and what you get is,
let's start again with the original signal, attenuated copy,
some noise added and then let's undo the attenuation.
Well what happens unsurprisingly is that the gain factor has
also amplified the noise that was introduced by the channel.
So you get a copy of the signal that is yes,
of a comparable amplitude to the original signal, but
in which the noise is much larger as well.
This is typical situation that you get in second
generation or third generation copies of say a tape.
Or if you try and do a photocopy of a photocopy, just
to give you an idea of what happens with this noise amplification problem.
Now, why is this very important?
This is important because if you have a very long
cable, so for instance, if you have a cable that goes from Europe
to the United States, and you try to send a telephone conversation over there.
What happens is that you have to split the channel into several
chunks and try to undo the attenuation of a chunk in sequence.
So you actually put what are called, repeaters along the
line that regenerated the signal to the original level every say,
ten kilometers of cable or so.
But unfortunately the cumulative effect of this chain of receiver, is that some
noise gets introduced at each stage and gets amplified over and over again.
So for instance, if this is our original signal which again, gets
attenuated and gets corrupted by noise in the first segment of the cable.
After amplication you would get this,
which is in that before.
Then this signal is injected into the second section of the cable.
It gets attenuated.
New noise gets added to it and when
you amplify it you get double the amplified noise.
And after N sections of the cable you have N times the amplified noise.
This can lead very quickly to a
complete loss of intelligibility in a phone conversation.
Let's now consider the problem of transmitting a digital
signal over the same trans oceanic cable.
Now, a digital signal as we said before,
is composed of samples whose values belong to a
countable finite set of levels and so their
values can be mapped to a set of integers.
Now transmitting a set of integers means that we can encode
these integers in binary format and therefore we end up transmitting
basically just a sequence of zeros and ones, binary digits.
We can build an analog signal associating say the level plus five
volt to the digit zero and minus five volt to the digit
one and we will have a signal and we will have a
signal that will oscillate between these two levels as the digits are transmitted.
What happens on the channel is the same
as before, we will have an attenuation, we will have the addition of noise and
we will have an amplifier at each repeater that will try to undo the attenuation.
But on top of it all, we will have what is called a
threshold operator that will try to reconstitute
the original signal as best as possible.
Let's see how that works.
If this is what we transmit, say an
alternation of zero and one mapped to these two
voltage levels, the attenuation and the noise
will reduce the signal to this state.
The amplification will regenerate the levels and will amplify the noise.
So, the noise is much larger than before.
But now we can just threshold and say, if the
signal value is above zero we just output five volts.
And vice versa, if it's below zero we will output minus five volts.
So the thresholding operator will reconstruct a signal like so.
So you can see that at the end of the first repeater we actually
have an exact copy of the transmitted signal and not a noise corrupted copy.
The effectiveness of the digital transmission schemes
can be appreciated by looking at the evolution
of the throughput, the amount of information
that can be put on a transatlantic cable.
In 1866,
the first cable was laid down and it had a capacity of
eight words per minute which corresponded to approximately five bits per second.
In 1956 when the first digital cable was laid down on the ocean floor
the capacity all of the sudden sky rocketed to three mega bits per second.
So, ten to the power of six.
Six order of magnitudes larger than the analog cable.
And in 2005 when a fiber cable was laid down another six
order of magnitude were added for a capacity of 8.4 terabits per second.
Similarly and literally closer to home, we can look
at the evolution of the throughput for in-home data transmission.
In the 50s the first voice-band modems came out of Bell Labs.
Voice-band meaning that they were devices designed
to operate over a standard telephone channel.
Their capacity was very low, 1200 bits per second, and they were analog devices.
With the digital revolution in the 90s digital modems
started to appear and very quickly reached basically the ultimate
limit of data transmission over the voice-band channel which was
56 kilobits per second at the end of the 90s.
The transition to ADSL pushed that limit up to over 24 megabits per second in 2008.
Now, this evolution is of course partly due
to improvements in electronics and to better phone lines.
But fundamentally its success and its affordability are
due to the use of digital signal processing.
We can use small yet very powerful and cheap general purpose processors
to bring the power of error correcting codes
and data recovery even in small home consumer devices.
In the next few weeks we will study
signal processing stun, starting from the ground up.
And by the end of the class will have enough tricks in our bag to fully
understand how an ADSL modem works. And so after this very far reaching and
probably rambling introduction, it's time to go back to basics and we will
see you in module two to discover what discreet time signals are all about.