Hello.
I want to extend a very warm welcome to all
of you through the fundamentals of digital image and video process.
My name is Aggelos Katsaggelos and I'm a professor here at
Northwestern University, in the department
of electrical engineering and computer science.
I'm really very glad to meet all of you in this new modern way of communication.
I'm really very excited about teaching this class, it clearly represents
a major departure from the traditional way of teaching a class.
I've been teaching the material we'll be covering in this, in this
class for almost 30 years now, here at Northwestern and around the world.
In all cases however, except maybe a few ones that there was a live feed of
the lecture I was giving, the students and
I, we were all physically in the same room.
So the situation clearly allows for interactivity.
We're all familiar with real time interaction; whereby students can stop me
and ask questions and that question brings up another one by another student.
And based on the question, I start discussing other
aspects of the material, and so on and so forth.
These are all the positive aspects of face to face interaction.
A potentially negative aspect of it
however is that it represents synchronous learning.
The underlying assumption is that we are all supposed to learn at the same rate.
This is hardly the case.
Each of us learns at their own rate,
based on our backgrounds and so many other factors.
And this is where asynchronous learning comes in.
You'll have the videos of my lectures available, and
you can slow it down or speed it up,
and you can go back and listen to parts
of it multiple times, and then something is clear.
You can actually skip a segment all
together if you all ready know the material.
You can learn at your own pace, at your own time and place suitable to you.
Some of you, you'll be in sunny places, and some in cold places with snow.
Some of us learn better first thing in the morning, while others, late at night.
Of course, due to the fact that most probably there
are students in all world time zones, we'll be active 24/7.
Now, regarding the material, there is probably little need
to sell the topic of image and video processing.
We are all surrounded by countless applications, which make use of images and
videos, and require knowledge of the material
we will be covering in this class.
We'll be covering, for example, the topic of image and video compression.
If compression technology was not successful, we should not be able to watch
this video, and we will not be able to have this course materialized.
You can go back and watch also the promotional
video for the course where we talk about other applications.
In any case, I assume you've registered for the course since you
either have some background of the topic and you want enhance it.
No pun intended here, since we will
be talking about image enhancing in the course.
Or you know that the class covers important
material that will serve you well in the future.
Or, you have a specific problem in mind, and you want to find out
if the material, we'll be covering here, will help you to solve this problem.
Or we just have a simple curiosity to find
out what the image and video processing is all about.
My hope and expectation is that no matter to which of these cases you belong to, and
of course there maybe other cases, upon completion
of the course you'll all have meet your goals.
This is not an easy task, and I don't want to mention it lightly.
It's not an easy task because, we will be drawing
material from different areas of topics within science and engineering.
Unlike a class at an engineering or computer
science department, where the background of the students
is rather uniform, In this case it's safe
to assume that your backgrounds vary quite a bit.
As we'll see we'll rely on background
from 1 dimensional signal processing, which we'll
extend to 2 dimensions, calculus, linear algebra,
probability, random variables and random processes, and optimization.
However, an important point I want to emphasize here is that the course has been
carefully designed so that it takes into
account this highly varying background of the students.
On one hand I'll try to first describe the background
information we'll be needing before we used it, use it.
While on the other hand, I try to emphasize the
intuition behind each and every processing step we'll be taking.
So, even if you don't have a background in the topics I just mentioned,
I believe you leave the course with
some basic understanding of the various technologies.
Even if not all the mathematical details like this are clear to you,
and also with some useful algorithms you can make use of right away.
Now, for those of you with strong background in the topics I
mentioned, I still believe you can find the material informative and challenging.
Regarding the logistics of the course, all information you need is on
the web page of the course under announcements and the various tabs.
Regarding the material to be covered in the
class, we'll start smoothly this week with an introduction.
Then we'll spend two weeks on fundamental signals and the fundamentals of two
dimensional linear and spacial invariant systems,
both in the spacial and frequency domain.
We'll then spend a week on motion estimation and
color, two topics we'll be making use throughout the course.
And then we'll look at important
applications, mainly enhancement, recovery, compression, and segmentation.
We'll finish the course describing recent advances in the
field which have to do with the concept of sparsity.
So let's start this week smoothly again by passing the title of the course and
defining what is digi, what is a digital
signal, and what constitutes processing of that signal.
With respect to the definition of a
digital signal or describe an end-to-end system
with input and output in the analog
worlds But processing performed in the digital world.
This is the topic of this segment.
We'll discuss the important concepts of sampling and quantization, which allow
us to move back and forth between the continuous and discrete.
Sampling actually, not quantization, both in space, time and also.
So let's start, and my best wishes to all of you for a successful course.
Let us first define what is a signal.
A general definition, is that it's a function in 1 dimensional.
The an x of t for example, function is the independent variable or
a 2 dimensional x of t1, t1, t2 are independent variables.
That contains information.
It tells us something useful about the behavior
or the nature of some phenomenon of interest.
In the physical world, we can say that any quantity that changes
with respect to time and/or with respect to space, is potentially a signal.
So the speed of a moving car represents a signal.
The pressure that I apply on the gas pedal of the car is a signal.
If I go and measure the height of the buildings
in a city, then this becomes a two dimensional signal.
It tells us, at each location, what the height is, and so on and so forth.
Another example here is the price of the oil over
the years, so the independent variable is time, because the
local axis here is time in years and the vertical
axis is the price of oil in dollars and cents.
It goes up as we all know and expect.
Similarly here is an E-K-G signal, an electrocardiogram.
Again, here is time on the horizontal axis, and
the object of the signal on the vertical axis.
So a signal like this shows the count of somebodies heart.
Signals and systems play a very important
role in many areas of science and technology.
From communication, to aeronautics, astronautics [INAUDIBLE]
design, bio-medical engineering and so on.
Clearly in this class images and videos are
the signals of interest we'll be dealing with.
We have analog and digital signals.
To put them in some perspective let's look at the speech generation,
transmission, process, and perception system, like
the one depicted in this figure.
So speech is generated, by the pressure provided by the
lungs, but become sound in the glottis in the larynx.
Which is then converted in to vowels and consonants by the vocal track.
Speech's an acoustic signal, that is
transmitted through sound waves, utilizing the vibration
of the aero m, air molecules, so here we have an acoustic signal.
So such a signal reaches,
microphone here, which is a transducer.
It converts one form of energy, acoustical
energy, into another form of energy, electrical energy.
So here, I have an electrical signal.
[SOUND]
Both the signals are analog signals, or continuous signals.
If I take a kind of toy, example here such a signal, lets say looks like this,
where here is the time max is, and here X of T is the amplitude of such a signal.
This is an analog signal because the independent variable t, as well as
the amplitude x, which is a function of t, are continuous numbers.
Now, I want to process, such a speech, speech signal by a digital computer.
And this is really the objective of any digital signal processing class.
And digital computers only understand zeros and ones or
understand digital signals and therefore I need to convert
my analog signal into a digital one, which is
a function of this A to D converter box.
In doing so, converting an analog to digital signal, one has to take 2 steps.
The first step is to sample the signal or discretize it in the time domain.
This means that I look at the values of the signal at equally
spaced points, so at value times zero
times capital T times
2T, 3T, 4T minus T and so on.
So now I have the values of the signal here at
this time, at this time instance, this time instance, and so on.
So, I have converted my signal into a discrete
time signal
through the process which is called sampling.
The second step is to discretize the amplitude of the signal.
This means that I have only certain values
available to represent x of t integer values.
So, let's assume that the values I have available
are this one, this one, this one, this one.
This means that, so let's call this zero delta to delta 3 delta.
So this means that this value here at zero will be represented by this value.
The second value will be represented by this value.
similarly, this one will be represented by this value.
This will be represented by zero.
And this one will also be represented by this value.
Okay?
So, through this process that is called quantization
[SOUND] I have now a discrete
[SOUND] amplitude [SOUND] of the signal, right?
So, the value at 3,t here is the
quantized value of x, x,3,t, right?
actually, here will also be the same value x quantized
at 4T, while the actual value here, this is X of 4T.
Right?
And after quantization, it becomes the blue value.
So, clearly, quantization introduces an error.
So after both sampling and quantization, I end up with a signal that has
both its independent variable as well as
its aptitude being represented by discrete values.
And, such a signal Is called a digital signal.
Okay?
So this signal is an input to the computer, which is going to
process it and then at the other end I follow somehow the reverse order.
In other words the signal the turned from
digital to analogue through the converter shown here.
Then this analogue signal, reaches another transducer, this
is also called the nuctuater, that will turn the current in to a vibration.
And therefore in to another acoustic signal, which is
going to be perceived by the human auditory system.
So here again, I have an electrical [SOUND]
signal.
And this is and again, an acoustic signal.
The objective of a digital signal processing class
is to focus onto the central part here, right?
To focus on the techniques that will allow us to manipulate
digital signals to perform certain tasks that depend on the application.
So we see here that we have 2 words,
we have the analog world here and here, right?
So analog, analog here, and the digital world
or the digital domain here in the middle.
That is of our primary interest.
A similar situation to the previous one is depicted by this block diagram.
That shows the generation, recording, processing, and sensing of an image.
So electromagnetic energy in the visible part of the spectrum is
leaving the sun, reflected by the object, travels through the air, and reaches here.
A sensor, a transducer, which is an analog camera
that through photochemistry converts light energy into chemical changes on the film.
Some of you might be too young to remember analog cameras that were used in film.
But there is still some around in existence.
So the, the brighter parts of the
image, have higher concentration of silver halide grains.
And this film, after it's processed, negative or black and white film.
Represents an analog image.
So again my objective here is to process the image by digital computer.
And therefore I have to convert my analog image
to a digital one, through this A to D block.
In this particular case, I can use
a densitometer, that measures the concentration of this
silver highlight planes on the film, or a scanner, to end up with a digital image.
Of course now days, you have digital systems are prevalent.
Which means that these 2 units here are grouped together,
and this becomes a digital camera that we are all famliar with.
So I
guess we should put here that this in here is an analog camera.
After the digital image is processed by the computer,
and again, this is the main objective of this class.
We'll talk about techniques that will allow us to cross those digital images.
We want to take the processed image and in many cases,
we want this image to be seen by the human viewer.
So the reverse path is followed whereby the
digital image is converted into an analog one.
And this then, signal will feed into a
monitor, an analog monitor, such as a CRT monitor,
that will convert the image into an electromagnetic
wave that is going to reach the human, visual system.
The eye of the observer.
Again, nowadays digital monitors or flat panels
are, are prevalent so, and LCD,
here will be fed by the digital signal directly.
[SOUND] A video adapter
plays the role of the
[UNKNOWN] converter.
So, in this class we'll say a few things about sampling.
[SOUND] And
quantization of images.
That I represent by this A to D block, and
actually could be [UNKNOWN] something or frequentization as we'll see.
We'll say a few things about properties of the visual system, but by and large our
main focus is in this center block where the
input is a digital image, and the output is a another processed digital image one.
To demonstrate sampling, or better yet,
re-sampling, let's consider this digital image.
It's a 256 by 256 image, which means it has 256 rows and 256 columns.
And it's an 8 bit image which means,
each element here which is called a pixel from picture element
has 2 to the 8, or one of 256 values.
Okay?
So, I'll take this image, and I will downsample it by
a factor of eight in both the horizontal and the vertical directions.
So out of eight samples, horizontally, I'll keep one.
I'll do the same in the vertical direction.
Or in other words, if you consider here an eight by eight
block I will only keep this value and throw away that s.
So, I keep one and throw away 63.
Okay?
If I do this, then I end up with a 32 by 32 image, this tiny image shown here.
If I bring it back to, the sixth, if I bring it back to
the sixth dimension for organization purposes, then,
that's how this image will look like.
So, you see, that due to this downsizing.
You see this blocking effects and this so called jagged edges, right?
They're not straight edges anymore, but they're jagged.
Of course drop sample they [INAUDIBLE] something very simple.
I used so called zero order hold.
Which means, I just took a, 8 by 8 block, I only had this value I was missing
the others, 63 values, and I simply gave to the other 63 values, the value of this.
So, each 8 by 8 block here's the same
value and this is apparent by looking at this image.
This is an 8 by 8 block for
example, and it has constant value or constant intensity.
Similarly, the idea of re-quantization is the depicted here.
Over here is the 8 bit per pixel image you started with.
So, here I have 2 to the 8th, 256 different grade values
and while here I have only 2 to the 4th equals
16 values to represent the different intensities.
And over here, I go down to 2 to the second that is 4 different values.
So clearly you see this so called
contouring effects, that visible here even, right?
So there's a you know, artifical, contours abound
that is have, nothing to do with original image.
So you save bits clearly in going from 8 bits to 4 to 2 bits
will present intensity, but at the same time
you introduce errors with so called quantization errors.