In this lecture, we're going to look more deeply into IPCA, we're going to talk about what does PC
actually accomplish?
Why do we use it?
What advantages does it have in this lecture?
You'll learn that PC actually does quite a lot more than what you might expect.
OK, so to start this lecture, I'm going to give you a summary of everything that PC can do.
And for the rest of his lecture, I'll explain each topic in more detail.
Again, let me remind you that this lecture is the intermediate level lecture.
So if you don't have the mathematical tools to understand this or you don't care, then you can skip
ahead to the code lecture.
OK, so the first thing that PCA can accomplish is dimensionality reduction, that is we might start
with 500 stocks, but when we transform them into factors, we end up with only three factors.
We saw this in the original height and weight example as well.
We started with two features, height and weight, and we discovered that they are only controlled by
one factor, which is size.
The second thing that PC can accomplish is visualization.
Hopefully it's not surprising that your computer screen is two dimensional and the physical world is
three dimensional.
So if you want to plot a five hundred dimensional object, humans don't have the capability to comprehend
what that even means.
This is directly tied to dimensionality reduction as well, which was the first task.
So if we reduce our dimensions from 500 to two, then we can plot these new dimensions and observe our
data.
Now, obviously, we lose some information in the process, but as you'll learn later, we lose the
minimum amount of information in some sense.
The third test that PC can accomplish is Decorrelation, as you recall, stock returns are correlated,
that is, there is redundant information contained in that PC removes this redundancy by having all
the principal components be uncorrelated.
So, for example, if one of the principal components refers to size, there won't be another principal
component that is also related to size.
Redundancy is removed.
Finally, the fourth task that PC can accomplish is the noisy, as you recall, from our height and
weight example, the direction of large as the variants represented size.
But there was another dimension with very small variance.
Our interpretation of directions with very small variance is that it's just noise.
So by discarding those dimensions, we are removing noise from the data.
OK, so now let's discuss each of these topics in more detail.
The first topic we're going to discuss is dimensionality reduction.
In fact, I've already told you pretty much everything you need to know.
If we remove dimensions of the data, which are just noise, then equivalently we are just reducing
the dimensionality.
If, for example, we start with 500 dimensions and we go down to five, and what we're saying is that
those other four hundred ninety five dimensions are just noise.
The practical question is how does this work?
ENCODE, encode.
As you know, we're going to call it a fit transform function and we're going to get back a matrix called
Z.
Previously, I told you that this matrix is of size and becae, which is only partly true, in fact,
it's actually owned by the same size as X. The original data, of course, this makes sense, given
the fact that I told you earlier that PCA is doing a rotation rotation in and of itself does not remove
dimensions, it just changes the directions.
The key point is we choose K ourselves typically to be a value much less than be conveniently.
When we get back the Z Matrix, it's already organized so that the left most columns are the principal
components with the greatest variation and the right most columns are the principal components with
least variation.
That is to say, they are sorted so we can pick the key most important principle components simply by
ignoring the rest of the columns of Z on the right of this matrix.
So that's why we can think of the Z matrix as being and by K we're taking the K leftmost columns of
Z, and we know that we can do this because each column of Z is sorted by importance.
Alternatively, there is an argument you can pass into the constructor of the class called end components.
This is where you can specify explicitly, but as you'll see, it's usually a good idea to just do the
full transformation and analyze the results yourself.
Furthermore, even if you choose K smaller than D, the training part of the model still must be done
in full.
So you don't gain much in the training process by choosing K smaller than the.
You'll see why this is in a later lecture.
The next topic of this lecture is visualization.
Now, this is pretty much self-explanatory after the previous topic, by reducing the number of dimensions,
the two, you can make a two dimensional scatterplot of your data.
Now, you might wonder whether you are losing information in the process of reducing your data dimensionality.
The answer is yes.
But the important question to ask is how much are you losing as per the previous slide?
We already know that by choosing the two left most components of Z, we have chosen the two principal
components with the most information for PCA.
We can think of variants as analogous to information, although technically there are more rigorous
measures for information.
We know that PKA sorts each column of Z so that the dimensions with highest variance will come first.
Well, it turns out that during the training process for PCA, which you will learn about next, we
also get the variance for each of the dimensions and Z.
So that tells us exactly how much each principal component contributes to the total variance.
For example, we might find that the first principle component contributes 50 percent to the total variance,
the next may contribute only 30 percent, the rest may contribute only one percent.
So that's the idea.
And just as a side note, this is often another visualization we like to make when doing PCA.
We don't just want to visualize the data itself with the scatterplot.
We want to visualize how faithful this representation is relative to the original data.
And we can do that by examining how much variance is contained within each principal component.
The next topic of this lecture is Decorrelation, it's important to remember that uncorrelated this
does not imply independence.
So just because your variables are uncorrelated does not mean they are independent.
However, being uncorrelated by itself can still be useful.
Recall that correlation is just the scale to covariance.
So when we say two factors are uncorrelated, that means their correlation is zero and this also means
that their covariance is zero.
Recall that the covariance is just the expected value of EXI minus Mooi times X minus Buji.
This is sort of a generalization of variance.
It tells us whether or not variable AI and variable J move the same as each other, move the opposite
as each other, or are just not related at all.
It turns out that the way we form Z, we make it so that each column of Z is uncorrelated.
And this is important in terms of dimensionality reduction for dimensionality reduction.
We don't want any redundancy.
If we have any redundancy, that means more dimensions can be removed.
So by having each dimension be uncorrelated, we can interpret each factor as being a totally different
hidden cause behind the stock's return.
The final topic of this lecture is the noisy, as you recall, one part of the cap is the noisy here.
It says that any variation of a stock's return beyond what is given by the market return is just random
noise.
But how do we distinguish between a signal and noise?
We return to this idea of variance being of central importance.
If variance is big enough, we usually consider that to be a signal.
If the variance is small, we usually consider that to be noise.
Intuitively, you can think of a radio which is trying to transmit a sinusoidal wave.
The wave that it wants to transmit is a perfect sinusoid, of course, but the receiver will receive
a fuzzy version of that sinusoid.
That fuzziness is a result of adding random noise at small amplitudes so you can still see the original
sine wave, but there is noise added to it.
What pieces does is it separates the large variations from the tiny variations, and by removing the
tiny variations, we effectively remove the noise.
And again, in a practical sense, this is accomplished by simply ignoring the rightmost part of Z since
Z is ordered in terms of decreasing variance.
So instead of keeping the full and by D version of Z, we just take the leftmost part of Z of the shape,
and by K work is much less than D.
So in that sense, this is equivalent to dimensionality reduction.