Hello my friends, and welcome to this new.
Practical activity of part.
Nine dimensionality reduction.
So in the previous. Section we experimented.
With PCA.
Principal component analysis.
And we indeed.
Got great results with our.
One data.
Set, which will be.
The same. Data set for this new section.
Because, you know, we want to compare.
Several. Dimensionality. Reduction techniques.
So there. We go. We're going to see if we can even.
Beat PCA, which only had.
One incorrect. Prediction.
So we're going to work with the same data set.
And therefore the implementation will be. Exactly.
The same except one cell.
Which will be the cell of course where we.
Implement LDA instead of. PCA. All right.
Are you ready?
Let's do this.
Before we get.
Into this for the part nine, let's make sure everyone here is on the same page.
I gave you the link to this folder containing all the codes.
And data sets right before this tutorial.
So make sure to connect to it. And now here we go.
Let's end to part nine dimensionality reduction.
And now we're going to go into.
Section 44.
Linear Discriminant analysis.
LDA which will be a new technique.
Of dimensionality reduction.
Very powerful as we will see.
So let's start with.
Python as.
Usual. And there.
We go. This folder just as a.
Previous one has. Two files.
This is the implementation.
And this is the same one data. Set which.
You know.
Belongs to one sub business owner who first.
Asked you, you know the most sound to data scientist.
To do.
Some cluster to identify different customer segments.
For each of.
You know, the wines of. This data set.
You know, each row of this data set.
Corresponds to a certain wine.
And for each one we have several wine features.
Or, you know, characteristics.
All these up to here.
And you used.
All these features to. Identify.
Those three customer. Segments or, you know.
Customer clusters.
And after which, you know, since.
This wine shop. Owner was so. Happy and.
Impressed by your job, well, then of course the owner asked you to do.
Another mission, which is the.
One we're about to do with LDA, which consists of.
Building a predictive model, combine two dimensionality reduction.
Apply to this data set so that for each new.
Wine. That this owner. Has in its wine shop.
Well, by.
Deploying this.
New predictive model, this owner will be able to predict which customer.
Segment this new one belongs to so that it can recommend this new wine to.
The right. Customers and therefore optimize.
Eventually the sales.
All right.
So that's exactly.
The same data set. And now let's move on to
our implementation linear discriminant analysis.
Which we. Can either.
Open with Google Collaboratory as.
I'm doing it now. Or Jupyter. Notebook.
And as you notice. I kept this.
Previous implementation we did on.
PCA so that. We can, you know, compare.
The. Results. Indian. Right.
This is PCA, this is LDA.
But you know, since this is in read only mode.
We're going to. Create now.
A copy so that we can.
Re-Implement that.
Cell that belong to. LDA model.
So there we go.
Save a copy in drive.
This will create a copy. Inside.
Which will be able to. Re-Implement.
The LDA model.
All right.
And now we can, you know. Close this so.
That we can have the two implementations next to each other.
You know, the two copies.
And now let's do this.
Let's quickly.
Remove, you know, the.
Cell that. Implements to. LDA. This one.
And let's.
Re-Implement this because you know.
All the rest is the same.
I will actually.
Remove. All these outputs here so that you. Don't.
See the final. Results.
And we can keep them as a surprise.
So let me just remove the outputs too.
Don't look. Too close.
And there. We go.
All right.
So basically all the cells of.
This implementation are.
Exactly the. Same.
As the previous. One. PCA.
Except of course this cell.
That implements. LDA right here.
So no need to explain all this.
Plus all these other cells result from our diverse toolkits.
So you're. Definitely 100% familiar.
With them.
All right.
So let's do this.
Let's, you know apply LDA.
So we're going to create a new code cell. And there.
We go.
Let's implement linear discriminant analysis.
So now you have two options. The first.
And the. Best option is to press.
Bus on the video and try to implement this.
Yourself by of course browsing.
The. Scikit learn API.
And find that.
LDA class that. Can implement that. LDA.
Dimensionality reduction technique.
And you will definitely end up. With the.
Same solution I will implement in a few seconds.
And the second option.
Is of course to, well, not.
Press pause on the video and implement with meet the solution in.
Let's say three seconds, three, two, one and go, all right, let's do this.
Let's implement together. LDA.
So as I've just said.
We're going to implement LDA.
Thanks to the secured library.
Therefore we're going to start from sklearn
from which we're going to get access to this time.
Not you know.
The well, let me go to PCA here.
Not the decomposition module of Cyclegan, but a new one, which.
Is very.
Easy to remember because. This is actually.
Discriminant
underscore analysis.
Okay. That's another module.
Of. Scikit-Learn that contains.
Of course, the class.
That can implement. LDA and that class.
Well you know. After this import here we have to.
Add the name of this class.
And the name of.
This class is capital L.
And then very simply. Linear.
Discriminant analysis.
All right. Very good. The reason why Google.
Collab.
By the way,
is not helping me with the suggestions is because the notebook is not running.
And remember to run the notebook or, you know, to connect it.
Well, you need to either. Run any of.
The first cells or upload the data set.
So let's do. It right now so that, you know.
Google Colab. Can assist me.
I really love it when it does it.
So right now I just clicked on this folder button.
And then let's click the upload button.
And we will end up in the you know. Previous folder.
For Principal components analysis.
But let me show you the path again.
I put my machine learning.
It is a folder in my desktop.
So inside we're going to go now to part nine and then section
44 Linear Discriminant Analysis and Python and then one.
All right.
So this is exactly the same dataset as before.
But I just wanted to. Show you the path.
All right. And there we go. We have to one.
And so now I'm going to show.
You if I retype this.
Linear discriminant.
See now it is helping me.
So that's maybe better to have, you know.
This reflex to upload a data set right at the.
Beginning. Okay. So linear discriminant analysis.
But since.
This class. Name is actually pretty long and pretty.
Not practical, well, let's just, you know, add a simple shortcut like LDA.
We can do this. That's fine. And now.
Let's press.
Enter to move on to the next step.
Which is of. Course naturally.
To create.
An. Object of this.
Linear discriminant analysis class.
All right.
So of course we're going to call it LDA.
Right. And now we're going to call this class.
And since we gave it the shortcut LDA.
Well we can simply call.
LDA this way.
And now well exactly the same as before.
This LDA class needs to take.
As input only one argument, which is. Exactly.
The same as before. And also.
It has the exact. Same name.
It is n components.
Which corresponds, of course, to the.
Final number of. Extracted features. You want to end up.
With after applying this.
Dimensionality reduction technique.
And of course.
As I.
Recommended in the previous section, we're going to start.
With two.
So that we can see if even with only.
Two extracted features.
Well, we can get great. Results.
And if that's the case, we'll not only.
Will get great results, but also cherry on the cake will be able to visualize the.
Results on a nice.
2D plot.
Indian. You know.
Thanks to these two code section.
All right.
But right now we need. To finish this.
So there we go.
We're going to extract only two features.
In the end. And to. Do this we. Need now of.
Course. To connect our.
LDA object to our data set.
But once again separately the training set and the test set.
And to connect. It well.
Of course we need to apply.
The fit. Transform method.
To the.
Training set and then only the transform method on the test set.
That's for the exact. Same reason as. Before.
It is to.
Avoid information leakage from the test set.
All right. So let's do this. That's our next step here.
So we're going to take.
First X.
Train right which we're going to. Update.
To become the new X train.
After we apply this LDA feature extraction technique.
And to do this well we need to take.
Of course our LDA object from which
we're going to call the Fit transform.
Method.
Which will take as input.
Well here. Be careful.
It's not going to be exactly the same input.
As before. Because you know, with.
PCA the fit.
Transform method took only Xtrain as input.
Because it only need the features to apply this PCA.
Dimensionality reduction technique.
But LDA is actually different.
In order to apply the technique, it needs not.
Only the features but also the.
Dependent variable. Right?
A dependent variable is a required.
Element inside. The.
Equation. Of LDA.
And therefore here in the fit transform.
Method, we need to.
Input not only x train the old version of xtrain.
Before we apply LDA.
And. Y train.
All right, so be very careful with this.
Whether you choose to apply LDA or PCA for PCA.
You only have to input X train, and for.
LDA you have to input both the features X.
Train and the dependent variable y train.
All right. Final step.
Well now that we have an.
LDA feature extractor object fitted.
To the training set, well we can apply. It to the test set.
By only calling the transform method. Right.
It doesn't make sense to.
Fit it. Again to the. Test set.
Because the test set is supposed to be new data on which we.
Deploy our model, like in production.
Therefore, we must only apply the transform method here.
And therefore I'm updating our.
X test variable the.
Following way.
By first calling our LDA object.
From which we're only going to call the trend form method.
And now, according to you, does.
It need to take only X test.
As input.
Or X test and Y test?
Well, obviously.
It's only need.
To take X test because we're not supposed to have Y test.
You know X test is like new data.
On which we're going to deploy our model.
Then we'll get. Our.
Predictions and Y original and we'll compare.
Y prior to white test.
But we're not supposed to have white test because white is are the real result
that. Contain the hidden truth.
You know, the ground truth.
So of course here.
We only need to. Apply X test.
And the reason why we could enter Y train.
Here is because indeed, we are supposed to get the ground truth of the.
Training set.
Otherwise we.
Wouldn't be.
Able to. Train our machine learning model.
All right. So X test and there we go.
Not only the implementation.
Of LDA is over, but also the whole.
Implementation. Is over as well.
So now we're going to do this run.
Oh now that we, you know.
Uploaded that. Data set into the notebook.
So we are 100% ready.
And let's just remind what we want to improve compared to previously.
Well, you know, in the principal component analysis implementation.
We had.
When obtaining the confusion matrix, only one incorrect prediction.
Resulting.
In having an accuracy of 97% and in the.
Test set results, which are the.
Most interesting ones.
Well, we had indeed an almost.
Perfect separation. Of the. Three classes.
And now we're going to see. It with our new feature extracted from LDA.
Well, we can get a perfect separation of the classes.
And therefore 100%.
Accuracy.
Are you ready?
Let's do this.
3 to 1 go run also.
Now all. The cells. Are running and there we go.
Oh, there we go.
We just had. A 100%. Accuracy.
So in other. Words logistic.
Regression model was.
Totally able to classify perfectly our three classes by separating.
Them separately.
And that's exactly what. We're going to. See.
And you know the. Test set results.
Because indeed.
Well we almost had an incorrect one here.
But as we see the real.
Ones, you know.
Which are all the points here, red.
Green and blue fall into the right.
Prediction regions.
That red prediction region where.
Our model predict that the wine belongs to customer segment number one.
Then this one where our model predicts that the wines belong to customer
segment number two.
And finally, this prediction region where our model.
Predicts that the. Wines should be recommended to.
Customer. Segment number. Three.
And thanks to these.
New extracted features.
You know, LG one and LG.
Two, well this time we have a perfect class.
Separator. In other words.
We have a perfect classifier.
And if you're wondering.
How did. LDA managed to.
Separate perfectly the classes?
Whereas, you know, in.
PCA we could see that it was very difficult to.
Separate. You know, the wines of the test set.
You know, especially this one.
This one falls in the middle of. The red. Wines.
Well, that's because the extracted features are different.
You know, they're not the same as PC1 and PC2.
They are, you know, in some other dimensions in which.
Well.
This time it is.
Possible to separate perfectly the classes.
And that's why this. Time it works.
We are, in other words, in another dimension.
Okay, so.
I guess now we don't have much of a challenge because it's impossible.
To beat this. This is just perfect.
I remind that I did not make this data set.
You know, it's a dataset.
Taken from.
The UCI. ML repository.
So, you know, it's very close to a real world data set.
But there you go. That shows the power of this.
Dimensionality reduction technique linear discriminant analysis.
So now we're going to move on to the next practical activity.
The next section on this time kernel PCA.
And we can just.
Hope that we'll get, you know, at least as good results as PCA
or some as good results as LDA. In other.
Words, let's hope that we get.
Maximum one incorrect. Prediction.
So I look forward to seeing you.
In this next section to implement kernel PCA.
And until then, enjoy machine learning.