All right, so I think I've explained enough.
Now we're relief that at least it's 100% clear for everyone.
And so there you go, my friends.
Let's implement one of the last tools of this data preprocessing toolkit,
which is indeed the split of the data
set into the training set and the test set.
All right. So how are we going to do this.
Well we're going to do it with a function a function by scikit learn.
You know the most popular and useful data science library.
Because once again this library contains a module that is called model selection,
which contains itself a function called train test split.
And this function will exactly do what we want,
which is to create four separate sets, actually not two, but four,
because we will actually create
a pair of matrix of features independent variable for the training set.
And another pair of matrix A features independent variable for the test set.
All right.
So we're basically going to get four set xtrain
which is a matrix
of features of the training set X test which is the matrix of features of
the test set Y train which is a dependent variable of the training set,
and white test which is a dependent variable of the test set.
That's exactly what we want.
And now why do we want this?
Well, it's not us.
It's actually the future
machine learning model that we will build in the next part,
which will be all of them expecting this format as inputs.
You know, for the training, it will expect X train and Y
train as inputs in the method actually called the fit method.
And for the predictions also called inference,
these models will predict X test. All right.
So that's the reason.
It is simply the format expected by the future machinery models.
And now let's get these four sets.
So we're going to get them from scikit learn.
Of course.
There you go.
From which we're going to get access to model selection
I really like Google Colab.
And then from which we're going to import that train
underscore test split function.
Perfect.
You see how we can be so efficient thanks to the assistance of Google Colab.
I hope you really like it as well.
All right, so now that we have this function, well we're going to use it.
And since we already know what this function will return as I just explained.
Well let's create these four variables returned by this Traintestsplit function.
And as we said they are first x train
the matrix of features of the training set, therefore containing
all the countries one hot encoded ages and salaries of the training set.
So extreme.
Then x test the matrix of features of the test set.
Then Y train, which is the dependent variable
of the training set, meaning all the purchased decisions
of the customers in the training set Y train
and then Y test, which same contains
all the purchase decisions of the customers in the test set.
All right.
So that's the four variables returned by this train test split function.
And since it is the function
that returns this variable, well let's take that function right away.
And let's add here an equals
and train test bit and then some parenthesis.
And now the question is what do we have to input inside this function.
All right.
So actually there are some parameters that we can guess right.
Because indeed this train test split is supposed to split something.
So one of the inputs will be that's something
which we're about to split and which is of course our data set.
However, of course this function does not expect the data set as a whole.
It expects.
Well, the
combination of the matrix of features X and the dependent variable vector y.
And that's the first two inputs of this function.
So let's input them here x.
First a matrix of features and y the dependent variable vector
grid y. Yes.
Then come up and then next arguments.
So we still have to input two more arguments which are going to be
first the split size.
You know, because we're not going to split this data
set into a training set and a test set of the same size.
Actually we need a lot of observations in a training set and a few in the test set.
But we need a lot of them in the training set.
So that's to give the future machine learning model more chance
to understand and learn the correlations in the data set.
So let me just tell you the recommended size of the split.
Well I recommend to have 80% observation
in the training set and 20% in the test set.
All right. This is a very good split.
And therefore here we're going to input a new parameter
which is test size.
And we'll set that equal to 0.2.
Right 20% observations will go into the test set.
And therefore here since we have ten observations in this data set,
that means that eight observations will go into the training set, meaning
eight customers will go into the training set.
And to in the test set.
And this is not necessarily the last two.
You know, they will be taken randomly,
but eight of them will go into the training set and to in the test it.
All right.
And now we'll add one final argument just for teaching purposes so that we can
have the same results displayed in here, you know, in the notebook.
Because then I'm going to run some prints
to show you these four elements returned by this traintestsplit function.
You know, the training set and the test set.
And since there are some random factors that are going to happen
during the split, right, because the observations
will be randomly split into the training set and the test set.
Well, to make sure we have the same random factors, we'll just add here
random state equals one.
Right?
We were just fixing the seed here so that we'll get the same split
and therefore the same training set and same test.