Welcome to this Python tutorial and use the toilet toils we already implemented a multiple regression
model that we fitted to the trainset.
But when we take a step back do you think it's actually the optimal model that we can make with the
data set that we have here.
You know because when we built this model we actually used all the independent variables.
But what is among these independent variables there are some that are highly statistically significant
that is have great impact on the dependent variable profit and some that are not statistically significant
at all.
That means that if we removed this non statistically significant variables from the moral world we would
still get some amazing predictions.
So the goal in this tutorial is to actually find a team an optimal team of independent variables so
that each in the betting Bible of the team has great impact on the dependent variable profits that is
each independent variable of the team is a powerful predictor that is highly statistically significant
and definitely has an effect on the dependent variable profit and this effect can be positive.
That is for an increase of 1 unit of the independent variable.
The profit will increase or it can be negative.
That is for an increase of 1 unit of the independent variable.
The profit will decrease and so we're going to divide this final step of building this optimal model
using backward elimination into three tutorials in this first tutorial that we are having right now.
I'm going to walk you through the backward elimination preparation because in python we actually need
to prepare something specific which you will see in this tutorial.
Then in the second tutorial we are going to start backward elimination together without completing it
up to the end.
That means that at the end of the second tutorial you'll get a homework which will consist of completing
what we started with backwardly nation.
So I'm sure we will have no problem because I'm going to walk you through the introduction of backward
elimination so that you can understand everything and have all the tools to complete the homework.
And then in the third tutorial I'll give you the solution of this homework and we will complete together
the backward elimination.
So I hope you're excited.
Let's start right now.
The backward elimination so we will of course as usual need a library which is going to be the stats
monohulls API library then we'll just need to do a little trick so that backwood elimination runs properly
and then we'll be good to go.
So let's do it let's first import stat's models formula that API and then let's give maybe a shortcut
because this is a long name so let's give the S M shortcut for a static model.
And now we are almost ready to start backwards elimination.
We just need to add a column of ones in our matrix of features or matrix of independent variables and
to understand why we just need to look back at our multiple in our regression equation which is here
y equals zero plus 1 x 1 plus B and accent.
Well you know what is in this equation that there is this constant zero which is not associated to an
independent variable x like there is no x zero here.
But actually there is we can associate next 0 to this constant zero.
It's actually zero equals 1.
So that means that if we said 0 1 then or multipole in our regression equation becomes why equals zero
zero.
Plus we want x 1.
Plus the unexcelled and actually the stats models library that we are about to use right now doesn't
take into account this BS you constant.
So we will somehow need to add it in our matrix of independent variables because as you can see our
matrix of features X which is here only includes the independent variables like the dummy variables
for the states the R&D spend and spend in marketing spend.
There is not this exit recalls one anywhere in this matrix of features and in most library like the
library that we use here to build a linear regression model its definitely included the fact that there
is a constant zero in the multiple in our regression equation but that's actually not the case for this
stats models library which we will use to compute the values and evaluate the statistical significance
of our independent variables.
So that's why we need to do something here and if something is to actually add this column of ones here
in our matrix of features.
So we will take our matrix of features X here and we'll just add a column of ones here that will correspond
to this zero equals one associated to our constant zero.
That's why our small library will understand that the multiple in the regression equation is actually
y equals zero plus you want X1 plus B2 X to be unexcelled.
So let's do this.
It's actually really simple we will just need one line to do this.
And let's do it right now.
So since we want to add this column of ones here to our matrix of features X we will start by taking
x here and then we'll use this append function of not B which will append this column of ones in our
matrix features X.
So let's do this we are going to call and B here Dot and then append which is our function.
Now let's add some parenthesis and this press command control plus to inspect the append function.
Ok so what are our parameters.
The first parameter is our four array.
So of course this is going to be the array to which we want to add this column of ones and therefore
it's going to be our matrix of features X.
So here is the first arguments we'll input our equals x.
OK and then what is the second argument.
The second argument is values and the sign.
Of course the values are all the ones that you want to add X and as you can see these values are in
the shape of an array.
That means that your values is going to be an array and that's perfect for us because we actually want
to add a column of ones as the first column of x.
So the array that we're going to import here will be a matrix of 50 lines because as you can see X has
50 lines and one column.
So we need to add a matrix of 50 lines in one column with only one values inside and there is actually
a trick for that.
We can do it very quickly we just need to use another function of number be called once which will create
an array of only one value inside.
So we will need to specify the number of lines of this array and it's number of columns.
So our second argument is going to be values equals and then we take a one function from that be.
So we add here and b that once and then since this function once returns a matrix of only one values
inside we need to specify the number of lines and the number of columns of this matrix of ones that
this function once returns.
And if we look at this one function and more details by inspecting its pressing command control plus
I.
Well we see here in the parameters that the first parameter is the shape that is the shape of the matrix
of what we want to create.
And as you can see we need to put it this way we need to put some parenthesis with inside the number
of lines and the number of columns.
So here we'll just need to input new parenthesis in which we add the number of lines which is 50 and
the number of columns which is one.
But now we need to do one more thing because otherwise we will get a data type error and to prevent
this data type error from happening we just need to convert this array of once here into the integer
type and to do this it's very simple we just need you actually add as type and in parenthesis.
And now we just need to add the last argument here which is access because you can use the pen function
here to add either a line of these values or call them of these values to the matrix here which is our
matrix of features X and therefore we need to specify if we want to add a column or line.
So if we want to add a line then its access equals zero.
And if you want to call them then its access equals one.
So here we'll just add access equals one.
And now I'm sorry guys there is just one tiny little thing to do before this line is ready to be executed.
It's related to the fact that right now we are indeed adding this column to 51 here to our matrix of
features X. But we're actually adding it at the end of our matrix of features X. And we would like to
have it at the beginning.
So according to you what can we do here.
We can use a very simple trick to actually add this call in the once here at the beginning of our matrix
if you just x what is the trick instead of adding this call in a 50 once here to our matrix X..
What if we added instead our matrix X to this column or 50 once here and so to do this it's actually
real simple we just need to invert and put it in or this way.
And then for values when put X so that right now we are adding or matrix of features X to this one called
him a 50 once here and there for this column a 50 once here will appear as the first column our matrix
features X.
And now of course I'm going to show you that this actually adds this column of one sort of matrix if
you features X at the beginning of the matrix.
So let's go to variable explore here.
Let's have a look at X right now.
So as you can see X contains all the independent variables without the first column of ones here.
So these are the two dummy variables for state.
This is already spent.
This is a minivan and this is marking inspanned.
And now if I select this line and press them under control and to execute.
Let's now have a look at X and you will see that now x contains this first column of ones here and this
column corresponds to this x zero equals one associated to this constant be zero.
And the multiple regression equation y equals zero plus you want x 1 plus 2 x 2 plus be an accent.
OK.
So let's press OK.
And now the moment we've all been waiting for.
We're going to start backward elimination but we're just going to take a little break now and we start
doing that in the next tutorial.
Until then enjoy machine learning.