Hello and welcome to this Python tutorial in Embree use the toils we already implemented a multiple
in our regression model that we fit into the training set.
But when we take a step back do you think it's actually the optimal model that we can make with the
data set that we have here.
You know because when we built this model we actually used all the independent variables.
But what if among these independent variables there are some that are highly statistically significant
that is have great impact or great effect on the dependent variable profit and some that are not statistically
significant at all.
That means that if we removed this non statistically significant variables from the moral world we would
still get some amazing predictions.
So the goal in this tutorial is to actually find a team an optimal team of independent variables so
that each independent variable of the team has great impact on the debate variable profit that is each
independent variable of the team is a powerful predictor that is highly statistically significant and
definitely has an effect on the dependent variable profit and this effect can be positive.
That is for an increase of 1 unit of the independent variable.
The profit will increase or it can be negative.
That is for an increase of 1 unit of the independent variable.
The profit will decrease and so we're going to divide this final step of building this optimal model
using backward elimination into three tutorials in this first tutorial that we are having right now
.
I'm going to walk you through the backward elimination preparation because in python we actually need
to prepare something specific which you will see in this tutorial.
Then in the second tutorial we are going to start backward elimination together without completing it
up to the end.
That means that at the end of the second tutorial you'll get a homework which will consist of completing
what we started with backward elimination.
So I'm sure you will have no problem because I'm going to walk you through the introduction of backward
elimination so that you can understand everything and have all the tools to complete the homework.
And then in the third tutorial I give you the solution of this homework and we will complete together
the backward elimination.
So I hope you're excited.
Let's start right now.
The backward elimination.
OK.
So let's do it let's first prepare what we need to do.
Backward elimination so we will of course as usual need a library which is going to be the stats Morell's
formula API library which will be calling as M as a shortcut for stats models.
Then we'll just need to do a little trick so that backwood elimination runs properly and then we'll
be begin to go.
So let's do it let's first import the required library.
So imports stats models that formula that API.
And then let's give maybe a shortcut because this is a long name so let's give the S and shortcut for
a static model and now we are almost ready to start backwards lamination.
We just need to add a column of ones in our matrix of features or matrix of independent variables and
to understand why we just need to look back at our multiple in our regression equation which is here
y equals zero plus b 1 x 1 plus B and accent.
Well you notice in this equation that there is this constant zero which is not associated to an independent
variable x like there is no 0 here but actually there is.
We can associate an x 0 to this constant be zero.
It's actually zero equals 1.
So that means that if we said x zero equals 1 then or multipole in our regression equation becomes y
equals zero zero.
Plus we want x 1 plus be an accent.
And actually the stats model's library that we are about to use right now doesn't take into account
this BS zero constants.
So we will somehow need to add it in our matrix of independent variables because as you can see our
matrix of features X which is here only includes the independent variables like the dummy variables
for the states that are in this band adman's spent in marketing spend.
There is not this zero equals 1 anywhere in this matrix of features and in most library like the library
that we use here to build a linear regression model is definitely included the fact that there is a
constant to be zero in the multiple in our regression equation.
But that's actually not the case for this stats models library which we will use to compute the p values
and evaluate the statistical significance of our independent variables.
So that's why we need to do something here and this something is to actually add this column of ones
here in our matrix of features.
So we'll take our matrix of features X here and we'll just add a column of ones here that will correspond
to this as zero equals one associated to our constant zero.
That's why our stats Morell's library will understand that the multiplying our regression equation is
actually y equals zero plus Bubon X1 plus b 2 x 2 plus bionics.
Otherwise if we didn't add this column of ones here in our X matrix features our matrix of independent
variables this library stats models would have thought that the multiple in our regression equation
would be y equals one X-1 please be 2 x 2 plus b 3 x 3 4 x 4 plus 5 x 5 without the constant busy road
here.
That is without the components below zero zero in the multiple you know regression equation.
So that's why right now we need to add a column of ones in our matrix of features in this matrix here
.
So let's do this.
It's actually really simple we will just need one line to do this.
And let's do it right now.
So since we want to add this column of ones here to our matrix of features X we will start by taking
x here and then we'll use this append function of non-P which will append this column of ones in our
matrix of features X.
So let's do this we are going to call and be here Dot and then append which is our function.
Now let's add some parenthesis and this press command control plus to inspect the append function.
Ok so what are our parameters.
The first parameter is our four array.
So of course this is going to be the array to which we want to add this column of ones and therefore
it's going to be our matrix of features X.
So here is the first argument will input our equals x.
OK and then what is the second argument.
The second argument is values and the sign.
Of course the values are all the ones that two onto X X and as you can see these values are in the shape
of an array.
That means that here values is going to be an array.
And that's perfect for us because we actually want to add a column to x.
You know we're going to add a column of ones as the first column of x.
So the array that we're going to input here will be a matrix of 50 lines because as you can see X has
50 lines and one column.
So we need to add a matrix of 50 lines in one column with only one values inside and there is actually
a trick for that.
We can do it very quickly we just need to use another function of not be called once which will create
an array of only ones value inside.
So we will need to specify the number of lines of this array and it's number of columns.
So our second argument is going to be values equals and then we take a one function from Numby.
So we add here and the that ones in parenthesis because one has a function and then since this function
once returns a matrix of only one values inside we need to specify the number of lines and the number
of columns of this matrix of ones that this function once returns.
And if we look at this one function in more details by inspecting the pressing command control plus
I.
Well we see here in the parameters that the first parameter is this shape that is the shape of the matrix
of ones we want to create.
And as you can see we need to put it this way we need to put some parenthesis with inside the number
of lines and the number of columns.
So here we'll just need to input new parenthesis in which we add the number of lines which is 50 and
a number of columns which is 1.
So be careful this.
All this here with the parenthesis included is the first argument of this one's function.
Well this is the only argument you will need to input but just not to get confused with the parenthesis
.
This parenthesis in yellow here are the parenthesis of the one function and this parenthesis inside
are the parenthesis of the first argument of this one function which is the shape.
So as I just mentioned this is the only argument we need to put we don't really care about this and
therefore N.P. one is ready to return this column of 51.
That will be appended to a matrix of features X.
But now we need to do one more thing because otherwise we will get a data type error and to prevent
this data type error from happening we just need to convert this array of ones here into the integer
type and to do this it's very simple we just need to actually add as type and in parenthesis as we're
just doing this to prevent a data type error from happening you can try to execute this line without
this S-type and you'll get a data type error.
So now we're good.
And now we just need to add the last argument here which is access because you can use the pen function
here to add either a line of these values or a column of these values to the matrix here which is our
matrix of features X and therefore we need to specify if we want to add a column or a line.
So if we want to add a line then its access equals zero.
And if you want to add a column then its axis equals 1.
So here we'll just add access equals 1.
OK.
And now I'm sorry guys there is just one tiny little thing to do before this line is ready to be executed
.
It's related to the fact that right now we are indeed adding this column to 51 here to our matrix of
features X but we're actually adding it at the end of our matrix of features X. And we would like to
have it at the beginning.
So according to you what can we do here.
We can use a very simple trick to actually add this column of ones here at the beginning of our matrix
if you just x.
What is the trick.
Well as you can notice here the first two parameters are arrays.
The ones function is expecting an array and both are here and values.
So what if instead of adding this column in 51 here to our matrix X what if we added instead our matrix
X to this column or 51 here.
And so to do this it's actually real simple.
We just need to invert and put it in our this way.
And then for values when put X so that right now we are adding our matrix of features X to this one
column a 50 once here and there for this column of 50 ones here will appear as the first column of our
matrix A futurist X which is exactly what we want because it will then be easier for us to interpret
.
You know we have the multiple regression equation in mind with first the constant be zero and therefore
zero equals 1.
And then we have all our independent variables x1 x2 XM.
OK so now it's ready.
Sorry about that it was a little much but this is actually very useful for you because you will encounter
some libraries where you will need to add this kind of wants to your matrix of features.
And so you will just need to copy this line and you know put the right matrix in features if it's X
or another one.
And then this will just add this first column of ones to your matrix feature.
So this is actually very useful.
And this was worth spending a little time on this.
And now of course I'm going to show you that this actually adds this column of one sort of matrix that
features X at the beginning of the matrix.
So let's go to variable expert here.
Let's have a look at X right now.
So as you can see X contains all the independent variables without the first column of ones here.
So these are the two dummy variables for state.
This is already spent.
This is a R.D.Spend and this is marketing spend and now if I select this line and press command control
press enter to execute.
Let's now have a look at X and you will see that now x contains this first column of once here and this
column corresponds to this zero equals one associated to this constant zero and the multiple in our
regression equation y equals zero plus 1 plus 2 x 2 plus be an accent.
OK so let's press OK.
And now the moment we've all been waiting for we're going to start backward elimination but we're just
going to take a little break now and we start doing that in the next Statoil.
Until then enjoy machine learning