Welcome to the first class on the chapter on machine learning.
We'll have a look at how to build simple models because in machine learning,
often the ruler is,
the simpler the model, the better.
Because simple models are easier to interpret read and are often very robust to noise.
Let's dive into it.
After loading our data,
we can import the linear regression model because we want
to predict house values and this exercise.
However, before we have to prepare our data in a certain way,
we need to split our data in two parts.
We want one training part and
we one set of data that the model has never seen during training time,
so we can validate that our model run something meaningful.
This is to avoid an effect that is called overfitting.
When our model basically remembers the training data and does not run
meaningful relationships between the data that
it can then apply to new data it has never seen.
That way, we take our data-frame and we split it into two parts randomly.
We could of course do this with a sub-setting that we
did in the previous section, however,
taking a random sample, that is,
absolutely sure to not overlap in any way,
is a much better way.
The train test split function that Scikit-learn supplies
is really good for this and it has some really need other functions that we can use.
This is also a really nice way to select our features.
For the simple model,
we'll just use the features of housing median age,
and then the total rooms as our training features.
The house value is going to be our target.
Those are usually saved in x and y.
So we know we have x train and x test,
and then we have y train and y test.
This is quite common.
We'll have a look at the shapes.
We have a bit over 20,000 rows here.
Our train data is going to be about 75 percent of that with 15,000 values and our y train
should have the same amount of targets because those are
sampled randomly but in the same row so the data obviously matches.
Our x tests should now have the remaining rows that are not in the train set.
Doing this is extremely important and there's
no way around splitting your data for validation.
Now it's time to build our model.
Our model is going to be the linear regression model that we imported before and
Scikit-learn makes it extremely easy for us to build models and assign models.
We just have to assign the object to some kind of variable, in this case,
we'll just call it model,
and you can see that you can change some of the hyper-parameters in the model,
but we'll keep it standard right now.
Now we fit our model to the data.
This is the training step where automatically our model is adjusted and the parameters in
our model are changed so that our model can predict y train from x train.
To score our model,
to test how well it's doing,
we can use the score method on our fitted model and
provided the data where we know the answers as well.
We can use x test and y test to see how well our model is doing on unseen data.
In this case, the regression is going to be the r-square.
R-square in statistic it's basically a measure of determinism.
How well does it really predict our data?
The best value there is one,
and then it goes down and can even be negative.
So 0.03 it's not impressive.
When we change our training data to include the median income,
we increase the score significantly.
Obviously, this is the most important part.
We have to find data that is able to give us information on other data that we want.
However, once we find that,
we can further improve our model by doing pre-processing on our data.
We have to be careful here though,
because we now do preprocessing and we'll test out different things,
if they work or if they don't.
What can happen is that we manually overfit our model.
That means to do proper data science right here.
We want to split our test data into two parts.
One validation holdout set and one test set.
The test set will not be touched in
the entire training process and not in our experimentation,
but only in the very very last part of our machine learning journey.
Here we define x-val and y-val and I made a little mistake here,
leaving that to y train that should of course be x test in the train test split.
Changing this means that this works,
and this is also a nice part about the train test split function.
It really makes sure that everything is consistent,
that all our variables match,
and we can see right here that
our test data set is now quite small with a thousand values.
So we can go back to the test split up here,
and actually provide a ratio of data that we can use.
In your data science and machine learning efforts,
you should always see that you can use the biggest test size you can afford really,
because that means you'll be able to have more certainty in your results.
Here can see that it's now split 50-50
and splitting our test set now further down into the validation set and the test set,
shows that our final test set has about 2,500 samples in there.
Which it's good enough for this case.
We'll define our standard scaler here and our model as the linear regression.
We fit our scalar onto the training data.
That means we can now re-scale
our entire data so that none of the columns are significantly larger than the others.
In a linear model,
we fit the slope and the intercept.
When we scale our data,
that means that our linear model now can work within the same ranges
of data and not be biased because one feature is significantly larger than the others.
We will create x scaled from our ex train data,
so we don't have to call the scalar transform in the training loop.
We can compare them,
here we can see that our scale data is now within,
well, centered around zero and all at the same scale.
Whereas before it was all over the place.
We can now fit our data on our model,
on the scale data.
Normal labels that we have obviously the label has to be y train in this case.
Then we can do the usual validation on our holdout data.
This case it's going to be x val and y val.
So we don't touch the test date or while we see
what kind of scaling and what kind of pre-processing works.
We have to transform our data because now our model expects scale data.
When we forget that,
we get terrible results,
and we can see that we improved our model by a small margin,
but it is still improved by just applying the scaling to the data.
If we try to use the robust scalar instead,
we can do this by just,
experimenting and using a different scalar.
This is the part which I mean where we need an extra holdout set
because just trying different things is a really good way to see what
works and is how you do data science.
Just seeing what sticks is really tantamount to building a good machine learning model.
Because sometimes you might not expect that you do have outliers in
your data and you try the robust scale and you see that it actually performs better,
or you realize that it performs worse.
Here we can train on our transformed data with a white terrain
again and score our results to check whether this works.
Try the min max scale that were used in previous class as well.
After we've done the experimentation and trying to find our model,
we can use this model to predict on any kind of data that has
the same columns and hopefully the same
distribution as our training data and the validation set.
To do this, we'll use model.predict and provide it with some data.
In this case, we'll use the training data just to have a look at how
the model stacks up against the actual ground truth data, the labeled data.
But of course, doing it on
the train data isn't the most interesting because the model has seen this data.
Eventually, we will do this on the test set.
But finally, you want to do this on
completely unseen data to actually get predictions from your machine learning model.
Another very nice way and why I really like
the train test split utility is that you can provide it with a keyword stratify.
Stratification is a means to make sure that some feature is
equally represented in each part of the train and test split.
If we want to make sure that our ocean proximity,
all of the island is in part in train and in part in tests,
we can do this by supplying this feature.
A reason why people like linear models so much is
because linear models essentially fit a line to your data.
If you think back to like fifth grade,
you may remember that a line is basically just the intercept
on the y and a coefficient for the slope.
What we can do is interpret our linear model and have a look at these coefficients.
Each column has a slope parameter right here.
Well basically, this parameter tells you how
much the slope of this data influences the prediction result.
Of course, we can have a look at the intercept with the y,
which gives us full overview of our model.
Essentially you could write it out on paper now.
In this class we learn how to use scikit-learn on
a simple machine learning model, a linear regression.
Basically fitting a line to our data.
We had a look at how scaling can improve our model and
even predicted on some data that the model has never seen.
So it's validating if we're actually learning something meaningful,
or if it just remembers the data.
In the next class,
we'll have a look at some more sophisticated models,
namely decision trees and random forests.