Hello my friends.
Welcome to this new practical activity on polynomial regression.
This time we're going to learn how to build together
a nonlinear regression model, which will allow us to tackle a problem
with a non-linear data set, meaning a data set with nonlinear relationships
on which therefore a multiple linear regression model would not be relevant.
Now we're all going to go into part two regression,
and this time we're going to go to section six polynomial regression to learn
how to build indeed this nonlinear regression model.
All right.
And as usual we're going to start with Python inside which you will find two
files polynomial regression ipynb which is of course your Python implementation
which you can open either in Google Colab or Jupyter Notebook.
And the data set called position salaries.
All right.
So as usual we going to start by describing the data set.
Once again I'd like to remind that this is a simple data set.
But no worries.
The further we progress in this course, the more we will work with real world
and complex data sets.
You will see at the end we will work with data sets with many more observations
and more complexities.
So what is this data set about?
Well let's imagine the following scenario.
Let's imagine that we are actually an HR department
and that we want to hire someone.
And we actually found someone that seems to be a great fit for the job.
So we would like to offer this person a position in our company.
And so this person says yes, but at the end of the interview process
comes the inevitable question what is your salary expectation?
And let's say that this person is, you know, very well advanced in its career,
and therefore that person is asking for $160,000 per year.
And then also as HR negotiators,
we ask this person, why are you expecting such a high salary?
And this person replies,
well, that's because that's what I earned in my previous company.
That was my salary.
In my previous company, I earned $160,000 per year.
So I'm expecting at least $160,000 per year in your company.
Is that the truth, or is that a bluff?
Well, that's exactly what we're going to figure out.
Thanks to our polynomial regression model.
We're going to build a polynomial regression model
to predict the previous salary of this candidate.
So how are we going to do this?
Well, of course in order to make such a prediction we need data.
And that's exactly the data we collected here.
So what is this data and how did we collect it.
This data is actually the different salaries of the previous company
for the different positions from business analyst to CEO.
And now how did we collect such data.
Well you know there are many websites online which actually display
the different salaries of the different positions in companies.
I can give you an example like Glassdoor.
Well, let's say that we did this and that's how we collected all this
data containing all the salaries for the different positions
of this previous company for which this person worked.
Okay.
So we have this data and now we need to know obviously
which position this person had within this previous company.
Well that's easy.
Let's say we went to LinkedIn and we checked out the profile of this person.
And we actually saw that this person was actually a region manager.
Okay.
However, on the LinkedIn we also see something else.
It turns out that this person actually has been a region manager
for quite a while, like let's say two years and therefore,
you know, the salary of this person should not exactly be $150,000,
as we can see on this data set, but instead it should be
somewhere between 100 and $50,000, the salary of position
number six and $200,000 the salary of position number seven.
So in order to extrapolate, we're going to suppose that
this person has a position in between 6 and 7.
And we'll consider this position to be 6.5,
so that then we can actually deploy our model,
you know, after training it of course, on the position level 6.5,
so that we can get the predicted salary of such a position level.
And we will compare this predicted salary to the salary expected by this person,
to see if indeed there is truth or bluff.
All right.
Are you ready?
Let's do this.
Let's build our polynomial regression model.