[MUSIC]
Okay, well like we discussed the other approach that we can take is to do
Gradient descent where we're walking down this surface
of residual sum of squares trying to get to the minimum.
Of course we might over shoot it and go back and
forth but that's the general idea that we're doing this iterative procedure.
And in this case it's useful to reinterpret this
gradient of the residual sum of squares that we computed previously.
So, this is what we've been working with.
But what I want to point out here is that this term,
Yi, this is our actual house sales observation.
And what is this term here?
Well, it's the predicted value if we use W0 and
W1 to form that prediction.
So, I'll call it the predicted value,
y at i, but I'm gonna write it a function of W0 and W1,
to make it clear that it's the prediction I'm forming when using W0 and W1.
Okay, so what we can do is re-write this residual sum of squares
in terms of these predicted observation values.
Then when we go to write our gradient descent algorithm,
what's the algorithm say?
Well we have, while not
converged, we're gonna take
our previous vector of W0 at iteration T,
W1 at iteration T and what are we going to.
We're going to subtract.
Going to write it up here, we're going to subtract eta times
the gradient, maybe I'll write it in two steps.
So we're subtracting eta times the gradient.
And what's the gradient?
The gradient is minus two
sum i = 1 to n,
yi- yi hat W0 to iteration T,
that's the value I'm using to predict my observation i, W1 at iteration T.
And likewise for the second component for W1.
But in this case, I'm gonna have to multiply by xi
because the gradient is a little bit different here.
W1t And then I'm gonna
multiply this by xi and
that is my update to form my
next estimate of w0 and w1.
Okay let me just quickly rewrite this in the way that I was going
to before where I see in both components of this gradient
vector I have this -2 term here and I have a minus eta.
So I'm going to bring that -2 out here.
So I'm just gonna erase this.
And rewrite this.
The minus two times minus eta is gonna turn into a plus sign.
So I'll just write this explicitly.
This was from minus two times minus eta.
Okay.
So we are doing gradient descent,
even though you see a plus sign, we're still doing gradient descent.
It's just that our gradient had a negative sign, and
that made it become a positive sign here, okay?
But I want it in this form to provide a little bit of intuition here.
Because what happens if overall, we just tend to be underestimating our values y?
So, if overall,
we're under predicting y hat i,
then we're gonna have that the sum
of yi- y hat i is going to be positive.
Because we're saying that y hat
i is always below, or in general,
below the true value yi.
So this is going to be positive.
And what's gonna happen?
Well, this term here is positive.
We're multiplying by a positive thing, and adding that to our W.
So, W zero is going
to increase.
And that makes sense, because we have some current estimate of our regression fit.
But if generally we're under predicting
our observations that means probably that line is too low.
So, we wanna shift it up.
And what does that mean?
That means increasing W0.
So, there's a lot of
intuition in this formula for what's going on in this gradient descent algorithm.
And that's just talking about this first term W0, but
then there's this second term W1, which is the slope of the line.
And in this case there's a similar intuition.
So, I'll say similar intuition, For W1.
But we need to multiply by this xi,
accounting for the fact that this is a slope term.
Okay.
So that's our gradient decent algorithm for
minimizing our residual sum of squares where,
when we assess convergence what we're gonna output is w hat zero, W hat one.
That's going to be our fitted regression line.
And this is an alternative approach to studying the gradient equal to zero and
solving for W hat zero and W hat one in that way.
[MUSIC]