In the last section, we continued to try to figure out how to condense these two separate equations
down to this one equation using some matrix operations.
So at this point we've gone through multiplying our features by our weights and then subtracting out
our labels.
And so if I took just this term right here and summarized it into one single diagram, what went on
with all that?
We would end up with something that looks like this right here.
So we took our features, we appended on or concatenated on that column of ones, then did the matrix
multiplication by our weights.
So that right there, everything in the gray box essentially calculated the X plus B values.
That gives us back an array with all of our different MX plus B values.
Then the next step was to subtract the actual values or the actual miles per gallon of all of our different
cars.
And that left us with a single tensor that we referred to as our differences.
So differences right here essentially represents in these two different equations.
It represents this part right here on equation for B and just.
This part right here for Equation M.
So now on Equation M, you'll notice that we've got the multiplication of the difference by X at I.
So that's our individual feature value.
But then on the B one we don't have any multiplication by x at I.
So now if we're really going to figure out how to combine both these equations down to this one, we
need to figure out some way of multiplying the differences by the feature values when we're trying to
calculate M but not when we're trying to calculate.
B.
Now I'm sure that gets like really confusing very quickly when I start to list all that stuff out.
But let's go through this last section and we're going to go through the last step of how we reduce
these two equations down to the one.
And I think you're going to very quickly start to connect the dots and understand how these matrix operations
are the same as running these two separate equations at the same time.
Okay, so let's get to it.
So if I go back to my single equation over here, you'll see that we want to take essentially that difference
is tensor and we want to do matrix multiplication.
That's what this symbol right here is for with our features tensor.
So let's kind of walk through that process.
How do we multiply features by that differences tensor that we've now learned how to calculate?
All right.
So right here, here we go.
On the left hand side, I've got my features with the concatenated column of ones on there, and I want
to do matrix multiplication with the differences that we just calculated in the last section.
And the first step to matrix multiplication is to look at the shape and determine whether or not these
two tensors are eligible for matrix multiplication.
So the shape over here on the left hand side is six rows and two columns.
And then over here on the right hand side, we get six rows and one column.
So we can then take these two shape values together and we will compare the inner shape values.
Are they equal?
Are they exactly equal?
Well, in this case, no, obviously not.
Two and six not equal.
So how are we going to multiply these two matrices by each other?
Are we just going to, like add on a bunch more columns of ones?
Well, not quite.
In order to be able to multiply these two matrices, we're going to use another little trick, a trick
similar to what we did previously in concatenating on this column of ones.
Now, again, very similar to the column of ones, you're going to say to me, Stephen, I don't understand
how doing this trick does not dramatically change the result here.
How does this not dramatically change the calculation?
You're going to say that, but just bear with me.
Just trust me, because as soon as we start to walk through the matrix multiplication after doing this
little trick, I am pretty convinced that for you everything is just going to kind of click and you're
going to say, Oh, I see what you did there.
Okay.
All right.
So let's do it.
Let's see what this little trick is that we're going to use to be able to multiply these two matrices.
So the trick we're going to use is to take the transpose of the matrix on the left.
We have not spoken about transposes yet.
We take the transpose of a matrix.
We essentially take each of its columns and we take that entire column right there and we turn it into
a row instead.
And so you can kind of imagine that we're like twisting this matrix or this tensor, so to speak.
Let's take a look at the next diagram and you're going to see what the transpose of this matrix is.
There we go.
That's the transpose of the matrix we were just looking at.
So we took the entire column right here and we turned it into a single row that spans out to the right.
We then take the second column right here and we turn it into the second row that spans out to the right.
So there's no math involved with it.
We're essentially just kind of reshaping the entire thing, take the entire column, turn it into a
row, take the next column, turn it into the next row and repeat the process for however many columns
we have.
So again, we've taken the transpose here and now you're probably thinking, how does this not dramatically
change the result?
Again, just bear with me for one second.
Let's go through the math now.
So I want to again calculate the different shape values.
So now for the transpose of our feature matrix on the left, our shape is going to be two rows and six
columns.
And then for the differences tensor on the right we have six rows and one column.
And so now when we put these two shapes together.
Yep.
Six is equal to six.
This is eligible for matrix multiplication.
And the resulting shape of the matrix that we get from multiplying these things too together is going
to be a two by one tensor, a two by one matrix.
That means it will have two columns or two rows and one column.
All right.
So looks like this will work out.
Now let's try going through the actual math here of how we would actually do the matrix multiplication
on these two tensors with each other.
Okay.
So I did the same little sequence again.
Here's what we might refer to as Matrix A and Matrix B.
So we're going to go along each row or each column here.
Sorry.
I mean to say like along a row.
That's why I say each row.
So we're going to go along the row and we're going to take each one of these values and multiply it
with the corresponding value in this column right here.
So let's go through it element by element.
So it'll be x one times difference one.
Now, rather than referring to this as difference one, let's just abbreviate it as like D1 and D2,
d3, d4 d5 and D6 just to make writing this stuff out a little bit more easy.
Okay, so we'll do x one times d one.
So that will be x one times D one.
Now the next term will be x two.
Times D2.
So we're going to add on x two times D2.
And then the next one is going to be X three with D three.
And I think you get the idea.
Let's just write out the rest of the terms here.
X3D3 plus x four times d, four plus x five times d, five plus x six times D66.
All right.
So now we'll move on to the second row here.
We're going to go across this row and still span the same columns.
So we'll do the first term.
That's going to be one times D one.
Remember one times D, one is equal to D one.
So we don't need to write out the one times part.
We can just put in here simply d one and then the next term will be one times D two.
We're going to add that on.
And so we'll go right along that row and it's all going to be once times D one, two, three, four,
five, six and six.
So this would be D one plus D, two plus D, three, D, four, D, five and D six like so.
So that's the output right there of matrix multiplication between the transpose of our feature matrix
and the differences tensor that we had previously calculated.
Now these two terms right here might not look super meaningful.
Remember, this represents an output matrix or an output tensor and it has two rows and one column.
Now, before I tell you exactly what these things are, I'm just going to copy these and pull them back
to a diagram that we've been looking at quite a bit throughout the last like two dozen videos of this
course.
Okay.
So I'm going to copy those two back over here.
I'm going to zoom out for just a second.
I'm going to paste in both of those cells and then I'm going to match them up with one of these equations.
So I'm going to take that one right there and put it there and take that one right there.
I'm going to put it there.
And now my question to you is, does it look like we just calculated the same thing that's mentioned
in each of these equations?
Let's omit the two n term out here in front and just consider the summation terms in here.
So remember, this entire thing right here represents that difference that we had calculated.
We just went through that entire calculation in the last section.
We had said that was this stuff right here.
Remember, we had just said differences D one through D six represents just that term and differences
represents just that term without the X's on there.
And so we had said that for M we need to make sure that we multiply the differences by our feature values
X1X2X at three, x at four and so on.
And so when you look at these, it looks like we kind of succeeded in doing exactly that for the soap.
With respect to B, we have taken all of our difference values or all this differences that we calculated
with that term right there.
And we've gone through that summation step which is indicated by that ugly little symbol right there.
So we found D one plus D, two plus three, four, five and six, and then for the second or the slope
with respect to M down here, we took all of our differences.
We multiplied by them by the respective feature values.
So like X one times D, one x, two times D two, and so on.
And then we went through that entire summation process, which is what was indicated by that nasty little
symbol there, and said that we had to take all those different multiplications or all those different
products and sum them all together.
So the product or the end result of doing the matrix multiplication between the transpose of our feature
matrix and our differences tensor over here is a two by one tensor that contains the exact slope values
for M and for B, So all that stuff, all the calculation we just went through over here, the end result
of it is this two by one tensor where the first entry in it is M and the second in it is B Well, sorry,
I say M I mean to say M slope and the second is B slope.
So that's it.
That's how we went from taking the two separate equations and then using a lot of clever little tricks
with a little bit of linear algebra mixed in there as well to calculate our m slope and our B slope
with exactly one equation instead.
All right.
So I'm hoping that at this point, some of this stuff makes a little bit of sense.
Now, a couple of things I want to remind you about.
Remember, we've only calculated the slope at this point in time.
If we go back over to our code editor, remember, we calculated the B slope right here and the slope
right here.
And so once we got those slopes, we still had to use them to actually update our guesses of M and B,
But hey, considering that we just reduced all this stuff down to a series of simple operations, well,
if you look at this stuff right here, it's going to be just as easy or it's going to be incredibly
easy to do this these steps right here using some more matrix operations.
So notice how we take the slope and B slope.
All we do is multiply by the learning rate, which we can easily accomplish with TensorFlow by using
the Mul or multiply function.
And then we do subtraction between our current weights of M and B with the result of that operation.
So even though we didn't quite go all the way here, we definitely got through all the by far the hardest
parts of learning how to calculate B slope and M slope at the exact same time.
Okay, so that's pretty much it.
Now, the last thing I want to mention here.
Why did we go through all this trouble of like, you know, I don't know.
Why did we go through all this?
Right.
You're probably thinking like, this seems like a lot of work, right?
Well, there's a really good reason.
Up until this point, everything that we've done around this linear regression and this gradient.
Sent stuff has all assumed that we had one single feature.
Right.
We've said all along that our only feature here is the car's horsepower.
But if you look back at our data set inside of that car's CSV file, you'll recall that we have many
other features in here.
Cylinders, displacement, weight, acceleration and those probably all factor in very much to the mile
per gallon value of the car.
So even though it seems like we just made our lives like more challenging than they have to be in reality,
this equation or all this work that we just figured out and put together, we can actually throw it
as many different features that we want at the same time.
And the exact same process is going to work for additional features that we add in.
So even though we made things maybe a little bit more challenging than we had to, if we wanted to add
in dramatically more features than what we have right now, we can use all the same exact code that
we're going to write in just a second to include many more features at the same time.
And you're going to see an example of this very quickly as we start to add in more than just our feature
current feature of the horsepower of the vehicle.
Okay.
So enough of me ranting about this stuff.
Let's take a quick pause right here.
When we come back to the next section, we're going to work on our linear regression JS file.
We're going to do a big refactor to this thing so that it starts to use this equation here as opposed
to the two separate equations that were a real pain in the rear to code up previously.
So quick pause and I'll see you in just a minute.