In the last section we continued to try to figure out how to condense these two separate equations down
to this one equation using some matrix operations.
So at this point we've gone through multiplying our features by our weights and then subtracting out
our labels.
And so if I took just this term right here and summarize it into one single diagram what went on with
all that we would end up with something that looks like this right here.
So we took our features we appended on or concatenate it on that column of once then did the matrix
multiplication by our weights.
So that right there everything in the gray box essentially calculated the m x plus P B values.
That gives us back an array with all of our different x plus B values.
Then the next step was to subtract the actual values or the actual mpg of all of our different cars.
And that left us with a single tensor that we refer to as our differences.
So differences right here essentially represents in these two different equations.
It represents this part right here on equation for B and just this part right here for equation m.
So now an equation m you'll notice that we've got the multiplication of the difference by X at I.
So that's our individual feature value.
But then on the B1 we don't have any multiplication by X at all.
So now if we're really going to figure out how to combine both these equations down to this one we need
to figure out some way of multiplying the differences by the feature values when we're trying to calculate
M but not when we're trying to calculate B.
Now I'm sure that gets like really confusing very quickly when I start to you know list all that stuff
out.
But let's go to this last section and we're going to go to the last step of how we reduce these two
equations down to the 1.
And I think you're going to very quickly start to connect the dots and understand how these matrix operations
are the same as running these two separate equations at the same time.
OK.
So let's get to it.
So if I go back to my single equation over here you'll see that we want to take essentially that differences
tensor and we want to do matrix multiplication.
That's what the symbol right here is for with our features tensor.
So let's kind of walk through that process.
How do we multiply features by that.
Differences tensor that we've now learned how to calculate.
All right.
So here we go on the left hand side I've got my features with the concatenated column of ones on there.
And I want to do matrix multiplication with the differences that we just calculated in the last section.
The first step to matrix multiplication is to look at the shape and determine whether or not these two
tensors are eligible for matrix multiplication.
So the shape over here on the left hand side is six rows and two columns and then over here on the right
hand side we get six rows and one call.
So it can then take these two shape values together and we will compare the inner shape values.
Are they equal.
Are they exactly equal.
Well in this case no.
Obviously not two in six not equal.
So how are we going to multiply these two matrices by each other.
Are we just going to like out on a bunch more column of ones.
Well not quite.
In order to be able to multiply these two matrices we're going to use another little trick a trick similar
to what we did previously in concatenating all in this column of once.
Now again very similar to the column of ones you're going to say to me Stephen.
I don't understand how doing this trick does not dramatically change the results here.
How does this not dramatically change the calculation you're going to say that but just bear with me.
Just trust me as soon as we start to walk through the matrix multiplication.
After doing this little trick I am pretty convinced that for you everything is just going to kind of
click and you're going to say oh I see what you did there.
OK.
All right.
So let's do it.
Let's see what this little trick is that we're going to use to able to multiply these two matrices.
The trick we're going to use is to take the transpose of the matrix on the left.
We have not spoken about transposes yet when we take the transpose of a matrix we essentially take each
of its columns and we take that entire column right there and we turn it into a row instead.
And so you can kind of imagine that we're like twisting this matrix or this tensor so to speak.
Let's take a look at the next diagram and you're going to see what the transpose of this matrix is as
we go.
That's the transpose of the Matrix we were just looking at.
So we took the entire column right here and we turned it into a single row that spans out to the right.
We then take the second column right here and we turn it into the second row that spans out to the right.
So there's no math involved with it.
We're essentially just kind of reshaping the entire thing.
Take the entire column turn into a row take the next column turn it into the next row and repeat the
process for however many columns we have.
So again we've taken the transpose here and now you're probably thinking how does this not dramatically
change the result.
Again just bear with me for one second.
Let's go to the math now.
So I want to again calculate the different shape values.
So now for the transpose of our feature matrix on the left or shape is going to be 2 rows and 6 columns.
And then for the differences tensor on the right we have six rows and one column.
And so now when we put these two shapes together six is equal to six.
This is eligible for matrix multiplication and the resulting shape of the matrix that we get from multiplying
things together it's going to be a two by one tenth or a two by one matrix that means it will have two
columns or so be two rows and one column.
All right.
So looks like this will work out.
Now let's try going through the actual math here of how we would actually do the matrix multiplication
on these two tensors with each other.
So I did the same little sequence again.
Here's what we might refer to as matrix A and matrix B.
So we're going to go along each row or each column here sorry I mean to say like along a row.
That's why I say you throw.
So we're going to go along the row and we're going to take each one of these values and multiply it
with the corresponding value in this column right here.
So let's go 3:8 element by element.
So will be x 1 times difference one never.
Rather than referring to this as difference one let's just abbreviate it as like D1 and D to the 3 4
d5 and D6 just to make writing the stuff out a little bit more easy.
OK so do x 1 times D-1 that will be X-1 times 81.
Now the next term will be x 2 times D2.
So we're going to add on x 2 times to.
And then the next one is going to x 3 with D-3.
And I think you get the idea let's just write out the rest of the terms here.
X 3 do three plus x 4 times before plus x 5 times d 5 plus x 6 times d 6 6.
All right so now we'll move on to the second row here.
We're going to go across this row and still span the same columns.
So we'll do the first term.
That's going to be one time Steve one remember one time one is equal to D-1.
So we don't need to write out the one time part.
We can just put in here simply D-1.
And then the next term will be 1 times D2.
We're going to add that on.
And so we'll go right along that row and it's all going to be ones times do you one two three four five
six and six.
So this would be D-1.
Let's do two plus two three four d5 and D6 like so.
So that's the output right there of matrix multiplication between the transpose of our feature matrix
and the differences tensor that we had previously calculated.
Now these two terms right here might not look super meaningful.
Remember this represents an output matrix or an output tensor and it has two rows and one column.
Now before I tell you exactly what these things are I'm going to copy these and pull them back to a
diagram that we've been looking at quite a bit throughout the last like two dozen videos of this course.
It's going to copy those two back over here.
I'm going to zoom out for just a second.
I'm going to paste in both of those cells and then I want to match them up with one of these equations.
So again take that one right there and put it there and take that one right there.
I'm going to put it there.
And now my question to you is does it look like we just calculated the same thing that's mentioned in
each of these equations.
Let's admit that we like to end term out here in front and just consider the summation terms in here.
So remember this entire thing right here represents that difference that we had calculated we just went
through that entire calculation the last section we had said that was this stuff right.
Remember we had just said differences D1 36 represents just that term and differences represents just
that term without the ice on there.
And so we had said that for him we need to make sure that we multiply the differences by our feature
values x at one exit to exit 3 exit 4 and so on.
So when you look at these it looks like we kind of succeeded in doing exactly that or the slope with
respect to B.
We have taken all of our different values are all this differences that we calculated with that term
right there.
And we've gone through that summation step which is indicated by that ugly little symbol right there.
So we found d 1 plus 2 plus 3 4 5 and 6.
And then for the second where the slope with respect to him down here we took all of our differences
we multiply them by the respective feature values so like x 1 times D-1 x 2 times 2 and so on.
And then we went through that entire summation process which is what was indicated by that nasty little
symbol there and said that we had to take all those different multiplication there all those different
products and sum them all together.
So the product or the end result of doing the matrix multiplication between the transpose of our feature
matrix and our differences tensor over here is a two by one tensor that contains the exact slope values
for him and for B.
So all that stuff all the calculation we just went through over here and results of it is this two by
one tensor where the first entry in it is in him and the second and it is B.
Well sorry I say him I mean to say M's slope and the second is B slope.
So that's it.
That's how we went from taking the two separate equations and then using a lot of clever little tricks
with a little bit of linear algebra mixed in there as well to calculate r m slope in our B slope with
exactly one equation instead.
All right so I'm hoping that at this point some of this stuff makes a little bit of sense.
Now couple things I would remind you about.
Remember we've only calculated the slope at this point in time.
If we go back over to our code Ed. remember we calculated the slope right here and the slope right here.
And so once we got those slopes we still had to use them to actually update our guesses of in and B.
But hey considering that we just reduced all this stuff down to a series of like simple operations.
Well if you look at this stuff right here it's going to be just as easy or it seems like incredibly
easy to do this to these steps right.
You're using some more matrix operations.
So notice how we take the slope and B slope.
All we do is multiply by the learning rate which we can easily accomplished with tensor flow by using
the mole or multiply function.
And then we do subtraction between our current weights of M and B with the result of that operation.
So even though we didn't quite go all the way here we definitely got through all the by far the hardest
parts of learning how to calculate be so open in so at the exact same time.
OK so that's pretty much it.
Now the last thing I want to mention here.
Why did we go through all this trouble of like you know I don't know why do we go through all this right.
You probably think like Stephen this seems like a lot of work right.
Well there's a really good reason.
Up until this point everything that we've done around this linear regression and this gradient descent
stuff has all assumed that we had one single feature right.
We said all along that our only feature here is the car's horsepower.
But if you look back at our dataset inside of that car start CFE file you'll recall that we have many
other features in here.
Cylinder's displacement weights acceleration and those probably all factor in it very much to the MPG
value of the car.
So even though it seems like we just made our lives like more challenging they have to be in reality
this equation or all this work that we've just figured out and put together we can actually throw it
as many different features that we want at the same time and the exact same process is going to work
for additional features that we add in.
So even though we you know may things may be a little more challenging than we had to if we wanted to
add in dramatically more features than when we have right now we can use all the same exact code that
we're going to write in just a second to include many more features at the same time.
And you're going to see an example of this very quickly as we start to add in more than just our feature
current feature of the horsepower of the vehicle.
Okay so enough for me ranting about this stuff let's take a quick pause right here.
When we come back the next section we're going to work on our linear regression.
Just file.
We're going to do a big factor to this thing so that it starts to use this equation here as opposed
to the two separate equations that were a real pain in the rear to code up previously.
So quick pause and I'll see you in just a minute.