Welcome back.
In the last video, we discussed a little bit more about matrix multiplication, but we're not done
there.
We looked at two of the main rules of matrix multiplication and we saw a few errors of what happens
if those rules aren't satisfied, particularly if the inner dimensions don't match.
So this is what I've been alluding to as one of the most common errors in deep learning, and that is
shape errors, because neural networks are comprised of lots of matrix multiplication operations.
If you have some sort of tensor shape error somewhere in your neural network, chances are you're going
to get a shape error.
So now let's investigate how we can deal with those.
So let's create some tenses, shapes for matrix multiplication.
And I also showed you the website sorry, matrix multiplication, z z.
I hope you had a go at typing in some numbers here and visualizing what happens because we're going
to reproduce something very similar to what happens here.
But with PyTorch code shapes for matrix multiplication, we have tensor a let's create this as tensor.
We're going to create a tensor with just the elements one, two, all the way up to let's just go to
six.
Hey, that'll be enough.
Six Wonderful.
And then tensor B.
Can be equal to a torch tensor.
Of where are we going to go for this one?
Let's go 710.
This would be a little bit confusing, this one.
But then we'll go 811 and this will go up to 12, nine, 12.
So it's the same sort of sequence as what's going on here, but they've been swapped around.
So we've got the vertical axis here instead of one, two, three, four.
This is just seven, eight, nine, ten, 11, 12.
But let's now try and perform a matrix multiplication.
How do we do that to stop map mall for matrix multiplication?
PS Torch also has torch, which stands for matrix multiplication, which is a short version.
So I'll just write down here so that you know, tensor A tensor B.
I'm going to write.
Torch is the same as Torch.
Matt Mole.
It's an alias for writing less code.
This is literally how common matrix multiplications are and PyTorch is that they've made torch Mm.
As an alias for Matt Mole.
So you have to type for less characters using torch jawed Mm.
Instead of Matt Mole.
But I like to write Matt Mole because it's a little bit like it explains what it does a little bit more
than.
Mm.
So what do you think's going to happen here.
It's okay if you're not sure, but what you could probably do to find out is check the shapes of these.
Does this Operation Matrix multiplication satisfy the rules that we just discussed?
Especially this one?
This is the main one.
The inner dimensions must match.
Well, let's have a look.
Oh, no.
Matt One and Matt.
Two shapes cannot be multiplied.
Three by two and three by two.
This is very similar to what we went through in the last video, but now we've got some actual numbers
there.
Let's check the shape o torch size three, two, torch size three, two.
Now, in the last video, we created a random tensor and we could adjust the shape on the fly.
But these tenses already exist.
How might we adjust the shape of these?
Well, now I'm going to introduce you to another very common operation or tension manipulation that
you'll see, and that is the transpose to fix our tensor shape issues.
We can manipulate the shape of one of our tenses using a.
Let's go.
Transpose.
And so.
All right, here.
We're going to see this anyway.
But I'm going to define it in words.
A transpose switches the axes or dimensions of a given tensor.
So let's see this in action.
If we go.
And the way to do it is you can go tensor.
B don t Let's see what happens.
Let's look at the original ten to be as well.
So DOT stands for transpose and that's a little bit hard to read.
So we might do these on different lines.
It tends to be.
And get rid of that.
So you see what's happened here.
Instead of tense a b, this is the original one.
We might put the original on top.
Instead of the original one having seven, eight, nine, ten, 11, 12 down the vertical, the transpose
has transposed it to seven, eight, nine across the horizontal and ten 1112 down here.
Now if we get the shape of this tends to be a dot shape.
Let's have a look at that and let's have a look at the original shape tends to be don't shape.
What's happened?
Oh, no, we've still got three.
Two.
Oh.
That's what I've missed out here.
I've got a typo.
Excuse me.
I thought I was.
You think code that you've written is working, but then you realise you've got something as small as
just a dot T missing and it throws off your whole train of thought.
So you're seeing these errors on the fly here now tends to be is this but its shape is torture.
Dot size three two and if we try to matrix multiply three two and three 210 to A and tensor B we get
an error Y because the inner dimensions do not match.
But if we perform a transpose on tensor B, we switch.
The dimensions around.
So now we perform a transpose with tensor b.
T.
Taste for transpose we have.
This is the important point as well.
We still have the same elements.
It's just that they've been rearranged.
They've been transposed.
So now tends to be still has the same information encoded but rearranged.
So now we have torch size to three.
And so when we try to matrix multiply these, we satisfy the first criteria.
And now look at the output of the matrix multiplication of tensor A and tensor B dot T transposed is
three three, and that is because of the second role of matrix multiplication.
The resulting matrix has the shape of the outer dimensions.
So we've got three two matrix multiply two three result in a shape of three three.
So let's certify some of this and we'll print out what's going on here just so we know we can step through
it, because right now we've just got code all over the place a bit.
Let's see here The Matrix.
Multiplication operation works when Tensor B is transposed.
And in a second.
I'm going to show you what this looks like visually.
But right now we've done it with PyTorch code, which might be a little confusing, and that's perfectly
fine.
Matrix multiplication takes a little while and a little practice, so original shapes is going to be
tensor a shape.
Let's see what this is and tend to be equals tense about shape.
But the reason why we're spending so much time on this is because, as you'll see, as you get more
and more into neural networks and deep learning, the matrix multiplication operation is one of the
most or if not the most, common same shape as above, because we haven't changed tensor A's shape.
We've only changed tends to be shape or we've transposed it.
And though intense a B transpose equals we want tends to be a dot shape.
Wonderful.
And then if we print.
Let's just print out oops, print and spelt the wrong word.
There, print.
We want.
What are we multiplying here?
This is one of the ways.
Remember our motto of Visualize, visualize, visualize.
Well, this is how I visualize, visualize, visualize things shape.
Let's do the at symbol for brevity tensor and let's get b t shape.
We'll put it on our little role here in a dimensions must match and then print.
Let's get the output.
Output will put that on a new line.
The output is going to equal torch dot.
Our output is already here, but we're going to rewrite it for a little bit of practice.
Tensor a tend to be don't.
T and then we can go print output and then finally print.
Let's get it on a new line as well.
The output shape a fair bit going on here, but we're going to step through it and it's going to help
us understand a little bit about what's going on.
That's the data Visualize motto.
There we go.
Okay, so the original shapes are, what, Torch size three, two and torch size three, two.
The new shapes tend to a stays the same.
We haven't changed ten to A and then we have tens of Beatty is torch size 23.
Then we multiply a three by two, by a two by three.
So the inner dimensions must match, which is correct.
They do match two and two.
Then we have an output of tensor 27, 30, 33, 61, 68, 75, etc. and the output shape is what the
output shape is, the outer dimensions.
Three three.
Now of course you could rearrange this maybe transpose tensor a instead of ten to be have a play around
with it, See if you can create some more errors.
Trying to multiply these two and see what happens if you transpose tensor A instead of ten to be.
That's my challenge.
But before we finish this video, how about we just recreate what we've done here with this cool website
matrix multiplication?
So what did we have?
We had ten to a which is 1 to 6.
Let's recreate this, remove that.
This is going to be one, two, three, four, five, six.
And then we want to increase this and this is going to be seven, eight, nine, ten, 11, 12.
Is that the right way of doing things?
So this is already transposed, just to let you know.
So this is the equivalent of ten to be on the right here tends to be empty.
So let me just show you.
If we go.
Tend to be.
Dot transpose.
Which original version was that?
But we're just passing in the transposed version to our matrix multiplication website and then if we
click multiply.
This is what's happening behind the scenes with our code of Matt Moore.
We have one time, seven plus two times ten.
Did you see that little flippy thing that it did?
That's where the 27 comes from.
And then if we come down here, what's our first element?
27 When we matrix multiply them, then if we do the same thing, the next step we get 30 and 61 from
a combination of these numbers.
Do it again.
33, 68, 95 from a combination of these numbers again and again.
And finally we end up with exactly what we have here.
So that's a little bit of practice for you to go through is to create some of your own tenses can be
almost whatever you want, and then try to matrix multiply them with different shapes.
See what happens when you transpose and what different values you get.
And if you'd like to visualize it, you could write out something like this that really helps me understand
matrix multiplication.
And then if you really want to visualize it, you can go through this website and recreate your target
tenses in something like this.
I'm not sure how long you can go, but yeah, that should be enough to get started.
So give that a try and I'll see you in the next video.