Hello and welcome back to the course on Deep Learning.
In today's tutorial, we're going to walk through the steps
that are included in the process of training an autoencoder.
So let's get started.
Here we've got an autoencoder and obviously get some inputs.
And then we'll get some outputs.
And in our case it'll get input as inputs will get the ratings of lots
and lots of users for these six movies.
And based on those ratings, it will come up with a way
to compress that information and, come up with those weights
that will allow it to encode or to basically set itself
up to be able to then encode it and decode it in the future.
So here we go.
These are our training steps.
Step one we need to start with you.
Start with an array
where the lines are correspond to users, and columns correspond to features.
The movies and each cell contains a rating from 1 to 5 zero.
If there's no rating, by that user, you so important to understand that
every single row in your data set is a unique user who rated those movies.
And by the way, these steps will be very valuable for you
when you are performing them or when you're doing the practical tutorials.
Because you will be actually working with exactly this situation.
You'll be creating a recommender system for, movies.
Step two the first user goes into the network.
The input vector contains all its ratings for all the movies.
So all the ratings of that user for all of the movies.
Step three the input vector is encoded into vector
Z of low dimensions, by a mapping function.
usually you'd use a sigmoid or hyperbolic tangent, as we mentioned.
And that's the equation that you would have.
Then step four z is then decoded into the output
vector of the same dimension as x aiming to replicate the input vector.
So this is all pretty straightforward stuff.
The reconstruction era
which is what how different is the output compared to the input is computed.
And the goal is to minimize your reconstruction era and then all this,
error is back propagated from right
to left through your network, and the weights are updated accordingly,
depending on how much they're responsible for that.
So you've got a gradient descent process happening there.
And the learning rate describes how much we update the weights.
And that's a parameter you can tune.
during you will be tuning or you can tune during decoding.
And you will see that in the practical tutorials.
And step seven repeat steps 1 to 6 and update weights after each observation.
That's reinforcement learning.
If it's done after each observation or if you do it after batches,
they'll be batch learning.
And finally step eight the when the whole training passes
through the neural network, that's an epoch.
And then just redo more epochs.
So there we go. Those are steps.
Let's have a look in a more visual way.
So step one that's our input.
In this case this input is the same file as we used for the Boltzmann machine.
Or same example in you can have ratings from 1 to 5.
As we mentioned in the example, 1
to 5 means basically more kind of a more advanced recommender system.
And then a zero would mean empty.
In our case, we got one zero and empty.
we can we can easily change them.
We can change this one minus one and put a zero in the empty cell.
So it's very easy to adjust.
And also you can you can build a recommender system
with ratings with binary ratings one and zero,
or one minus one depending on how you want to structure it.
It will be a good recommender system, but of course it will be
a more powerful recommender system if you build it with ratings from 1 to 5.
And in the practical tutorials,
good news, you'll be building one with ratings from 1 to 5.
Okay, so there we go.
That's our, input.
Then step two, we take the first row,
we put it into our our encoder.
And then step three, we calculate the hidden nodes.
voila.
And again at the very start you'll have some randomly initialized weights.
Then that process is called encoding.
Step four.
We're going to calculate our visible output nodes again with.
Initially it'll be some randomly random starting weights.
That's process called decoding.
And then so basically this is a summary.
That yellow arrow is a summary.
We've just put the information
the data through our our encoder from left to right.
And step five
we're going to compare the results to the actual ratings for those movies.
We'll be 123456.
And then we'll calculate the error.
Step six.
And then we will propagated back through the network.
We will adjust the weights accordingly.
And next we will take the next row in our data set.
And we'll do the same thing.
And so on and so on and so on.
We'll continue through all the rows.
And then once we're done we move on to step eight.
That means we've finished a whole epoch.
And then we just repeat these epochs.
We continue doing that
with our data set depending on how many times we want to do that.
And hopefully our training converges to some sort of value.
There we go.
That's how you train on our encoder.
Very straightforward steps.
And as I mentioned at the start of this section,
that simply because you've already done all of the previous parts of this course,
all of this probably sounds very trivial to you.
If you look back at how you,
we're starting out into this course,
this probably would have been a bit more advanced.
But if this sounds trivial, that's good news.
That just means that all of the knowledge that you've acquired
throughout the course before, is benefiting you now.
And that's great.
That's why we were able to fly through this part very quickly,
and we can move on to some more interesting things.
And by the way, additional reading.
We've got some additional reading today.
Here is a blog by Francois Shalit, the creator of Keras.
It's called building Autoencoders in Keras.
check it out.
It's got some actual practical applications.
So even if you want to get some hands on experience before you start
the practical choice, we've had learned this could be a good place to to do that.
There's some quick exercises
which you can follow and get some cool insights already,
and then you'll be even more prepared when you go into the practical tutorials.
So check it out.
It's called Building Autoencoders in Keras and that's it for today.
I look forward to seeing you next time and until then, enjoy deep learning.