Hello and welcome to this tutorial here we will loan gradient descent.
Let us begin below.
We have understood there are two key aspects with which a neural network loans.
And these are activation function and cost function.
We are still missing a key step here.
That is the actual learning process of neural network to understand the learning process be how to understand
gradient descent
the gradient descent is an optimization algorithm that is used for minimizing the cost function minimizing
the cost function means we are minimizing the error the gradient descent update the various parameter
of a machine learning model to minimize the cost function.
Let us understand how the gradient descent works and minimize the cost function.
As you can see here in this diagram on the x axis we have weights and on the y axis we help cost function
and we we're considering this in only one dimension so here we have taken a random value of cost function.
Now we have to minimize that cost function and to do that we have to apply the gradient applying the
gradient means we have to take the derivative of that value here.
The gradient descent is applied here to minimize the cost function after applying the gradient cost
function is minimizing now.
And at the end of this process we will get an optimized cost function.
So this is the pictorial representation of gradient descent in one dimension.
Finding the minimum value of a cost function looks very simple in one dimension but in our cases we
will have multiple parameters that is we will have multiple dimensions so we will use the built in linear
algebra libraries in deep learning to minimize the cost function
after understanding the gradient descent.
We help to understand back propagation using gradient descent.
We can figure out the best parameters for minimizing the cost function.
Now the question arises that how can we adjust the optimal parameters of weights across the entire network
and to do that views back propagation
back propagation is used to calculate the error contribution of each neuron after a batch of data is
processed back propagation calculates the error at output and after that it distributes that output
back throughout the network layers to do that.
It requires a known desired output for each input value.
So this is how back propagation works
as you can see here.
This is the pictorial representation of forward propagation in this process error contribution of each
neuron as well as error at output is calculated
and in back propagation these calculated errors are distributed throughout the network.
The process of forward propagation and backward propagation is done multiple times and at the end we
get optimized value of cost function.
So district ordeal about the gradient descent and back propagation ends here.
I will see you in the next one.
Till then Happy learning.