Hey, everyone, and welcome back to this class, modern, deep learning in theno, intensive, slow,
deep learning in Python Part two.
In this lecture, I'm going to transition you to what I consider to be the second half of this course,
what you've seen up until now is what I consider to be the first half.
And it's all the material I had planned for the original incarnation of this course.
Originally, my goal for this course was to get you to go from writing a feed for neural network in
Mumbai, which was the prerequisite to this course, to writing a feed for neural network in V.A. intensive
low.
They, of course, also had the additional goal of introducing ways to speed up and improve neural networks.
So let's recap what we've done so far.
We looked at momentum and adaptive learning rates, which we've demonstrated can greatly speed up training.
We looked at choosing hyper parameters, which doesn't speed up training necessarily, but improves
the performance of your neural network.
We then looked at how to build the neural networks in theno intensive flow, which also doesn't directly
speed up training, but it lets us do automatic differentiation, which means we can build arbitrarily
complex neural networks without worrying about how to train them.
So it speeds up your ability to prototype new architectures.
And of course, one of the big advantages of V.A. intensive low, other than automatic differentiation,
is that they can take advantage of the GPU, which can make training orders of magnitude faster.
So that's everything I wanted to cover in the original incarnation of this course from here, you can
feel free to move on to any of the sequels to this course convolutional neural networks, which focuses
on using deep learning for images, unsupervised learning, which looks at the unsupervised learning
paradigm and how that applies to neural networks and recurrent neural networks where we incorporate
time or in other words, sequential data into deep learning.
There are more classes than this, but these are the three direct sequel's.
And these all require knowing how to use V.A. intensive flow, along with making use of techniques like
momentum.
At the same time, this course is not over.
So while you can at this point move on to other advanced deep learning courses, there is still some
more advanced topics in this course that I want to cover.
These sort of all fall into the category of modern deep learning.
So what do I mean by modern deep learning?
Well, it's essentially all the new techniques we've invented since the resurgence of deep learning
around 2010 and later, to give you one example, we're going to learn about dropout regularisation.
Basic vanilla regularisation is just L1 and L2 regularisation, where we add a penalty term to the cost,
but drop out, which was invented by Jeff Hinton's team, works in a completely different way where
instead of adding a penalty term, we randomly drop nodes in the neural network during training.
And all of these modern techniques will be implemented in V.A. intensive flow, so you'll get some extra
practice with using these libraries.
In a sense, all of this course is really about modern, deep learning, because the V.A. intensive
little libraries themselves are part of the modern era of deep learning.
In the old days, people would just write their neural networks and training algorithms from scratch,
which, of course, limited the complexity of those neural networks.
In the old days, people weren't taking advantage of special chips like the GPU and the Tipu to speed
up neural network operations, in the old days we didn't have the amount of data that we do now.
Nowadays, we have lots of data and lots of computing power, which has been instrumental to the success
of deep learning in recent years.
And all of the techniques you're learning in this course are part of this modern era.
This is opposed to the last course, which focused on some of the fundamental mechanics of deep learning
and the techniques we discussed there have been around for decades.
In other words, they are not what you would call modern.
At the same time, the techniques from this course all apply to feed for neural networks.
So while the sequels to this course introduce some new architectures, this course sticks with the same
architecture, but with new techniques added on top of it.
So hopefully that helps you understand how these techniques fit into the grand scheme of things, and
so just a reminder that after this lecture, it's going to be all V.A. intensive, slow and just practicing
implementing new stuff in these new libraries that you just learned about.