You've now learned about several highly effective neural network and 2
ConvNet architectures. 3
What I want to do in the next few videos is share with you some practical advice on 4
how to use them, first starting with using open source implementations. 5
It turns out that a lot of these neural networks are difficult or finicky 6
to replicate because a lot of details about tuning of the hyperparameters such 7
as learning decay and other things that make some difference to the performance. 8
And so I've found that it's sometimes difficult even for, 9
say, a higher deep loving PhD students, even at the top universities 10
to replicate someone else's polished work just from reading their paper. 11
Fortunately, a lot of deep learning researchers 12
routinely open source their work on the Internet, such as on GitHub. 13
And as you do work yourself, I certainly encourage you 14
to consider contributing back your code to the open source community. 15
But if you see a research paper whose results you would like to build on top of, 16
one thing you should consider doing, 17
one thing I do quite often it's just look online for an open source implementation. 18
Because if you can get the author's implementation, you can usually get going 19
much faster than if you would try to reimplement it from scratch. 20
Although sometimes reimplementing from scratch could be a good exercise 21
to do as well. 22
If you're already familiar with how to use GitHub, 23
this video might be less necessary or less important for you. 24
But if you aren't used to downloading open-source code from GitHub, 25
let me quickly show you how easy it is. 26
Let's say you're excited about residual networks, and you want to use it. 27
So let's search for residence on GitHub. 28
And so you actually see a lot of different implementations of residence on GitHub. 29
And I'm just going to go to the first URL here. 30
And this is a GitHub repo that implements residence. 31
Along with the GitHub webpages if you scroll down we'll have some text 32
describing the work or the particular implementation. 33
On this particular repo, this particular GitHub 34
repository was actually by the original authors of the ResNet paper. 35
And this code, this license under an MIT license, 36
you can click through to take a look at the implications of this license. 37
The MIT License is one of the more permissive or 38
one of the more open open-source licenses. 39
So I'm going to go ahead and download the code, and to do that, click on this link. 40
This gives you the URL that you can use to download the code. 41
I'm going to click on this button over here to copy the URL to my clipboard and 42
then go over here. 43
Then all you have to do is type git clone and then Ctrl+V for the URL and hit Enter. 44
And so in a couples of seconds it has download, 45
has cloned this repository to my local hard disk. 46
So let's go into the directory and let's take a look. 47
I'm more used in Mac than Windows, but I guess let's see, let's go to prototxt and 48
I think this is where it has the files specifying the network. 49
So let's take a look at this file, because this is a very long file that specifies 50
the detail configurations of the ResNet with a 101 layers, all right? 51
And it looks like from what I remember seeing from this webpage, 52
this particular implementation uses the Cafe framework. 53
But if you wanted implementation of this code using some other 54
programming framework, you might be able to find it as well. 55
So if you're developing a computer vision application, 56
a very common workflow would be to pick an architecture that you like, 57
maybe one of the ones you learned about in this course. 58
Or maybe one that you heard about from a friend or from some literature. 59
And look for an open source implementation and 60
download it from GitHub to start building from there. 61
One of the advantages of doing so also is that sometimes these networks take 62
a long time to train, and someone else might have used multiple GPUs and 63
a very large dataset to pretrain some of these networks. 64
And that allows you to do transfer learning using these 65
networks which we'll discuss in the next video as well. 66
Of course if you're computer vision researcher implementing these things from 67
scratch, then your workflow will be different. 68
And if you do that, 69
then do contribute your work back to the open source community. 70
But because so many vision researchers have done so much work implementing these 71
architectures, I found that often starting with open-source implementations 72
is a better way, or certainly a faster