what's going on everybody and welcome to
part three of our deep learning with
Python tensorflow and chaos tutorial
series in this video what we're gonna be
covering is convolutional neural
networks and applying the data set that
we worked on building in the last video
to a convolutional neural network in
hopes that we can get this neural
network to classify dogs versus cats now
first what we should do is cover quickly
how convolutional neural networks work
just to get some of an intuition of why
we use confidence with imagery types of
data so let me just pull this over from
the text-based version I didn't really
want to have to redraw everything again
so in general you're gonna have the
following steps are gonna be taken
you're gonna do some convolution and
then pooling and then convolution again
in pooling so let's talk about that so
first of all the the idea is convolution
is to find you know some sort of useful
thing and then we pool these useful
things together for the combination of
convolution and pooling for your
convolutional neural network so the idea
this is like let's say you take an image
and then we convert that image to pixel
data so this is just my you know bad
representation of pixels but anyways you
converted to grid or after it's been
converted to pixels like this in theory
could be a convolution on top of already
you know there's already been a few
convolution layers but anyways the idea
is then you have a window like a
convolutional window in this case it's a
3 by 3 window and in that window the
convolution is going to attempt to
simplify what it finds down to some sort
of value now when it does that the
window is going to shift over and do the
exact same thing just a bunch of times
so in general you've got the window
you've got how big of a stride over do
you want to take and so on now with
Kerris you can really just specify the
window size and that's it and like
everything else like someone was asking
in the previous well the first video
about bias if you want to learn more
about the deeper details of deep
learning go to the practical machine
learning too
toriel series where everything is much
more deeply explained because it has to
be deeply explained because the way we
were writing that was with raw
tensorflow code so definitely go check
that out if you want to see more of the
inner workings and how things are done
like for example as you keep sliding
what happens when you slide over the
edge you know most cases we're probably
gonna Pat it but then do you want to Pat
it like or same or all this stuff so
anyways getting back to what we're doing
here so it'll produce features so in
general and then you'll do it many many
times so it's a way to kind of slowly
extract features from an image so
convolution convolution and let's say
the convolution produces a 2 a 1 a 4 and
a 3 now these are it's not gonna look
just like that but just for simplicity's
sake let's say it's a 2 1 4 3 then let's
say you've got a bunch of convolutions
right this is just a bunch the next
thing that's gonna likely happen you
don't have to but this is pretty common
is to do a pooling layer so in general
the most common form of pooling is max
pooling so again let's say you do a max
pooling of a 3 by 3 window well all the
max pooling does is find the max value
so in this window the maximum value it's
a 4 then it shifts it over and again and
again so you can see how it works
basically down here so that's
convolution in max pooling now the
higher-level idea of how this actually
works is basically it's it's slowly
extracting value from the image so lower
the more initial layers in a
convolutional neural network are going
to be finding things like edges and
lines and then as you go you're gonna
find like maybe maybe squiggles and then
maybe you'll find circles and then maybe
you'll find squares and then soon you're
gonna find I don't know like very small
like features so for example if you're
curious I don't know if this is gonna
play some sound it is let me just mute
it this was a deep dream and what you
can see actually going through this is
how what a convolutional neural network
is kind of basically a deep dream is
going to extend
- wait what it finds so these are the
initial layers where it just finds a
bunch of straight lines and then as you
go a little little deeper you start
finding slight curvatures and lines and
then as you continue on you'll find
hopefully something different yeah so
like you get kind of more square shapes
right and then as you continue along we
start getting like faces and eyes stuff
like that as we continue even further
you start getting like you can almost
recognize a dog there and then as you
get further and further in the network
you start seeing things like scales and
stuff like this was trained on like what
was it not pets I'm trying to think of
the term but this was off of I'm sure I
think it was the image I think was the
image net data set anyways this was
inception off of the image net data set
but anyway you can check out a deep
dream of a neural network if you want
also you can check out the that series
the unconventional neural network series
for more on more interesting projects
like that so ok so the first thing that
we want to do is bring in tensor flow
and all the things that we need to make
a convolutional neural network so we're
going to not do that we're going to
import tensor flow as TF we're gonna say
from tensor flow Karos donna datasets
actually we don't need to import any
daisies just kidding what we're gonna do
first is bring in from tensor flow dots
Quiroz models we're gonna import
sequential because we're gonna use that
sequential model then from tensor flow
caras layers we're gonna import
dense so your typical dense layer we're
gonna import dropout we're gonna import
activation and so first of all we're
going to import the the dense layer
because a lot of times you're going to
end either a recurrent net or a
continent with a final dense just fully
connected layer and then also flatten
and we're going to use flatten before we
feed it through that final dense layer
then we also want to bring in con 2 D
capital D and then max pooling 2 D okay
so then we're gonna import or pickle
pickle and we're gonna do that pickle or
actually we could just do this we can
say x equals pickle dot load and we'll
load in open X dot pickle pickle RB and
then we'll do the same thing for y so X
x and y so then the first thing we want
to do before we feed data through a
neural network is consider normalizing
that data the easiest way to normalize
data is going to be to scale that data
in our case we're using imagery data so
actually the easiest way to scale the
imagery data is we know what the maximum
value and the men value is the min is 0
the max is 255 for pixel data so we can
actually just do that / 255 need more T
so but there is I think it's chaos dot
utils dot normalize I think it is
anyways in many cases you're gonna need
something more like that but for now we
can actually just do x equals x divided
by 255 then once we've done that we can
come down and begin building our model
so we'll start with not a capital m
model equals sequential because there's
just gonna be a simple sequential model
and then we're gonna start using that
model dot add syntax that we've been
using so we're gonna say conf 2d so
we're gonna start with that
convolutional layer and this can be we
could probably will probably go away
with almost anything we pass here I'm
gonna pass 64 though and then we're
gonna pass the window that we want to
use
three by three and then finally the
input underscore shape is gonna be evil
to the X dot shape and then here you can
just say 1 : so we're gonna skip that
first like negative 1 so each because
negative 1 again is was how many of
those feature sets did we have right so
we don't need that cuz that's not
actually the input shape it's gonna be
the shape of the data we're actually
throwing in so it's not negative 1 but
it is you know 50 by 50 by 1 that's true
so that is how we can get that input
shape really dynamically so once we've
done that the next thing that we're
gonna do is I feel like we shouldn't be
okay
whatever the next thing we're gonna do
is model dot add and we're gonna add so
in this case you so the the the max
pooling again to have so in general
convolutional and max pooling are gonna
go together so in this case what we
could say is after the convolutional
layer we could pass either an activation
or a pooling I'm gonna go with we're
gonna just do an activation layer so
activation and the activation will go
with here is just the same one that
we've been using which is rectified
linear then we're gonna do the pooling
so model dot ad I'm just gonna copy this
actually copy paste max pulling to D and
then again here you're gonna specify the
pool size and let's just do a 2 by 2
okay so that's our first sort of layer
now we can do this again and this time
we don't need is an input shape so now
we've got a 2 by 64 layer convolutional
neural network and then like I said
before we're gonna pass a final the you
don't have to do this again we probably
don't even need it for this data set but
just for good
model dot add will add a final dense
layer and that's going to be a
sixty-four node now before you do that
you have to flatten the data because
convolutional is 2d whereas that dense
layer again once a 1d data set so we're
gonna say model that add and then we
just throw in a flattened don't forget
whenever you want to do stuff like this
you do have to import that layer but a
flattened n64 cool so then now all we
need is that output layer so model dot
add and we'll add a dense and we'll go
with one and then we need that
activation layer or the activation for
it add I feel so silly calling that a
layer but it is coming from the layers
I'm kind of surprised that we would call
activation for example a layer but ok
that's what we're doing so anyways Marla
dot add and then activation I mean I
guess it's one more thing but in though
in the old ways of doing things like you
would because you could also say dense
so like here you could say dense one and
then you can actually pass activation
equals something so there's a kind of
two ways to do it so I'm not really sure
why they have that in layers but anyways
activation someone could correct me if
I'm wrong but I don't really think like
I think if you kissed her if you walked
around calling the activation function a
layer in a neural network like if you
considered this neural network to not
you know when I look at this I see one
layer two layer three layer and then
like the output layer so really I would
say this is three layers but if you
started calling this a one two three
four five six seven eight nine ten layer
ComNet I think most people would
disagree but I don't know they're
throwing that in layers so I don't know
if that kind of opinion has changed in
the ml community or what
anyways model dot compile and that will
do loss loss equals and we're going to
use categorical actually will use so you
could use cats dogs and cats so in this
case
we can use categorical or I'm sorry
binary Chris Wow how many times my trip
of that but you could use categorical
and then optimizer in this case we're
gonna go with Adam because that's what
we always go with and then metrics
equals accuracy now what we want to do
is run the fitment so we're just going
to say model that fits X Y we can
specify the batch size this is how many
at a time do we want to pass again we
got 25,000 samples you probably want to
pass one at a time but you also don't
really want to pass all at the same time
it's generally a good idea to there's
something between like somewhere in the
20s to maybe 200 usually I find negative
results start coming in if I start
trying to do like a batch size of like
thousands but it depends on the size of
your data so it is really gonna kind of
scale with how big your data is if
you've got like a data set of a billion
samples you could probably get away with
a batch size of a few thousand but
anyways yeah so my outfit batch size
we'll talk more on that in a little in
the next tutorial so anyways validations
split this is the kind of out-of-sample
data rs8 we're gonna save 30 percent and
in fact that's a pretty high let's have
to 30 percent let's just do 10% so a
model that fit 10% and great so let's
let's train the model make sure we don't
hit an error we did X looks like a
lowercase X we need to uppercase the X I
knew I felt weird there let's see 1 or a
kernel size do ok now I know what I've
done I knew I felt like I was like wait
something is wrong here so I closed off
with this 64 and we need to put it here
so so I always get confused with why are
you doing it I would get confused when
it comes time to do parentheses why I'm
doing tutorials so let me just space
everything out row nicely so this is the
conv layer right it's 64 units followed
by the next parameter which is the the
window size followed by the next
parameter which is the input shape okay
oh my gosh how many oh we did it cuz we
copy and paste it got it
chickity-check please no more errors
they'll be really nice all right so I
think we're training finally yes all
right so we're already at 64 or 65 we're
doing pretty good I don't know if we set
oh we didn't specify epochs so let's go
ahead and specify so even after 1 we got
a 73% validation accuracy which is
pretty good let's go ahead and pass in
one more epochs equals 3 let's do it a
few more boxes this data
even better than the last one okay so
that's looking pretty good we were up to
74 78 it looks like we're gonna maybe
breach 80 we'll see no not yet anyway
but if we kept training this probably
for like 10 epochs or something like
that
in fact I'll plug in 10 for the rest of
my am I talking here chances are we'll
probably breach 80 which is pretty good
because that means we're classifying
cats and dogs pretty darn accurately so
in the next tutorial what we're going to
talk about is okay how do we read these
numbers like what is you know how do we
actually read laws how do we measure you
know how are you like read the accuracy
and especially like what about accuracy
versus validation accuracy like in this
case this is kind of wonky we got a
better validation accuracy over accuracy
it's kind of weird and we did it again
this is the first time we didn't so I'll
talk about like little stuff like that
that should be kind of red flags at
least initially it's not that big of a
deal but if your model consistently does
that that's a problem so we'll talk
about some of the patterns that you
should start seeing and that you should
watch out for as you begin to train
models so anyways that's all for now if
you've got questions comments concerns
whatever you can feel free to leave them
below if you like this content you can
support the content by clicking on that
beautiful join button right down there
if you join it helps support me in the
channel also you get various purse like
a little badge next to your name you
also get early access to videos just
like this one and also shout out from
time to time when people join so my most
recent member is Tejas thank you very
much for your support good sir you
helped support my mug addiction okay so
that's it for now in the next tutorial
we're going to be talking all about
tensor board which is absolutely
essential for training models and just
kind of analyzing them as they go it's
super super useful so anyways that's
what got me talking about the next
tutorial I will see you there