All right.
Welcome to a brand new module.
In this lesson we're going to start talking about neural networks and why they are so incredibly exciting.
Truth be told I've been looking forward to talking about neural networks from host of the course and
one of the reasons is is that you and I have some things in common with this machine learning framework.
You see artificial neural networks were inspired by our desire to understand the human brain thinking
was that maybe just maybe we would be able to understand ourselves better if we were able to build a
model of how the human brain worked.
And at the same time we were asking ourselves a very different question right.
If computers were getting more and more powerful why are they so bad at certain simple tasks like telling
the difference between a dog and a cat looking at things like Moore's Law.
You'd think that we'd be there by now.
Right.
More transistors I have clock speeds more processing power and faster and faster calculations faster
than the human brain in many respects.
And yet raw computation only took us so far.
There's only so much that you can accomplish with the logic in an if statement as amazing as if a whole
statements are it still the programmer has to sit down and explicitly write all that code.
The computer itself couldn't learn anything but this very point brings us to another question.
How does learning happen anyhow.
How does our human brain actually work.
Well if we take the brain and we put it under a microscope and then we examine the cells that make up
our brain our neurons then we see that the human brain is comprised of approximately 10 billion neurons
and all of these neurons are connected to each other.
So each neuron is actually connected to about 10000 other neurons.
How are these neurons connected.
Well here's a schematic.
This picture here represents an individual neuron.
And on the left side and the right side you see these like funky purple lines coming out.
And there's two types of lines that come in and out of a neuron.
You see neurons actually receive signals from other neurons via their dendrites.
So on one side of the neuron you've got these inputs.
And once a neuron receives a signal it might pass that signal on right.
And the neurons pass these signals on.
On the other side of the cell namely through their axons This is where they transmit their output the
signals themselves being transmitted between these dendrites and these axons are electrical and chemical.
If a neuron receives many signals from the previous neurons then it will activate and transmit the signal.
In other words if the sum of the signals from all the upstream neurons is strong enough then this neuron
will pass the signal on through its axons to the other neurons downstream.
And then this process repeats itself thousands and thousands of times over.
The key though is that a neuron only fires if the total signal received at the cell body exceeds a certain
level or a certain threshold.
And also the neuron other fires or doesn't fire there aren't any different grades of firing.
It's more like a light switch the switches either on or off.
Now that doesn't sound familiar to an electrical circuit.
I don't know what does the crazy thing though is that our entire brain is composed of these interconnected
transmitters individually they're all very simple processing units.
Suppose all these dots here represent an individual neuron right.
So I've got three on the left one in the middle and four on the right.
And all of these neurons are connected.
So I'm going to draw these arrows between them now not all connections are created equal.
So I've drawn some of these arrows thicker than other ones.
But let's focus on this blue neuron in the middle.
When the pink neurons upstream fire they will pass a signal along to our blue neuron and this bad boy
will take the weighted sum of all of its inputs.
So the blue neuron will consider the signals that it's getting from the neurons that fired as well as
the signals from the neurons that didn't fire and this means that the blue neuron itself might or might
not fire whether or not it fires is the turn but the total input that it has received this total input
has to exceed a certain level or a certain threshold.
Now our blue neuron did indeed receive a very strong signal by two of its strongest connections.
So indeed it does fire.
It gets triggered and it will pass the signal on to all of its 10000 neurons that are connected to it
this was the line of thinking that led Warren McCulloch and Walter Pitts to create a computational model
for neural networks all the way back in 1943.
Individually each neuron is not complicated.
In other fires or doesn't fire based on a threshold but working together the brain can nonetheless perform
some very very complex tasks.
But the one thing we didn't talk about yet is how learning happens.
In 1949 a Canadian psychologist named Donald HEB published his theory.
He pointed out that the neural pathways are strengthened every time that they are used.
If two neurons fire at the same time the connection between them is enhanced.
And this concept is essential to how humans learn.
According to Donald HEB it's the connections between the neurons that matter so for the sake of argument
say you're trying to learn a foreign language So you're trying to learn.
I don't know Japanese.
Every time you see something in Japanese certain neurons in your brain start firing and every time you
practice speaking the connections between these neurons get stronger and that's why the more you speak
the easier it gets to express yourself and remember the words activating the neurons again and again
strengthens the connections between them and there's actually a word for the strength of a connection.
This strength is referred to as the weight so as you're practicing speaking in your foreign language
you're effectively training your neurons and changing the weights between them.
This concept was Donald heads this big insight.
The learning comes down to adjusting the weights between the neurons and this is also the model that
artificial neural networks are based on.
So given that scientists thought at working on this all the way back in the 1940s the question then
becomes did we succeed in modeling the human brain based on neural networks.
We've been at it for a long time after all right.
Well not quite.
Even though the biology provided the inspiration for artificial neural nets we still got a long long
way to go to figuring out how the brain actually works.
So far we haven't come close to modeling the sheer complexity of the human brain but neural networks
have proven themselves to be extremely good at tasks which are very difficult for traditional computers
say like image recognition.
So where does all of this leave us.
Well we've covered a couple of very important points already.
Neural networks are composed of individual nodes or neurons.
All of these notes are connected to each other and the strength of these connections are called the
weights and the process of learning both for humans and for machines involves adjusting the weights
between the notes.
We'll be talking a whole lot more about this in the next lesson.
I'll see you there.
Take care.