[MUSIC PLAYING]
Last episode, we used a decision tree as our classifier.
Today we'll add code to visualize it
so we can see how it works under the hood.
There are many types of classifiers
you may have heard of before-- things like neural nets
or support vector machines.
So why did we use a decision tree to start?
Well, they have a very unique property--
they're easy to read and understand.
In fact, they're one of the few models that are interpretable,
where you can understand exactly why the classifier makes
a decision.
That's amazingly useful in practice.
To get started, I'll introduce you
to a real data set we'll work with today.
It's called Iris.
Iris is a classic machine learning problem.
In it, you want to identify what type of flower
you have based on different measurements,
like the length and width of the petal.
The data set includes three different types of flowers.
They're all species of iris-- setosa, versicolor,
and virginica.
Scrolling down, you can see we're
given 50 examples of each type, so 150 examples total.
Notice there are four features that are
used to describe each example.
These are the length and width of the sepal and petal.
And just like in our apples and oranges problem,
the first four columns give the features and the last column
gives the labels, which is the type of flower in each row.
Our goal is to use this data set to train a classifier.
Then we can use that classifier to predict what species
of flower we have if we're given a new flower that we've never
seen before.
Knowing how to work with an existing data set
is a good skill, so let's import Iris into scikit-learn
and see what it looks like in code.
Conveniently, the friendly folks at scikit
provided a bunch of sample data sets,
including Iris, as well as utilities
to make them easy to import.
We can import Iris into our code like this.
The data set includes both the table
from Wikipedia as well as some metadata.
The metadata tells you the names of the features
and the names of different types of flowers.
The features and examples themselves
are contained in the data variable.
For example, if I print out the first entry,
you can see the measurements for this flower.
These index to the feature names, so the first value
refers to the sepal length, and the second to sepal width,
and so on.
The target variable contains the labels.
Likewise, these index to the target names.
Let's print out the first one.
A label of 0 means it's a setosa.
If you look at the table from Wikipedia,
you'll notice that we just printed out the first row.
Now both the data and target variables have 150 entries.
If you want, you can iterate over them
to print out the entire data set like this.
Now that we know how to work with the data set,
we're ready to train a classifier.
But before we do that, first we need to split up the data.
I'm going to remove several of the examples
and put them aside for later.
We'll call the examples I'm putting aside our testing data.
We'll keep these separate from our training data,
and later on we'll use our testing examples
to test how accurate the classifier is
on data it's never seen before.
Testing is actually a really important part
of doing machine learning well in practice,
and we'll cover it in more detail in a future episode.
Just for this exercise, I'll remove one example
of each type of flower.
And as it happens, the data set is
ordered so the first setosa is at index 0,
and the first versicolor is at 50, and so on.
The syntax looks a little bit complicated, but all I'm doing
is removing three entries from the data and target variables.
Then I'll create two new sets of variables-- one
for training and one for testing.
Training will have the majority of our data,
and testing will have just the examples I removed.
Now, just as before, we can create a decision tree
classifier and train it on our training data.
Before we visualize it, let's use the tree
to classify our testing data.
We know we have one flower of each type,
and we can print out the labels we expect.
Now let's see what the tree predicts.
We'll give it the features for our testing data,
and we'll get back labels.
You can see the predicted labels match our testing data.
That means it got them all right.
Now, keep in mind, this was a very simple test,
and we'll go into more detail down the road.
Now let's visualize the tree so we can
see how the classifier works.
To do that, I'm going to copy-paste
some code in from scikit's tutorials,
and because this code is for visualization
and not machine-learning concepts,
I won't cover the details here.
Note that I'm combining the code from these two examples
to create an easy-to-read PDF.
I can run our script and open up the PDF,
and we can see the tree.
To use it to classify data, you start by reading from the top.
Each node asks a yes or no question
about one of the features.
For example, this node asks if the pedal width
is less than 0.8 centimeters.
If it's true for the example you're classifying, go left.
Otherwise, go right.
Now let's use this tree to classify an example
from our testing data.
Here are the features and label for our first testing flower.
Remember, you can find the feature names
by looking at the metadata.
We know this flower is a setosa, so let's see
what the tree predicts.
I'll resize the windows to make this easier to see.
And the first question the tree asks
is whether the petal width is less than 0.8 centimeters.
That's the fourth feature.
The answer is true, so we proceed left.
At this point, we're already at a leaf node.
There are no other questions to ask,
so the tree gives us a prediction, setosa,
and it's right.
Notice the label is 0, which indexes to that type of flower.
Now let's try our second testing example.
This one is a versicolor.
Let's see what the tree predicts.
Again we read from the top, and this time the pedal width
is greater than 0.8 centimeters.
The answer to the tree's question is false,
so we go right.
The next question the tree asks is whether the pedal width
is less than 1.75.
It's trying to narrow it down.
That's true, so we go left.
Now it asks if the pedal length is less than 4.95.
That's true, so we go left again.
And finally, the tree asks if the pedal width
is less than 1.65.
That's true, so left it is.
And now we have our prediction-- it's a versicolor,
and that's right again.
You can try the last one on your own as an exercise.
And remember, the way we're using the tree
is the same way it works in code.
So that's how you quickly visualize and read
a decision tree.
There's a lot more to learn here,
especially how they're built automatically from examples.
We'll get to that in a future episode.
But for now, let's close with an essential point.
Every question the tree asks must be about one
of your features.
That means the better your features are, the better a tree
you can build.
And the next episode will start looking
at what makes a good feature.
Thanks very much for watching, and I'll see you next time.
[MUSIC PLAYING]