JOSH GORDON: Classifiers are only 选择好的特征能让分类器变得有效 JOSH GORDON: Classifiers are only as good as the features you provide.
as good as the features you provide. 这意味着找到好的特征 That means coming up with good features
That means coming up with good features 是机器学习中最重要的工作之一 is one of your most important jobs in machine learning.
is one of your most important jobs in machine learning. 但是怎么样才能获得好的特征 你怎么才能知道 But what makes a good feature, and how can you tell?
But what makes a good feature, and how can you tell? 如果你正处理二分类问题 If you're doing binary classification,
If you're doing binary classification, 那么好的特征能够使得辨别 then a good feature makes it easy to decide
then a good feature makes it easy to decide 两个不同的类变得容易 between two different things.
举个例子 假设我们想要编写一个分类器 For example, imagine we wanted to write a classifier
For example, imagine we wanted to write a classifier 用来区分两个种类的狗 to tell the difference between two types of dogs--
to tell the difference between two types of dogs-- 灵缇犬和拉布拉多犬 greyhounds and Labradors.
这里我们会使用两个特征 分别是狗的身高的英寸数 Here we'll use two features-- the dog's height in inches
Here we'll use two features-- the dog's height in inches 和狗的眼睛的颜色 and their eye color.
针对这个示例 让我们做一些关于狗的假设 Just for this toy example, let's make a couple assumptions
Just for this toy example, let's make a couple assumptions 让问题变得简单些 about dogs to keep things simple.
about dogs to keep things simple. 首先 我们假设灵缇通常会比拉布拉多高 First, we'll say that greyhounds are usually taller than Labradors.
First, we'll say that greyhounds are usually 其次 假设狗的眼睛只有两种颜色 Next, we'll pretend that dogs have only two eye
taller than Labradors. 分别是蓝色和棕色 colors-- blue and brown.
Next, we'll pretend that dogs have only two eye 同时我们假设它们眼睛的颜色 And we'll say the color of their eyes
colors-- blue and brown. 与狗的品种无关 doesn't depend on the breed of dog.
And we'll say the color of their eyes 这代表其中一个特征是有用的 This means that one of these features is useful
doesn't depend on the breed of dog. 而另外一个特征没有用 and the other tells us nothing.
This means that one of these features is useful 想要知道为什么 我们利用简单的数据集进行可视化展示 To understand why, we'll visualize them using a toy
and the other tells us nothing. 我将创建一个这样的数据集 dataset I'll create.
To understand why, we'll visualize them using a toy 先从狗的身高开始 Let's begin with height.
dataset I'll create. 你认为这个特征可以起到什么作用 How useful do you think this feature is?
Let's begin with height. 通常 灵缇会比 Well, on average, greyhounds tend
How useful do you think this feature is? 拉布拉多高几英寸 但并非总是如此 to be a couple inches taller than Labradors, but not always.
Well, on average, greyhounds tend 毕竟还是会存在很多变异 There's a lot of variation in the world.
to be a couple inches taller than Labradors, but not always. 所以当我们提到特征 So when we think of a feature, we
There's a lot of variation in the world. 我们需要考虑如何在总体中找到不同的值 have to consider how it looks for different values in a population.
So when we think of a feature, we 让我们用 Python 编程示例 Let's head into Python for a programmatic example.
have to consider how it looks for different values 我创建了一个数量为 1000 的狗的样本 灵缇和拉布拉多各占一半 I'm creating a population of 1,000 dogs-- 50-50 greyhound Labrador.
in a population. 我还会给它们设置一个身高的值 I'll give each of them a height.
Let's head into Python for a programmatic example. 在这个例子里 假设灵缇 For this example, we'll say that greyhounds
I'm creating a population of 1,000 平均身高为 28 英寸 拉布拉多平均身高为 24 英寸 are on average 28 inches tall and Labradors are 24.
dogs-- 50-50 greyhound Labrador. 现在 所有的狗都存在一些差异 Now, all dogs are a bit different.
I'll give each of them a height. 假设身高是正态分布的 Let's say that height is normally distributed,
For this example, we'll say that greyhounds 我们使得所有的值增加或减少 4 英寸 so we'll make both of these plus or minus 4 inches.
are on average 28 inches tall and Labradors are 24. 这样我们可以得到两个数组的数据 This will give us two arrays of numbers,
Now, all dogs are a bit different. 我们用直方图将它们进行可视化 and we can visualize them in a histogram.
Let's say that height is normally distributed, 我会增加一个参数 使得红色代表灵缇 I'll add a parameter so greyhounds are in red
so we'll make both of these plus or minus 4 inches. 蓝色代表拉布拉多 and Labradors are in blue.
This will give us two arrays of numbers, 让我们运行一下脚本 Now we can run our script.
and we can visualize them in a histogram. 这展示了在总体中特定身高的狗的数量 This shows how many dogs in our population have a given height.
I'll add a parameter so greyhounds are in red 屏幕上有许多的数据 There's a lot of data on the screen,
and Labradors are in blue. 让我们将其简化 逐步分解看 so let's simplify it and look at it piece by piece.
Now we can run our script. 我们从分布在最左边的开始看 We'll start with dogs on the far left
This shows how many dogs in our population have a given height. 高 20 英寸的会是哪种狗 of the distribution-- say, who are about 20 inches tall.
There's a lot of data on the screen, 想象下让你通过狗的身高来预测这只狗 Imagine I asked you to predict whether a dog with his height
so let's simplify it and look at it piece by piece. 是拉布拉多还是灵缇 was a lab or a greyhound.
We'll start with dogs on the far left 你会怎么做 What would you do?
of the distribution-- say, who are about 20 inches tall. 你可以得到每个种类的概率 Well, you could figure out the probability of each type
Imagine I asked you to predict whether a dog with his height 在给定狗的身高的情况下 of dog given their height.
was a lab or a greyhound. 看来 这很可能是只拉布拉多 Here, it's more likely the dog is a lab.
What would you do? 另外一方面 我们继续看下去 On the other hand, if we go all the way
Well, you could figure out the probability of each type 在直方图的右边会看到 to the right of the histogram and look
of dog given their height. 身高为 35 英寸的狗 at a dog who is 35 inches tall, we
Here, it's more likely the dog is a lab. 我们相信 35 英寸身高的狗必定是灵缇 can be pretty confident they're a greyhound.
On the other hand, if we go all the way 那身高在中间的会是什么种类的狗呢 Now, what about a dog in the middle?
to the right of the histogram and look 你会发现图中提供的信息略少 You can see the graph gives us less information
at a dog who is 35 inches tall, we 因为每种类型的狗的概率十分相近 here, because the probability of each type of dog is close.
can be pretty confident they're a greyhound. 身高是一个有用的特征 但它不够完美 So height is a useful feature, but it's not perfect.
Now, what about a dog in the middle? 所以在机器学习中 你总要 That's why in machine learning, you almost always
You can see the graph gives us less information 需要多个特征 need multiple features.
here, because the probability of each type of dog is close. 否则 你只能写 if 语句 Otherwise, you could just write an if statement
So height is a useful feature, but it's not perfect. 根本算不上是分类器 instead of bothering with the classifier.
That's why in machine learning, you almost always 为了弄明白需要使用哪种特征 To figure out what types of features you should use,
need multiple features. 让我们做个思考实验 do a thought experiment.
Otherwise, you could just write an if statement 假装你是分类器 Pretend you're the classifier.
instead of bothering with the classifier. 如果你试着辨识这只狗是 If you were trying to figure out if this dog is
To figure out what types of features you should use, 拉布拉多还是灵缇 你需要知道哪些其他的信息 a lab or a greyhound, what other things would you want to know?
do a thought experiment. 你也许会问它们头发的长度 You might ask about their hair length,
Pretend you're the classifier. 或者跑多快 或者它们的体重 or how fast they can run, or how much they weigh.
If you were trying to figure out if this dog is 你到底需要使用多少特征 Exactly how many features you should use
a lab or a greyhound, what other things would you want to know? 更多是艺术而非科学 is more of an art than a science,
You might ask about their hair length, 但是依据经验 请考虑下 but as a rule of thumb, think about how many you'd
or how fast they can run, or how much they weigh. 你需要解决多少问题 need to solve the problem.
Exactly how many features you should use 现在让我们看一下另外一个特征 狗的眼睛颜色 Now let's look at another feature like eye color.
is more of an art than a science, 这个示例中 我们假设 Just for this toy example, let's imagine
but as a rule of thumb, think about how many you'd 狗的眼睛只有两种颜色 蓝色和棕色 dogs have only two eye colors, blue and brown.
need to solve the problem. 假设它们眼睛的颜色 And let's say the color of their eyes
Now let's look at another feature like eye color. 与狗的品种无关 doesn't depend on the breed of dog.
Just for this toy example, let's imagine 这时直方图会是这个样子 Here's what a histogram might look like for this example.
dogs have only two eye colors, blue and brown. 对于大部分值 呈 50/50 分布 For most values, the distribution is about 50/50.
And let's say the color of their eyes 可见这个特征起不到什么作用 So this feature tells us nothing,
doesn't depend on the breed of dog. 因为它与狗的种类不相关 because it doesn't correlate with the type of dog.
Here's what a histogram might look like for this example. 在你的训练数据里包含这样一个无用的特征 Including a useless feature like this in your training
For most values, the distribution is about 50/50. 会影响到分类器的准确性 data can hurt your classifier's accuracy.
So this feature tells us nothing, 有时候凑巧这样的特征是有用的 That's because there's a chance they might appear useful purely
because it doesn't correlate with the type of dog. 特别是当你只有少量的 by accident, especially if you have only a small amount
Including a useless feature like this in your training 训练数据 of training data.
data can hurt your classifier's accuracy. 你的特征也需要是独立的 You also want your features to be independent.
That's because there's a chance they might appear useful purely 独立特征提供不同类型的信息 And independent features give you different types of information.
by accident, especially if you have only a small amount 假设我们已经有一个特征 身高的英寸数 Imagine we already have a feature-- height and inches--
of training data. 该特征来自我们的数据集 in our dataset.
You also want your features to be independent. 请问 它是否有用 Ask yourself, would it be helpful
And independent features give you 如果我们增加另外一个特征 例如身高的厘米数 if we added another feature, like height in centimeters?
different types of information. 回答是否定的 因为身高的英寸数和厘米数是完全相关的特征 No, because it's perfectly correlated with one we already have.
Imagine we already have a feature-- height and inches-- 好的方法是除去高度相关的特征 It's good practice to remove highly correlated features
in our dataset. 将它们从你的训练数据集中除去 from your training data.
Ask yourself, would it be helpful 因为许多的分类器 That's because a lot of classifiers
if we added another feature, like height in centimeters? 不够聪明 没有办法知道身高的英寸数和 aren't smart enough to realize that height in inches
No, because it's perfectly correlated with one 厘米数是一回事情 in centimeters are the same thing,
we already have. 所以它们会重复计算该特征的重要性 so they might double count how important this feature is.
It's good practice to remove highly correlated features 最后 若你希望你的特征易于理解 Last, you want your features to be easy to understand.
from your training data. 举个新的例子 假设你希望 For a new example, imagine you want
That's because a lot of classifiers 预测两个城市之间 to predict how many days it will take
aren't smart enough to realize that height in inches 邮寄一份信需要多少天 to mail a letter between two different cities.
in centimeters are the same thing, 城市之间的距离越远 需要的时间越久 The farther apart the cities are, the longer it will take.
so they might double count how important this feature is. 距离将是一个好的特征 A great feature to use would be the distance
Last, you want your features to be easy to understand. 即两个城市间的距离的公里数 between the cities in miles.
For a new example, imagine you want 更糟糕的一对特征是 A much worse pair of features to use
to predict how many days it will take 利用经度和纬度坐标来代表城市的位置 would be the city's locations given by their latitude and longitude.
to mail a letter between two different cities. 这是为什么呢 And here's why.
The farther apart the cities are, the longer it will take. 从距离来看 I can look at the distance and make
A great feature to use would be the distance 我能很容易想到这代表了邮寄一份信需要花费的时间 a good guess of how long it will take the letter to arrive.
between the cities in miles. 但是学习经度 维度 和时间之间的关系 But learning the relationship between latitude, longitude,
A much worse pair of features to use 是十分困难 需要提供更多的例子 and time is much harder and would require many more
would be the city's locations given by their latitude 从训练数据中获取例子 examples in your training data.
and longitude. 你可以使用很多技术 Now, there are techniques you can
And here's why. 来搞清楚哪些是有用的特征 use to figure out exactly how useful your features are,
I can look at the distance and make 甚至找到最佳的特征组合 and even what combinations of them are best,
a good guess of how long it will take the letter to arrive. 你无需听天由命 so you never have to leave it to chance.
But learning the relationship between latitude, longitude, 接下来的一集我们会提到这个 We'll get to those in a future episode.
and time is much harder and would require many more 紧接着 我们将继续培养 Coming up next time, we'll continue building our intuition
examples in your training data. 对监督学习的直觉 for supervised learning.
Now, there are techniques you can 我们将展示不同的分类器 We'll show how different types of classifiers
use to figure out exactly how useful your features are, 如何用不同的分类器解决同样的问题 深入了解 can be used to solve the same problem and dive a little bit
and even what combinations of them are best, 分类器是如何工作的 deeper into how they work.
so you never have to leave it to chance. 感谢观赏 下次再见 Thanks very much for watching, and I'll see you then.
We'll get to those in a future episode.