感谢Curiosity Stream支持PBS Digital Studios。 Thanks to Curiosity Stream for supporting PBS Digital Studios.
嗨，我是Jabril，欢迎您参加速成班AI！ Hey, I’m Jabril and welcome to Crash Course AI!
语言是人类所做的最令人印象深刻的事情之一。 Language is one of the most impressive things humans do.
这就是我现在如何将知识从我的大脑转移到您的大脑！ It’s how I’m transferring knowledge from my brain to yours right this second!
语言的形状和大小多种多样，可以说或写成各种语言 Languages come in many shapes and sizes, they can be spoken or written, and are made up
不同文化背景中句子，单词和字符等不同组成部分的区别。 of different components like sentences, words, and characters that vary across cultures.
例如，英语有26个字母，而中文有成千上万个字符。 For instance, English has 26 letters and Chinese has tens-of-thousands of characters.
到目前为止，我们一直在使用AI和机器学习技术解决的许多问题 So far, a lot of the problems we’ve been solving with AI and machine learning technologies
涉及处理图像，但是我们大多数人与计算机交互的最常见方式 have involved processing images, but the most common way that most of us interact with computers
是通过语言。 is through language.
我们在搜索引擎中输入问题，与智能手机通话以设置闹钟， We type questions into search engines, we talk to our smartphones to set alarms, and
有时，我们甚至会从Google Translate的西班牙语作业中获得一些帮助。 sometimes we even get a little help with our Spanish homework from Google Translate.
因此，今天，我们将探索自然语言处理领域。 So today, we’re going to explore the field of Natural Language Processing.
介绍 INTRO
自然语言处理（NLP）主要探讨了两个大概念。 Natural Language Processing, or NLP, mainly explores two big ideas.
首先，有自然语言理解，或者我们如何从组合中获得意义 First, there’s Natural Language Understanding, or how we get meaning out of combinations
字母。 of letters.
这些是可以过滤垃圾邮件的AI，可以判断亚马逊是否搜索“苹果” These are AI that filter your spam emails, figure out if that Amazon search for “apple”
是在杂货店或电脑购物，还是指导您的自动驾驶汽车如何前往朋友家 was grocery or computer shopping, or instruct your self-driving car how to get to a friend’s
屋。 house.
其次，是自然语言生成，或如何从知识中生成语言。 And second, there’s Natural Language Generation, or how to generate language from knowledge.
这些是执行翻译，摘要文档或与您聊天的AI。 These are AI that perform translations, summarize documents, or chat with you.
这两个问题的关键是理解单词的含义，这很棘手，因为 The key to both problems is understanding the meaning of a word, which is tricky because
单词本身没有意义。 words have no meaning on their own.
我们为符号赋予含义。 We assign meaning to symbols.
为了使事情变得更加困难，在许多情况下，语言可能会模棱两可，其含义 To make things even harder, in many cases, language can be ambiguous and the meaning
一个单词的含义取决于它所使用的上下文 of a word depends on the context it’s used in
如果我告诉您在河岸见我，没有任何背景，我可能是指河岸或 If I tell you to meet me at the bank, without any context, I could mean the river bank or
我在抢钱的地方。 the place where I’m grabbing some cash.
如果我说“这台冰箱很棒！”与“这台冰箱完全不同”的意思 If I say “This fridge is great!”, that’s a totally different meaning from “This fridge
很棒，持续了整整一个星期才打破。” was *great*, it lasted a whole week before breaking.”
那么，我们如何学习将意义赋予声音呢？ So, how did we learn to attach meaning to sounds?
我们怎么知道伟大（热情）意味着与伟大（讽刺）不同？ How do we know great [enthusiastic] means something different from great [sarcastic]?
好吧，即使“猫”这个词没有内在的含义可以告诉我们 Well, even though there’s nothing inherent in the word “cat” that tells us it’s
柔软，pur叫和追逐老鼠……当我们小时候，有人可能告诉我们“这是 soft, purrs, and chases mice… when we were kids, someone probably told us “this is
一只猫。” a cat.”
或gato，māo，bilee，qut。 Or a gato, māo, billee, qut.
当我们解决自然语言处理问题时，是否是自然语言理解 When we’re solving a natural language processing problem, whether it’s natural language understanding
或自然语言生成，我们必须考虑我们的AI如何学习 or natural language generation, we have to think about how our AI is going to learn the
理解单词的含义，了解我们潜在的错误。 meaning of words and understand our potential mistakes.
有时我们可以通过查看它们共享的字母来比较单词。 Sometimes we can compare words by looking at the letters they share.
如果单词具有词素，则效果很好。 This works well if a word has morphology.
以“游泳”这个词根为例。我们可以使用规则对其进行修改，因此，如果有人 Take the root word “swim” for example.We can modify it with rules so if someone’s
现在就去做，他们在游泳，或者是做游泳的人。 doing it right now, they’re swimming, or the person doing the action is the swimmer.
饮酒，饮酒者，思考者，思想家…………你明白了。 Drinking, drinker, thinking, thinker, … you get the idea.
但是我们不能对所有单词都使用词法，就像知道面包车是交通工具一样 But we can’t use morphology for all words, like how knowing that a van is a vehicle doesn’t
让我们知道一个破坏者在车窗里被砸了。 let us know that a vandal smashed in a car window.
许多真正相似的词，例如猫和汽车，是完全无关的。 Many words that are really similar, like cat and car, are completely unrelated.
另一方面，“猫”和“猫科”（“猫科”这个词的意思） And on the other hand, cat and Felidae (the word for the scientific family of cats) mean very
类似的事情，只分享一封信！ similar things and only share one letter!
猜测单词具有相似含义的一种常见方法是使用分布语义， One common way to guess that words have similar meaning is using distributional semantics,
或查看哪些单词经常出现在同一句子中。 or seeing which words appear in the same sentences a lot.
这是NLP依赖语言学领域见解的众多案例之一。 This is one of many cases where NLP relies on insights from the field of linguistics.
正如语言学家约翰·费思（John Firth）曾经说过的那样：“您将知道它所拥有的公司所讲的一句话。” As the linguist John Firth once said, “You shall know a word by the company it keeps.”
但是要使计算机理解分布语义，我们必须表达概念 But to make computers understand distributional semantics, we have to express the concept
在数学上。 in math.
一种简单的技术是使用计数向量。 One simple technique is to use count vectors.
计数向量是一个单词出现在同一文章或句子中的次数 A count vector is the number of times a word appears in the same article or sentence as
其他常用词。 other common words.
如果两个单词出现在同一句子中，则它们的含义可能非常相似。 If two words show up in the same sentence, they probably have pretty similar meanings.
假设我们要求使用以下算法比较汽车，猫和猫科动物这三个词， So let’s say we asked an algorithm to compare three words, car, cat, and Felidae, using
计算向量以猜测哪些具有相似的含义。 count vectors to guess which ones have similar meaning.
我们可以为每个单词下载Wikipedia页面的开头，以查看哪个/ other /单词 We could download the beginning of the Wikipedia pages for each word to see which /other/ words
出现。 show up.
这是我们得到的： Here’s what we got:
而且，许多重要词都相同：in和of。 And a lot of the top words are all the same: the, and, of, in.
这些都是功能词或停用词，它们有助于定义语言的结构， These are all function words or stop words, which help define the structure of language,
并帮助传达准确的含义。 and help convey precise meaning.
就像“苹果”是指任何苹果一样，但是“苹果”特别指定了一个。 Like how “an apple” means any apple, but “the apple” specifies one in particular.
但是，因为它们改变了另一个单词的含义，所以它们本身没有太多的含义， But, because they change the meaning of another word, they don’t have much meaning by themselves,
因此我们现在将其删除，并简化复数形式和共轭形式。 so we’ll remove them for now, and simplify plurals and conjugations.
让我们再试一次： Let’s try it again:
基于此，看起来猫和猫科动物几乎是同一回事，因为它们 Based on this, it looks like cat and Felidae mean almost the same thing, because they both
在他们的Wikipedia文章中出现很多相同的单词！ show up with lots of the same words in their Wikipedia articles!
而且它们都不像汽车一样。 And neither of them mean the same thing as car.
但这也是一个非常简化的示例。 But this is also a really simplified example.
计数向量的问题之一是我们必须存储大量数据。 One of the problems with count vectors is that we have to store a LOT of data.
要使用这样的计数比较一堆单词，我们需要每个单词的大量列表 To compare a bunch of words using counts like this, we’d need a massive list of every
我们在同一个句子中见过的单词，这是很难处理的。 word we’ve ever seen in the same sentence, and that’s unmanageable.
因此，我们想学习捕获所有相同关系的单词表示形式 So, we’d like to learn a representation for words that captures all the same relationships
和作为计数向量的相似性，但紧凑得多。 and similarities as count vectors but is much more compact.
在无监督学习情节中，我们讨论了如何通过构建表示来比较图像 In the unsupervised learning episode, we talked about how to compare images by building representations
这些图像。 of those images.
我们需要一个可以建立内部表示并可以生成预测的模型。 We needed a model that could build internal representations and that could generate predictions.
而且我们可以对单词做同样的事情。 And we can do the same thing for words.
这称为编码器-解码器模型：编码器告诉我们应该怎么想， This is called an encoder-decoder model: the encoder tells us what we should think and
记得我们刚刚读的书... remember about what we just read...
然后解码器使用这种思想来决定我们要说或做什么。 and the decoder uses that thought to decide what we want to say or do.
我们将从该框架的简单版本开始。 We’re going to start with a simple version of this framework.
让我们创建一个填补空白的小游戏，看看我们需要哪些基本知识 Let’s create a little game of fill in the blank to see what basic pieces we need to
训练无监督的学习模型。 train an unsupervised learning model.
这是一个简单的任务，称为语言建模。 This is a simple task called language modeling.
如果我有这句话： If I have the sentence:
我有点饿，我想我要点_____巧克力。 I’m kinda hungry, I think I’d like some chocolate _____ .
在该位置最可能出现的单词是什么？ What are the most likely words that can go in that spot?
以及我们如何训练模型来对句子进行编码并为空白进行猜测？ And how might we train a model to encode the sentence and decode a guess for the blank?
在这个例子中，我猜答案可能是“蛋糕”或“牛奶”，但可能不是 In this example, I can guess the answer might be “cake” or “milk” but probably not
像“土豆”之类的东西，因为我从未听说过“巧克力土豆” something like “potatoes,” because I’ve never heard of “chocolate potatoes” so
他们可能不存在。 they probably don’t exist.
绝对不存在。 Definitely don’t exist.
那不应该是一回事。 That should not be a thing.
可以填补空白的单词组是AI可以使用的无监督群集。 The group of words that can fill in that blank is an unsupervised cluster that an AI could use.
因此，对于这句话，我们的编码器可能只需要关注“巧克力”一词， So for this sentence, our encoder might only need to focus on the word chocolate so the
解码器具有一组“巧克力食物词”，可以从中提取出来以填补空白。 decoder has a cluster of “chocolate food words” to pull from to fill in the blank.
现在让我们尝试一个更困难的例子： Now let’s try a harder example:
戴安娜（Dianna）是圣地亚哥的我的一个朋友，他非常热爱物理，他正在过生日 Dianna, a friend of mine from San Diego who really loves physics, is having a birthday
下周聚会，所以我想找____的礼物。 party next week, so I want to find a present for ____.
当我读这句话时，我的大脑识别并记住了两件事：首先，我们 When I read this sentence, my brain identifies and remembers two things: First, that we’re
从27个单词开始谈论戴安娜！ talking about Dianna from 27 words ago!
其次，我的朋友戴安娜（Dianna）使用代词“她”。 And second, that my friend Dianna uses the pronoun “her.”
这意味着我们希望编码器构建一个捕获所有这些片段的表示形式 That means we want our encoder to build a representation that captures all these pieces
句子中的信息，因此解码器可以为空白选择正确的单词。 of information from the sentence, so the decoder can choose the right word for the blank.
如果我们继续这句话： And if we keep the sentence going:
戴安娜（Dianna）是圣地亚哥的我的一个朋友，他非常热爱物理，他正在过生日 Dianna, a friend of mine from San Diego who really loves physics, is having a birthday
下周聚会，所以我想为她找到一份与_____有关的礼物。 party next week, so I want to find a present for her that has to do with _____ .
现在，我可以记得戴安娜（Dianna）从句子的开头就喜欢物理学。 Now, I can remember that Dianna likes physics from earlier in the sentence.
因此，我们希望编码器也记住这一点，以便解码器可以使用该信息 So we’d like our encoder to remember that too, so that the decoder can use that information
猜答案。 to guess the answer.
因此，我们可以看到模型构建的表示形式实际上必须记住关键细节 So we can see how the representation the model builds really has to remember key details
我们所说或听到的。 of what we’ve said or heard.
模型可以记住的数量是有限的。 And there’s a limit to how much a model can remember.
雷·穆尼（Ray Mooney）教授曾有句著名的话说：“我们将永远 Professor Ray Mooney has famously said that we’ll “never fit the whole meaning of
一句话变成一个向量”，我们仍然不知道是否可以。 a sentence into a single vector” and we still don’t know if we can.
Mooney教授可能是正确的，但这并不意味着我们不能做有用的事情。 Professor Mooney may be right, but that doesn’t mean we can’t make something useful.
到目前为止，我们一直在使用单词。 So so far we’ve been using words.
但是计算机不能像这样工作。 But computers don’t work words quite like this.
因此，让我们脱离对语言建模的高级了解，尝试预测 So let’s step away from our high level view of language modeling and try to predict the
无论如何，使用神经网络在句子中的下一个单词。 next word in a sentence anyway with a neural network.
为此，我们的数据将包含很多我们从诸如某人讲话之类的句子中收集的句子 To do this, our data will be lots of sentences we collect from things like someone speaking
或书籍中的文字。 or text from books.
然后，对于每个句子中的每个单词，我们将玩一个空白游戏。 Then, for each word in every sentence, we’ll play a game of fill-in-the-blank.
我们将训练一个模型来编码最多该空白，然后预测应该 We’ll train a model to encode up to that blank and then predict the word that should
去那里。 go there.
由于我们拥有完整的句子，因此我们知道正确的答案。 And since we have the whole sentence, we know the correct answer.
首先，我们需要定义编码器。 First, we need to define the encoder.
我们需要一个可以在输入中读取的模型，在这种情况下，该模型是一个句子。 We need a model that can read in the input, which in this case is a sentence.
为此，我们将使用一种称为递归神经网络或RNN的神经网络。 To do this, we’ll use a type of neural network called a Recurrent Neural Network or RNN.
RNN中有一个循环，使它们可以重用单个隐藏层，该层将被更新 RNNs have a loop in them that lets them reuse a single hidden layer, which gets updated
因为模型一次读取一个字。 as the model reads one word at a time.
慢慢地，模型建立了对整个句子的理解，包括哪些单词 Slowly, the model builds up an understanding of the whole sentence, including which words
首先出现还是最后出现，哪些单词修饰了其他单词，还有一大堆其他语法 came first or last, which words are modifying other words, and a whole bunch of other grammatical
与意义相关的属性。 properties that are linked to meaning.
现在，我们不能只将单词直接放入网络中。 Now, we can’t just directly put words inside a network.
但是我们也没有可以轻松测量和提供模型的功能。 But we also don’t have features we can easily measure and give the model either.
与图像不同，我们甚至无法测量像素值。 Unlike images, we can’t even measure pixel values.
因此，我们将要求模型为模型上的单词学习正确的表示形式 So we’re going to ask the model to learn the right representation for a word on its
自己的（这是无监督学习的来源）。 own (this is where the unsupervised learning comes in).
为此，我们首先为每个单词分配一个随机表示- To do this, we’ll start off by assigning each word a random representation -- in this
情况是一个随机数字列表，称为向量。 case a random list of numbers called a vector.
接下来，我们的编码器将采用这些表示中的每个表示并将它们组合为一个 Next, our encoder will take in each of those representations and combine them into a single
/ shared /代表整个句子。 /shared/ representation for the whole sentence.
在这一点上，我们的表示可能有点乱，但是为了训练RNN， At this point, our representation might be gibberish, but in order to train the RNN,
我们需要它来做出预测。 we need it to make predictions.
对于这个特定问题，我们将考虑一个非常简单的解码器，即单层网络 For this particular problem, we’ll consider a very simple decoder, a single layer network
它接受句子表示向量，然后为每个 that takes in the sentence representation vector, and then outputs a score for every
我们词汇表中可能的单词。 possible word in our vocabulary.
然后，我们可以将得分最高的词解释为模型的预测。 We can then interpret the highest scored word as our model’s prediction.
然后，我们可以使用反向传播训练RNN，就像我们之前使用神经 Then, we can use backpropagation to train the RNN, like we’ve done before with neural
速成课程AI中的广告网络。 networks in Crash Course AI.
因此，通过训练可预测下一个单词的模型，该模型可为 So by training the model on which word to predict next, the model learn weights for
编码器RNN和解码器预测层。 the encoder RNN and the decoder prediction layer.
此外，模型会更改我们在一开始就给每个单词的那些随机表示。 Plus, the model changes those random representations we gave every word at the beginning.
具体来说，如果两个词的含义相似，则模型会使它们的向量更多 Specifically, if two words mean something similar, the model makes their vectors more
类似。 similar.
使用向量来帮助绘制情节，我们实际上可以可视化单词表示。 Using the vectors to help make a plot, we can actually visualize word representations.
例如，前面我们讨论了巧克力和物理学，所以让我们来看一下 For example, earlier we talked about chocolate and physics, so let’s look at some word
Google的研究人员培训过的陈述。 representations that researchers at Google trained.
在“巧克力”附近，我们有很多食物，例如可可和糖果： Near “chocolate,” we have lots of foods like cocoa and candy:
相比之下，与“物理学”具有相似表示的单词是牛顿和宇宙。 By comparison, words with similar representations to “physics” are newton and universe.
整个过程都使用了无监督的学习方法，这为我们提供了学习一些基本方法的基本方法 This whole process has used unsupervised learning, and it’s given us a basic way to learn some
非常有趣的语言表示形式和单词簇。 pretty interesting linguistic representations and word clusters.
但是部分句子并预测下一个单词只是冰山一角 But taking in part of a sentence and predicting the next word is just the tip of the iceberg
对于NLP。 for NLP.
如果我们的模型使用英语并制作西班牙语，那么我们将拥有翻译系统。 If our model took in English and produced Spanish, we’d have a translation system.
或者我们的模型可以读取问题并产生答案，例如Siri或Alexa尝试这样做。 Or our model could read questions and produce answers, like Siri or Alexa try to do.
或者我们的模型可以将指令转换为控制家用机器人的动作…嘿 Or our model could convert instructions into actions to control a household robot … Hey
约翰·格林·波特（John Green Bot）？ John Green Bot?
只是在开玩笑，您就是自己的机器人。 Just kidding you’re your own robot.
没有人控制你。 Nobody controls you.
但是我们的模型为一种任务学习的单词表示可能不会 But the representations of words that our model learns for one kind of task might not
为别人工作。 work for others.
例如，如果我们基于阅读一堆烹饪食谱来训练John-Green-bot， Like, for example, if we trained John-Green-bot based on reading a bunch of cooking recipes,
他可能会知道玫瑰是用糖霜制成的，并放在蛋糕上。 he might learn that roses are made of icing and placed on cakes.
但是他不会知道蛋糕玫瑰与有刺和刺的真正玫瑰是不同的 But he won’t learn that cake roses are different from real roses that have thorns and make
漂亮的花束 a pretty bouquet.
获取，编码以及使用书面或口语知识来帮助人们，这是一个巨大的过程 Acquiring, encoding, and using written or spoken knowledge to help people is a huge
和令人兴奋的任务，因为我们在很多事情上都使用语言！ and exciting task, because we use language for so many things!
每次您键入或与计算机，电话或其他小工具交谈时，都会有NLP。 Every time you type or talk to a computer, phone or other gadget, NLP is there.
现在我们已经了解了基础知识，下周我们将深入研究并建立语言模型 Now that we understand the basics, next week we’ll dive in and build a language model
一起在我们的第二个实验室！回头见。 together in our second lab! See you then.
感谢CuriosityStream支持PBS Digital Studios。 Thank you to CuriosityStream for supporting PBS Digital Studios.
CuriosityStream是一种订阅流媒体服务，可提供纪录片和非小说类作品 CuriosityStream is a subscription streaming service that offers documentaries and non¬fiction
来自各种电影制片人的作品，包括CuriosityStream的原创作品。 titles from a variety of filmmakers, including CuriosityStream originals.
例如，您可以串流主持人西格妮·韦弗（Sigourney Weaver）提出问题的“梦想未来”， For example, you can stream Dream the Future in which host Sigourney Weaver asks the question,
“未来会是什么样？”当她检查新发现和研究如何 “What will the future look like?” as she examines how new discoveries and research
在2050年将影响我们的日常生活。 will impact our everyday lives in the year 2050.
您可以在curiositystream.com/crashcourse中了解更多信息 You can learn more at curiositystream.com/crashcourse
或单击描述中的链接。 Or click the link in the description.
速成班Ai是与PBS Digital Studios联合制作的！ Crash Course Ai is produced in association with PBS Digital Studios!
如果您想让所有人永久免费参加速成课程，可以加入我们的社区 If you want to help keep Crash Course free for everyone, forever, you can join our community
在Patreon上。 on Patreon.
如果您想了解有关人脑如何处理语言的更多信息，请查看此剧集 And if you want to learn more about how human brains process language, check out this episode
速成课程心理学。 of Crash Course Psychology.