- [Instructor] Linear algebra
is one of the main building blocks of machine learning.
At its base, it is a study of vectors, matrices,
and linear functions.
So you have to be familiar with vectors,
matrices, and their characteristics,
as you're going to deal with them in ML.
What about machine learning algorithms?
Without knowing linear algebra, you cannot understand them
and therefore cannot change and improve them.
Let's explore nine most important applications
of linear algebra.
Data set and data files.
In machine learning we fit the model on a data set.
This data set is either a matrix or a vector.
For example, our model could be a fitness related model
that predicts quality of sleep.
Images and photographs.
If you're working on a computer vision application,
then you're dealing with images or photographs.
In this case, you cannot send an image to a model
and expect it to understand it.
Each image is made of pixels
that are basically just colored squares
of varying intensities.
If you have a black and white image
then it's a single pixel.
And for colored images, it's three pixel values for RGB.
All images are stored as a matrix.
Each operation that is performed on the image
from cropping, scaling, et cetera,
is described using the notation and operations
of linear algebra.
Data preparation.
Data preparation, such as dimensionality reduction
and one hot encoding requires linear algebra.
Usually we come across data
that is made up of thousands of variables
and our model becomes extremely complicated.
That's when dimensionality reduction comes into play.
Data sets are represented as matrices
and then we can use matrix factorization methods
to reduce it into its constituent parts.
One hot encoding is used when working with categorical data,
such as the class labels for classification problems
or categorical input variables.
It is common to encode categorical variables
to make them easier to work with.
Linear regression.
It is usually used in machine learning
for predicting numerical values
in simple regression problems.
The most common way of solving linear regression
is via the least squares optimization
that is solved using matrix factorization methods
from linear regression.
Regularization.
Overfitting is one of the greatest obstacles
in machine learning.
When a model is too close a fit for the available data
to the point that it does not perform well
with any new or outside data.
Regularization is a concept from linear algebra
that is used to prevent the model from overfitting.
Simple models are models
that have smaller coefficient values.
Regularization is a technique that is often used
to encourage a model to minimize the size of coefficients
while it's being fit on data.
Principle component analysis.
Modeling data with many features is challenging
and it's hard to know which features of data are relevant
and which are not.
One of the methods
for automatic reducing the number of columns of a data set
is principle component analysis.
This method is used in machine learning
to create projections of high dimensional data
for both visualization and for training models.
The core of the PCA method is a metric factorization method.
Latent semantic analysis.
LSA is a form of data preparation
used in natural language processing,
a subfield of machine learning for working with text data.
In this case, documents are usually represented
as a large matrices of word occurrences.
Then we can apply matrix factorization methods to them
in order to be able to easily compare, query,
and use them as the basis for the ML model.
Recommender systems.
Recommender systems are predictive modeling problems
that involve the recommendation of products.
They are used each time you buy something on Amazon
or a similar shop and you get recommendations of products
based on your previous purchases.
Deep learning.
Deep learning is a specific subfield of machine learning.
Scaled up to multiple dimensions,
deep learning methods work with vectors, matrices,
and tensors of inputs and coefficients.