Hello, my dear friend.
So as you remember from our last lesson, I hope we talked about machine learning methods.
So in this lesson, we're going to talk about them again.
But this time we're going to get into some of the nitty gritty details.
Our first method is probabilistic modelling.
So what the heck is that?
Probabilistic model, it's the application of principles of statistics, data analysis.
It was one of the oldest forms of machine learning, and it's still widely used to this day.
One of the best known algorithms in this category is the naïf baize algorithm.
Naïf baize is a type of machine learning classifier based on applying Baz's theorem, and it will assume
that all properties in the input data are independent.
In other words, a strong or naive assumption.
And that's where the name actually comes from, right?
Now, this form of data analysis predates computers and was implemented by hand decades before the first
computer implementation most likely dates back to even the early 1950s.
But the foundations of Bayes Theorem and statistics date back to the 18th century.
And they are all that you need to know when you start using the naives classifiers.
A closely related model is logistic regression, or you could call it by its short name, like Greg.
But it's sometimes regarded as the hello world of modern machine learning and don't be fooled by its
name, lol, Greg is a classification algorithm rather than a regression algorithm.
But like naïf long drag was long before computing.
But it's useful today, thanks to its simple and versatile nature, so this is often the first thing
a data scientist will try on a data set to get an idea of the classification task at hand.
So let's have a look at some of the other methods, early neural networks.
So early iterations of neural networks have been completely supplanted by the modern variants covered
in these pages, but it's helpful to be aware of how deep learning originated, where it came from and
all that good stuff.
Right?
Well, maybe just for a moment.
And although the core ideas of neural networks were investigated and toy forms as early as 1950s, the
approach took decades to get started.
For a long time, the missing piece was an efficient way to train large neural networks.
And this changed in the mid 1980s when multiple people independently rediscovered the back propagation
algorithm.
Which is a way to train chains of parametric operations using gradient descent optimization.
And then they started applying it to neural networks, the rest, as they say, is history.
The first successful practical application of neural net came in 1989 from Bell Labs when John McCune.
Combined, the earlier ideas of convolution neural networks and back propagation and then applied them
to the problem of classifying handwritten digits, so the resulting network dubbed Lonette.
Was used by the United States Postal Service way back in the 1990s to automate the reading of zip codes
on mail envelopes.
All right, so that's enough with the fun facts about early neural networks, but now we need to talk
about Colonel Method's.
So is neural networks starting to gain some respect among researchers in the 1990s, thanks to this
first success, a new approach to machine learning rose to fame and quickly sent neural nets back into
oblivion.
Colonel Method's.
Colonel methods are a group of classification algorithms and the best known of which is probably the
support vector machine or SBM, the modern formulation of an SVM was developed by Vladimir Wapnick and
coronate Cortez in the early 1980s at Bell Labs and published in 1995, although an older linear formulation
was published by APNIC and Alexey Jevin Inca's as early as 1963.
So as we aim at solving classification problems by finding good decision boundaries between two sets
of points belonging to two different categories, so we can see that on this figure.
A decision boundary can be thought of as a line or service separating your training data into two spaces
corresponding to two categories.
To classify new data points, you just need to check which side of the decision boundary they fall on.
As VMS proceed to find these boundaries in two steps.
The data is mapped to a new high dimensional representation where the decision boundary can be expressed
as a hyper plane, if the data was two dimensional, as in a figure, a hyper plane would be a straight
line, you know.
And a good decision boundary.
A separation hyper plane.
Now, this is computed by trying to maximize the distance between a hyper plane and the closest data
points from each class.
This is a step called maximizing the margin.
So this allows the boundary to generalize to new samples outside of the training data set.
The technique of mapping data to a high dimensional representation where a classification problem becomes
simpler may look pretty good on paper, but in practice it's often computationally intractable.
So that's where the kernel trick comes in.
The key idea that colonel methods are named after now is the gist of it to find good decision hyper
planes in new representation space, you don't have to explicitly compute the coordinates of your points
in the new space.
You just need to compute the distance between pairs of points in that space, which can be done efficiently
using a kernel function.
So, Colonel, function is a computationally tractable operation that maps any two points in your initial
space to the distance between these points in your target representation space, completely bypassing
the explicit computation of the new representation.
So, Colonel, functions are typically crafted by hand rather than learned from Detta.
In the case of NCM, only, this separation hyper has learned.
So the time they were developed as firms exhibited state of the art performance on simple classification
problems and were one of the few machine learning methods backed by extensive theory and amenable to
serious mathematical analysis, making them well understood and easily interpretable.
Because of these useful properties, his films became extremely popular in the field for a long time.
But as Williams proved hard to scale to large data sets and didn't provide good results for perceptual
problems such as image classification, because in SVM is a shallow method.
Applying SVM to perceptual problems requires first extracting useful representations manually, and
that's a step called feature engineering, which is difficult and brittle.
Now, there is another method you may have heard of decision trees.
Decision trees are flowchart like structures that allow you to classify input data points or predict
output values according to those given inputs.
They're easy to visualize and interpret.
So decision trees learn from data began to gain significant research interest in the 2000s and by 2010,
they were generally preferred over core methods.
In particular, the random forest algorithm.
As broad and robust and practical approach to decision tree learning.
Which involves generating a large number of custom decision trees and then stitching their outputs together.
Random forests can be applied to a wide variety of problems, you could even say they're almost the
second best algorithm, Rashelle machine learning tasks.
When the popular Machine Learning Competition website Kagle launched in 2010, random forests quickly
became the platform's favorite until 2014, when gradient boosting machines took over.
Much like a random forest, a gradient boosting machine is a machine learning technique that relies
on putting together weak predictive models that are often decision trees.
So it uses gradient boosting, which is a way to improve any machine learning model by iteratively training
new models that specialize in addressing the weaknesses of the previous models.
So when applied to decision trees, the use of gradient boosting technique, well, results in models
with similar characteristics while often performing absolutely better than random forest.
So it may be one of the best, if not the best algorithms for dealing with non perceptual data today.
But besides deep learning, it's one of the most commonly used techniques in Kagle competitions.
All right, very good.
So now we can get back to neural networks again.
So around 2010, although neural networks were almost completely shunned by the scientific community,
a large number of people still working on neural networks started to make some important breakthroughs.
The groups of Geoffrey Hinton at the University of Toronto, Yahshua Bengoa, the University of Montreal,
John, looking at New York University and Izia in Switzerland in 2011, dancers sound from idiom began
to win the academic image classification competitions with GPU train deep neural networks, the first
practical success of Modern Deep Learning.
But the watershed moment came in 2012 with the entry of Hinton's group in the yearly large scale image
classification challenge image that.
The image that challenge was notoriously difficult at the time, consisting of classifying high resolution
color images into 1000 different categories after training on one point four million images.
In 2011, the top five accuracy of the winning model based on classical approaches to computer vision
was only seventy four point three percent.
Then in 2012, a team led by Alex Kraszewski and advised by Gregory Hinton was able to achieve a top
five accuracy of eighty three point six percent.
Now, that's a significant breakthrough.
The competition has been dominated by deep, convolutional neural networks every year since.
By 2015, the winner reached an accuracy of ninety six point four percent and the classification task
on image net was considered to be completely solved.
Problem.
Since 2012, deep convolutional neural networks or nets became the go to algorithm for all computer
vision tasks.
More generally, they work on all perceptual tasks at major computer vision conferences in 2015 and
2016.
It was nearly impossible to find presentations that didn't involve Conn's nets in some form or another.
At the same time, deep learning has also found applications in many other types of problems, such
as natural language processing.
It has completely replaced as VMS and decision trees and a wide range of applications as well.
For instance, for several years the European Organization for Nuclear Research, or CERN, used decision
tree based methods for analysis of particle data from the Atlas detector at the Large Hadron Collider
LHC.
I'm sure you've heard of it, but CERN eventually switched to Charis based deep neural networks due
to their higher performance and ease of training on large data sets.
All right, my friend, so that's it in this lesson, we talked about different learning methods.
So the next lesson, we're going to examine them well, very deeply so I want to see in the next lesson
till then, have a really nice day.