Okay okay okay hey morning everyone welcome back
Sowal teacher for the bolts logistic regression and
Generalized linear models and all of the
Discriminative learning today would like to do is share
How generative learning algorithms work
By the end of the day you know how that turns out that compared to say logistic regression for a complication
GTA is actually a simpler and maybe more completely efficient algorithm to
In some cases so and it sometimes works better if you have a very small data sets
Comparison between genitive learning albums which is a New Concept albums
About today versus the strength of learning algorithms and then we'll talk about now you face and how you can use that
Build a spam filter
Pineapple
So we'll use finally classification as the motivating example but
Today and if you have a
Data set that looks like this
What's the cost is then what a discriminative learning album like logistic regression with you
Is you straight into send to search for a line that separates the positive negative example
Finished night randomly initialise Panthers Maybe
Stop system Vision foggy like that and over the closer Graton to send you know the line my grades are
What evolves after you get maybe a line like that that separates the positive and negative examples
The Impressions really searching for a line searching for the gym boundary that separates the positive and negative examples
And so if this was the malignant tumors
And the benign tumors example
Right that's that's what this expression with you now does it different
Which isn't searching for this separation maximize the likelihood that the way you saw
Last week which is pissing alternative which is
Trying to find a separation is that the album is going to look at the closest one at a time
Malignant tumors right in the cancer example and try to build a model for what
Malignant tumor malignant tumor is roughly
All the malignant tumors Rafi living got the lips and then
Can you look nice in isolation and say oh look like
And then a complication time it doesn't new you no patience in your office
What does speeches it would then look at this new patient and compare it to
The malignant tumor Mal doll compared to the benign tumor model and then say in this queso looks like this one
I looked a lot more like the benign tumors I had previously seen as I'm going to buy that as a benign tumor
So
Rather than looking at both houses simultaneously of searching for a way to separate them
A genitive learning algorithm instead built a model of what each of the crosses look like
Almost in isolation details were going about later and then a test time
The new example against the Vietnam although eval this against in the Lakeland Mall though and try to see which of the two models
Matches won't go see the game
Celeste Holm Eliza's on
A discriminative
Learning algorithm learn
Gfy
Give an ex
Or
What would learn
Write some mapping from X to Y directly
I think we talked about the perception album
Mapping from X to the labels direct positive negative
In contrast a generative learning algorithm
Ignorance
Key of
X given y
So this says what are the features like
Given the class so it's up to you if y given X without gfx
Given why is another was given to the tumor is malignant what are the features likely going to be like giving the tumor is benign
What are the features X going to be like
And then and then you also
Generative learning algorithm will also learn Psy fire
Probably lucky I guess
Call the cops when a patient walks into your office before you even examined them before I even seen them
What are the odds that their tumors malignant vs. benign right before you see any future
And so using Bayes rule
If you can build a model for key of x given Y and v y
If you know both of these quantities than using basil when
You have a new Test example
With peaches that you can then calculate the transit why bank with at 1
Right
Where to Ducks by the
And so if you learned this term p x given y then you can plug that in
Ciara
And if you've also learned this term. Why you can plug that in here
So Olympia backs in the denominators
Have you learned both both of those terms in the red square and the RN Square you could plug it into all the
Therefore used to calculate p r y equals 1 given X
Give me the new patient with feature that you can use this phone
Is two quantities in the red energy
Orange circle
So that's a framework was used to build generative learning out
Today you see two examples of generative learning albums I want for Christmas Valley pictures
We can use for constipation and 1/4 tsp features which you can use for
Building with an email spam filter download
See how positive negative sentiment on Twitter isn't natural language processing example Lisa
So let's talk about
Galton discriminant analysis
So
List about this model assuming that the features X are continuous values and when we develop
Generative learning algorithms I'm going to use X & R N so you know I'm going to drop
X0 = 1 convention
We're not going to need that extra Vehicles one so exit now are ended
R n + 1
And the key assumption in gas industry analysis is we're going to spew
That P of x given y
Is distributed gaussian right and otherwise condition on the tumor being malignant the
Mission of the feature is this God damn
Yeah the future is like the size of the size of the tumor of the cell adhesion do whatever features you use to measure to
And conditions on the beam but 9 the distribution is also to Halcyon so
Familiar with a multivariate gaussian
The top of you 1/3 2/5 alright
Article almost everyone
But let me let me go through what multivariate gaussian distribution
Testimony about shaped a multivariate gaussian is a generalization of this familiar a bell-shaped curve
OverWatch random variable to multiple random variables in the same time
So
It security galaxion
With cement back to Mew
And some covariance Matrix Sigma so if Z is
An RN then you would be our end as well as sick mother
Covariance Matrix 2 dimensional
The expected value of Z is equal to
Demean
Andium
Covariance of Z
If you're familiar with multi reticle varices
This is the formula
Right thumb
And this simplifies which One Direction housing addition Alexa knows
So you're following sometimes semi standard convention I'm sometimes the square brackets
Expected value of Z sometimes I just
Show me the square brackets to simplify denotation of it
The derivation from this depth at this step is giving the lecture notes
And so
The probably density function
And this is one of those formulas that I don't know
Over and over all the people is almost
She's running memorizing every time you need it I'll get it so many times I seem to have it stored in my brain by now.
Most people don't even when you just sitting a few you end up memorizing it
Let me show you some pictures of what this looks like since I think I would
That might be more useful in density has to Frances
They control the mean and the variance of this density
So this is a picture of the Gallatin density does a two-dimensional
Alton bump and now I'm upset that mean parameter to zero sum use the to the machine
Franza
At zero
And
The covariance Matrix is the identity identity
So so
So so you have to stand it this also called the standard calcium distribution which means mean 0 and covariance equals
Identity know I'm going to take the program's Matrix and shrink it right so technical grants Matrix and X
A number less than one that should shrink the variance reduce the variability in distributions by do that
The density
The presidency function becomes taller density function
Always integrate the one right the area under the curve is this one as Hell by reducing the
Variance from Identity to 0.6 times as MCA reduces the spread of the Gospel density
Is also makes it taller as a result because of the area under the curve must demonstrate the one now that's
Make it fatter let's make the covariance 2 * identity
They end up with a wider distribution where the values of I guess the actress here
The subpoena Z1 and is easy to access the two dimensions of gas in NC right increases the variance of the density
Set go back to standard golf in Otsego 101
Off diagonal entries
Is calcium density
The optical element is a 0-0 increase that to 0.5 and see what happens so you do that
Then the godson Dead Sea
How can see the change from this round shape to the slightly narrower thing is the farthest point a point
Philadelphia's up
Looking like that where it now
It's more likely that Z1 900 Z1 and Z2 a positively correlated
So let's go to all of these thoughts but now looking at a contour Sophie Skelton densities and selfies
Mickey pump so this is the
Contours of the Gallatin density with a covariance matrix is the identity Matrix
This is supposed to be perfectly round circle V
The actor Frasier makes us look little bit faster but this supposed to be perfectly round circles and so
When the commands Matrix is the identity Matrix Z1 and Z2 uncorrelated
Consoles that the gods of accountancy look like round circles
Then it looks like that further 2.2.8
It looks like that
The way now most of the property most part most probably density function places value on
Z1 and Z2 being positively correlated
Next let's look at what happens if we set the off diet other minister negative
Values right so
AccuWeather think will happen
Let's hit the off diagnosed next appoint 5.5
People seem to need that hand gesture
You and down with the two random variables in negative correlation so you end up with them
This type of property density function
Lights off and then Contours it looks like this
Where is now flaunted the other way so now is even as each other negative correlation and Espanol
All right so so far we've been keeping that mean Vector as zero and just hearing the covariance Matrix
Oh yes every covariance Matrix is symmetric
Grand duchesses interesting calling back to the plantation directions not real
Let me think
Maybe yeah
I think the karez I take this always symmetric and so I would
Usually not look at single Columns of the covariance Matrix and isolation
Bonus analysis top of the eigenvectors of The covariance Matrix
Izakaya reservations point to the principal axis of the ellipse
Thought that's defined by the Constitution
Standard gaussian with mean 0 so the calcium pump is dented a00 cuz Mew
2000 1.20
By 5 so that moves the calcium
Deposition of calcium density right now to move it to a different location move it 2-1
Play 5-1 as though by varying the value of Mew you could also shift the sensor of the godson.
So hope this gives you a sense of them as you very two promises to me and the covariance Matrix of the
2D gaussian density the sorcerer probably. NC functions you can get as result of changing
Are questions about this
So
Chair is the GPA
Let's eat
So
Remember for GPA we need to model G of x given y
X given y equals 0
What's the average density of the features if is a benign tumor
Is gaussian
Samuel e Wright. The formula for calcium
And then slowly I'm going to assume that if is a malignant tumor
Y is equal to 1. The density of the features is also gaussian
And someone point out couple things to the parameters of the GBA model
A new zero
New one
And sigma and the reasons we're going to a little bit we use the same signal
Shuffle classes
I will use different
Mean 0 and 1
And we can come back to this later if you want you could use separate Francis deal Sigma 0
But that's not usually done so going to assume that the two gaussians for the positive negative
The same covariance Matrix but adapter fin means you don't have to make this assumption that this is the way it's most commonly done
The top of the reason why we tend to do that in the second
So this is the model poppy of Y given X the other thing we need to do is Maldo.
Why so why is just a Bernoulli random variable takes on the values 0
And so I'm going to write it like this
Bye to the y x 1 - 5 to the 1 Linus
And you saw this kind of notation
When we talked about logistic regression but all this means is that you know probably FY
Beyond the one is equal to 5
Right because why is he the 01 Pinto does the way of writing
Wife was wonderful.
And I just saw a simple explanation notation when we're talkin about Mississippi
And so the Loft around answer is fine so this is
RN
This is also our end this is our end by end and that's just a real number
So for any less he's so if you can fit Museum
U16 Martin Fighters your data then these parameters will Define
Purevax given Y and Z if y
And so if at test time you have a new patient walk those your office and you need to compute the
This then you can compute write these things in the red in the orange boxes he produces and number and I put in all
All these numbers together number off of your life with one given X and you can then predict molecular
Tuma
So let's talk about how to fit the parameters so you have a training set
As usual I'm going to write the tree train set like this
Iyi for iPhones 1 through end the usual training set
What we're going to do in all the 50 strong is maximize the joint
It's like retards
And in particular
Let me Define the likelihood of the parameters
3 equal to the product from iPhones 1 through hem
RPF SI
Why I
Traumatized by
The Francis
And I'm just going to drop the proms this year
And
The big difference between
A generative learning album like this compared to a discriminative learning album is that
The cost function you maximize is this joint likelihood which is pfx
, why where is Fred discriminative learning algorithm
We were maximizing this other thing
Which is sometimes also called the conditional likelihood
So the big difference between the these two cost functions is that for logistic regression
Linear regression with generalized linear models
You were trying to choose from the stator the maximize pfy
Give an ex
But for genital Journey albums going to try to choose from systematic size pfx
Nyrp f x, y
Right
So
So if you use them matching likely exclamation
Choose Francis by new zero new one
And sigma that maximize the log likelihood
Where this Define has hit a log but the likelihood that redefined out there
And so we actually asked you to do this as a problem stats in the next homework but so the way you Maxima
This is
Look at that formula for the likelihood take logs take the riches of this thing said that there is equal to zero
And it's all for the values of promises and maximize this whole thing and I'll just tell you the answer is it supposed to get
I'll bet that I just have to do the derivation
Write the value of Phi
The maximizes this is you know not that surprising me so it's a fire
Probably why being equal to 1
Right so what's the chance when the next patient walks into your doctor's office that
They have a
Malignant tumor and so the master like we're supposed to fight it
Pictures of all of your training examples with the fraction with label y equals 1
Bias of a coin toss is just a fraction of his you.
Another way to write this is
Song from Michael's 1 cm
Indicator
Pussy so indicated notation on Wednesday
Indicated notation
This notation is an indicator indicator y equals 1
Is the 1001 depend on whether the thing inside is true phrase so does it indicate a notation
And which are indicator of a true statement that you were the one an indicator of a false statement
So that's another way of writing
Liking his formula
And then the maximum likelihood estimate Bermuda 08
Is this artist right now
Sol if you are now
That the mean of all of the features for the benign tumors right what what you do is you take all the
Benign tumors your trainee said they just took the average that's usually very visible wait just look your training set
Look human is all those I guess and then just pick them
Meaning of Ethan.
Average the Future's So This is a way of writing out that intuition so the denominator is Sumter
Carmichael's wants to M indicator y equals 0 and so the denominator will count up the number
Examples that have benign tumors
Right because everytime why are you close zero
You got an extra one and does some
And so the denominator ends up being the total number of
Benign tumors in your training set
And the numerator some particles one indicator is a benign tumor X
Xxi so defective that is
Whenever
A tumor is benign is 1 times the features
Whenever an example is malignant is 0 * a features
As of the numerator is summing up all the features
All the feature vectors for all of the examples that I've been 9
Official actors
What are the examples
Wolf Michael zero
Number examples
What y equals 0
And then if you take this racial if you take this fraction then just tell me all the teachers actors to the benign
Documents divided by the total number of benign tumors in the training set and so that's just it mean that the future back to us
Of all the death
Good night
Examples
And then
That's what I could from you one no surprises is still kind of what you'd expect some of all of the positive examples
Divided by the total number pause examples and get that mean so that's it
I feel like you're ready for me one and then I just write this out
If you're familiar with
Surprise you but if you're less familiar
I guess you can see the details in the
Homework
Don't worry too much about that you can unpack the details in the hill
Ellipse
Microsoft Samsung Galaxy S2 E11 covariance Matrix
To both of these
So these are the
So so so the way I saw the way I'm motivated this was you know I said well if you want to estimate the mean of a coin
Coin toss discounted fraction
Look at these examples and pick the meat or isn't that the truth of explanation to how you get these formulas but the
Fatty sound way to get these formulas is not digest in truth of argument that I just gave is instead
To look at the likelihood
Take logs to the log likelihood take Services Center 1000 solve all these values
Approve more formally that these are the actual values that maximize this thing right bye-bye 1000
Philadelphia self
In problem sets
So
Finally
Happy Fitness parameters
If you want to make a prediction
So give it a new patients how do you make a prediction for weather they are tumors
So
Did you want to predict the most likely
Cross label you choose Mass over y a p r y
Give an ex
And by Bayes rule this is Maxwell the why of pfx given
Py /
Pfx
Now
I want to introduce one what one more piece of notation which is
Location
Most of you guys too
Example
Sodium
Lessee
Boy
So you know the men over the
Of a
0 - 5 squared is equal to 0 because the smallest possible value of Z 0
And the augment
Open z a z - 5 squared
Is equal to 5
So the men is the smallest possible value attained by the thing inside and you augment is the value
You need to plug in to achieve that smallest possible value right so the prediction you actually want to make
If you want to Output the value of a why you don't output a probability
Why that maximizes is Max of this and this v201
So that you cannot mess with that and you notice that this denominator
It's just the constant right doesn't doesn't disappear back Sis why doesn't even appear in there just composite number
And so this is equal to
Does OfficeMax have a y p f x given y
Time of your life so when
Implementing
When when making predictions with gas industry in and out with a generative learning album sometimes to save on
Computation you don't bother to calculate the denominator of all you care about is to make a prediction
Give me the probability then you have to normalize a probability
Okay
So
That's
Examine what the album is doing
All right
So that's the same day the stats and compare and contrast what a discriminative loading our oven versus a j
Genitive running album will do on this data set
His example with two features X1 X2 and positive and negative examples
Running out of them that's a you initially the parenthesis randomly typically when you run
Russian I almost always initialize francis's 0 bit but this just yolks
And then one of the ratio that Brayden descends on the conditional
All likelihood one of the Basin of witches Russian
Where's the Line there that's two iterations reiterations
For the relations and so on and after about 20 duration so converge to
Pretty decent distributive
Valerie
Aggression really searching for line that separates positive and negative examples how about that
Generative learning algorithm
What it does is the following which is fit with gas industry analysis
What do is fit gaussians
To the positive and negative examples
Describe is as if we look at the two houses separately because we use the same covariance Matrix Sigma for the positive and negative
She don't quite look at them totally separately but we do fit 2010 citiz to the positive negative
Apples and then what we do is each point
Try to decide what does it cost label using Bayes rule using that formula that turns out that
This implies the following decision boundary right so Point City upper right of this decision
Boundary that that straight line I just drew you are closer to the negative thoughts you end up qualifying
Apples and points to the lower-level that line you enter California slavery as a positive examples
And I'm also a drone in green here
That does your laundry for this information so so so these to our lives actually come up with slightly different
Does your boundaries for the way you arrive at these two does your boundaries are little bit different
So
Let's go back City
Any questions about this year
Ohio yes good question why why why
We used to separate means news-review one and the single covariance Matrix Sigma
Oh that's that if you choose to build them all this way
Bouncy ends up being linear and sell for a lot of problems in wanted they need to serve on jury
Yeah and it says that you could choose to use to separate transmission
Koke right to do so as well but double the number 4
Has roughly and you end up with a boundary that isn't linear anymore but is that she know that
No
This one
Don't this one very interesting property
About
Got into screen analysis and it turns out that
This mess
Compare
GTA Telecom
Anthem
Exit the premises
Celeste
Learn some say the Francis
Clock
P f y = 1 given X by all these things
Write as a function of x
I'm going to do this exercise in a second but what this means is
What this phone is our this is equal to p of X given y
Equals 1
You know who's the strongest Rise by
Write the nearest promises
Pause p a y equals 1
Spotify Spotify
/ pfx depends on all the promises are yes
So of my basil
Get out this formula is equal to this
The whole thing was just as we saw earlier I guess right once you're pick up ramps
The number you compute by valetine Gallatin density
This is a brand-new D probability is a c p a y equals one primetime
Is mod calculator denominator you can compute
This racial and dusk are the number for the transit YB little one
One example of
What function you get for give y = 1 given X or Y function
Dissidia she thought this for different values of x
So
Let's see let's see if just one feature XOXO
And let's say that you have a few
Negative examples there and a few positive examples there
Simple data set
Alessi white gas industry analysis with you on Thursday to send what just one feature so that's why I
Call the dentist Posner 1D
Let me map all this data to an X axis
I just have to say that Matthew down and
If you fit the Galaxy into each of these two data sets then
Gandalf would you know
Caltrans as follows where this bump on the left is p f x given y equals 0 + 2
Spfx
Given
White Horse one
Navigate does a technical detail that we set the same variance to mm
What does Siri look like
Calcium bumps like this 50/50 equals 1
Blackhawk Tire
No let's go through that exercise I just drive on the left of trying to plot. Why
Plus one given X
Four different values of x equals 1
Giving different values of x
So let's pick a point
Far to the left here
Right with this Maldo you if I should calculate this ratio you find that
If you have a point here
It almost certainly came from this galaxy end on the left
Right here almost became from the Crosby Road calcium
Because the charms of this galaxy Ingenuity example all the way to the left is almost zero
Pure life is very small
Zephyr Point like that you end up with a point
You know very close to zero
Let's pick another point or how about this point
The midpoint
Deposit Gallatin contel so this is really 50/50 so I guess if this is a
Open 5 but that midpoint you would have
POF rifles
0.5
And then if we go to the point way to the Ryukin example way here then you could please show this came from the father
Examples and so
Ghetto
You get a point like that
Now it turns out that if you repeat this exercise a sweeping from left
Write the many many points on the x-axis you find that four points thoughts of the left
The chance of
This coming from the bicycles one causes very small
And as you approach this midpoint
Didn't resist to 0.5 and it surpasses 0.5
And then beyond a certain point
It becomes very very close to 1
Can you do this exercise and actually just put every point on the x-axis evaluate
This formula
What should give you a number between 0 and 1 probability and go ahead and spot you know the values you get
Across like this and it turns out that if you connect the dots then this is
Exactly a sigmoid function
The shape of that turns out to be exactly as she took my phone and then you proved isn't the problem sets as well
So
Both logistic regression and golf industry analysis Ashley end up using
A sigmoid function
To calculate your p r y equals y given X
Sigmoid function uses calculation rather than computer
But the specific choice of the Frances they end up choosing are quite different than you saw
When I was projecting the results on the displays at the Holland Hall points that the two out of them. She come up
What two different decision boundaries so let's discuss when
A genitive algorithm by GTA is superior and when it is from My Little Secret
Is superior
Let's see
Alright so GTA
Analysis
So the generative approach
This assumes that x given y
Equal zero this is gaussian woke Mean musir Uncle Grand Sigma LSU's x given y
Rifles one this is gaussian with me and you want to go over and see, and why is Bernoulli
Amazon
Project fi
And what logistic regression does
This is Ada's prison album
Change when of the back is it
All right
Yeah boy
No don't go does Justice carry you and report on global warming over the weekend. We don't already have phones here
Glowing but hopefully all right
Hurricane stop
Lissie
Soap
What logistic regression accuses me of y equals 1 given X
You know that this is not coming by logistic function right
What was some details about x 0 equals 1
Rights of Justice
Victoza and otherwise
POF
Why closed when given axis logistic
And the argument that I just described just now
If y equals when given X point-by-point the sigmoid curve I drew on the other board
What that illustrates it doesn't prove that you prove yourself in the homework problem
Is that this set of assumptions
Implies
That kid wife was when given X is
Gotham by logistic function
But it turns out that the invitation in the opposite direction is not true
So if you assume. Wife was going to give an ex is governed by logistic function by by the
Shape this does not in any way shape or form assume that x given Y is Gallatin
Steven Michaels
So
What this means is that
GTA
The genitive running album this case this makes us stronger than assumptions
Androgen suppression
Mesa
We have set of assumptions because you could prove these assumptions from these assumptions
And
By the way ASL
Lessee
And so what you see in the lot of learning algorithms is that if you make strongly Malden
Assumptions assumptions are roughly correct then your mother will do better because you're telling me
More information
To the album so if indeed x given Y is gaussian
Benji jwu
Do better because you're telling the album x given Y is gaussian and so can be more efficient than so even if
Very small dataset if these assumptions are brought the correct then GTA will do better and
The problem with ETA is if these assumptions turn out to be wrong so if x given Y is not allowed outside
Then this might be very Bassett of assumptions to make you might be trying to fit the gaussian dusty the dater that is not at all Galveston
And then she did with you
Monopoly
Soap
Just one
Fun fact is another example which is them
Let's say the following are true that say that x given y equals 1
Is Pau Gasol
With France alarm to 1
An x given y equals 0
Is pain song
With off mean
Lambda 0 Orlando 0 and Y is the forest Bernoulli
It turns out that this set of assumptions
Also imply
DatPiff life of 1 given X
Instatistic
Proof of any generalized linear model
I see where where where the difference between these two distributions varies only according to the
Michael franzese July's name
And so what this means is that if you don't know if your date is golfing a plus
If you're using logistic regression you don't need to worry about it we're fine either way
Some soyou maybe 15 days maybe 15 model
And you don't know is it is it some other account
Maybe you just don't know
But there's
But if your data was actually possible but you assumed it was gaussian
That your model might do
Piper Laurie
So the
Cheap
High-level principles when you take away from this is
If you make weak reception as in logistic regression then your algorithm will be more robust
To maulding assumptions such as access you see me today to the Galveston that is not but on the flip side
Small data set then using a model that makes more sanctions will actually
Allow you to do that. Because by making more sandwiches you just telling the album or truth about the world
Hey album the world is calcium and then it will she do to Becca
Oh yeah you got some coffee you know
Yeah you know it's another theory right most daighter on this universe is calcium
Except at the speed dater I guess
Located that you bought you know will not really be gaussian but a lot of it you can convince yourself as they freak out
Tell you the way I choose to use them
The whole world has moved to watch using bigger than 3/8 datasets right
I would probably use logistic regression because with more data
You could overcome
Tell me the adverb of less about the world
So so the album has two Souls of knowledge one source of knowledge what did you tell it what is
Lot of data you know there is a strong Trend to use me to take a Russian which makes less assumptions
Figure out what it wants to figure out from the day.
Right now one practical reason why I still use our rooms like I get a journalist
Like this
Is that is actually quite compilation efficient and sodas I should one use case
We just need to fit a ton of models and don't have the patience to run logistic regression over and over that turns out confusing
Mean and variances of
Covariance matrices is very efficient and so does that apart from the assumptions type of benefit
Which of the general philosophical point will see again later on this for about a week
General principle in machine learning that will see a game in other places but as a very concrete
That's what I think of the phone better from accuracy point of view but as I should very efficient hourly wage of computer games
Is malt
Motivated by computation and Best Buy
Performance
But the general principle is one that will come back to you again later I waited all day to schedule an hour.
The data is
Oh right so what happened to The covariance Matrix is not difference it turns out that Sarah is still
Ends up being a business function for quadratic terms in the oven
Does your boundary you know what that that that looks like this right with positive examples separated by some by some other shape
I knew you could you could you can think that she
Like an Umpire
Play rounders Brown
Yeah is there a test to see if his Galaxy and I can tell you what done in free
I think in practice if you have enough data to do a Cisco test and convection you prom
Probably have an update that they just use logistic regression
Liberty Fair
And it looked really don't Galaxy and then you say but what happens often
Is that
Sometimes you just have a very small tree and it's just a matter of judgment
50 examples of healthcare records then you just have two awesome doctors in Oswald you think
I think that
The machine learning world has
Little bit overhyped big data and yes it's true that way or more days are is great
I love Tater and all happy more days are pretty much never hurts and usually the most
Good job telling people that high-level message no more data almost always help but I think a lot of this
Skilled in machines running these days is getting your elbows to work even when you don't have a million and example if you don't even
We just don't have him attend examples
And it's tender skill in designing learning avermaet is much more
Examples and so the performance difference between teams
Second grade performance examples
Then the high school teams will actually do much much much much better than the Philippines where is it
Performance jacket smaller when you have jaundice
Self and I think that is because of intuitions your what assumptions use genitive discriminative that actually
Distinguishes the high school teams in the in the end unless experienced teams and drive law to perform differences
We have small data if someone goes to you and says I owe you an apology examples you never do any
I don't know if it is a competitor saying that it's a great
Well I don't know
Make a difference
States that still allow
How to drive must pick up their performance then Aloha school team would be able to
What does this so what's a general statement if x given wife with y
One comes from an externship and diffusion asking why it was your account.!
Browser
This would be logistic
I think this is one of the midterm homework problem to prove differently but yeah
Art stick one Loft
Oh just performance Improvement Humanity increase number of coffee
Yeah I think so yes indigenisation of this would be the softmax regression which they talk about
I think it's looking for his true
Lights off
Videography courses in songs
Oh yes right. Bacteria living
And this this thing that we have less days are the album
Needs to rely more on assumptions you code in this is a recurring a female come back to it as well this is One Imports
Principles of a blessed day so you'll still at coding in your knowledge Mathis
This is a fee will come back to you when we talk about much more complicated than the album
So when a freshwater
So you said you did it in the context of continuous.
Peaches
The Lost thing I want to do today is
Taco one more generative learning algorithm called naive Bayes
I'm going to use as an example email spam qualification but this this is this
Price for Inter National language processing
Knock-down examples
Someone's trying to sell something and you write a text description right hey I have a second hand you know remember I'm trying to sell
On eBay how do you take that text that someone wrote a description
Examples of problems
One of two categories. One on Stanwell one of maybe thousands of calories and trying to take a profit
Description of fighting to one of the classes
And so the first question we will have is
Email problem given email complication problem how do you represent it as a feature
And so on
NIU baseball games to do is take your
Email take a piece of email and first map it to a feature Vector X
And we'll just as follows which is first that starts with a
Let's talk to the English dictionary and make a list of all the words in the English dictionary
Seth weathers aardwolf
Yo you know Sam and buy stuff so they buy and then
And in the Lost World in my dictionary is Vic Ricky
Which is a technological chemistry that refers
To that fermentation process in Brewing
Look at the top 10000
They're think that's anywhere in the other way through this to look through your own email Lopez just fine
The top ten thousand of a current that you emailed
Gmail account from my phone
Even if it doesn't appear in the official was it like
All right
And so given an email what we would like to do is then
Take this piece of text I represented the speech effexor and so one way to do this as you can
Create a binary feature vector
That
Put the one
If it wet appears in the email
And put a zero if it doesn't fit going to email that I asked you to
AFE hasn't emailed for the one there
Did not try to sell avocado oil for zero there
Bye
An email
And turn it into a binary feature vector
And so here the feature Vector is 0 1
To the end because of the end of national finally feature Vector where where for the purpose of illustration
And it's 10,000 because using that appear in your email
Chinese at Asda
Dictionary
That you will use
So
So another words x i
Is indicator
Would I
Appears in the email right depending on whether or not I
From this list
Appears in your email
No
In the 90s album we're going to build a generative
Learning algorithm
And so we want them although
P of x given y
But
There are two to the ten thousand
Possible values of x
Because X is up on your back to this 10000 dimensional model X
Industry portway as a multinomial distribution over
Get out to the ten thousand
Possible outcomes then you need do you need to get out to the ten thousand
Write a lot to the 10000 - 1
Because of excessive number of Ramsey
So in a naive Bayes algorithm princess you
AR
A conditional Independence
Given y
Let me just write down what this means FX1
Given y by the chain rule of probability
This is equal to p f x 1 given y
X p x 2 given
X 1 and y x Civic SI given x 1
Why up to your pfx 10000
Given right
So I haven't made any assumptions yet this is just a true statement of fact is always true by they buy their chamber will probably
And what we're going to see you which is what does assumption is
Is that this is equal to this first term no change
E x 2 given y
10000 given wines
So this assumption is called a conditional Independence
But you're assuming that so long as you know
Why
The chance of seeing the word
Ong Bak
And you email does not depend on whether to write a appears in the you know
And this is not a true assumption in that there's just not enough time to choose
Just that sometimes your data isn't perfectly calcium to the Galaxy and you can kind of get away with it so this
Assumption is not true in a mathematical sentence but it may be not so horrible that you can't
Get away with it
Right hand soap
If you have any problems
This assumption is summarized in this picture and you haven't because Fiesta 28 this picture won't make sense but don't worry
Once you know the cloth label is this a mano spam whether or not each word appears with a
Does not appear is independent
Understood the mechanics of this assumption is really just captured by this equation
And you just use this equation that's all you need to drive 90 phase but the intuition is that
If I tell you whether this piece if I tell you that this piece of email is spam
Then whether the wit by if he isn't it doesn't affect you believe the weather the word mortgage off discount or whatever
Understand me words appear
So just to summarize this is product from I = 1 through end
Appearance attorney
Given y
All right
So the parameters of this model
RR
I'm going to write fine so strip
Hey
Given y = 1 as the probability that makes J equals 1
Given y equals 1
My sister J
And then I'm
Bye
Adjusted distinguish all these lies from each other
Hilda Speranza says if a spam email
His wife was want to stand why is yours mom's bad if it's spam email what's the chance of where J appearing
In the email if it's not a spam email what sitcoms of where she appear in the you know and then also what
What's the cost Pryor what's the probability that the next email you receive annual
And so
You would similar to golf industry analysis
Write out the John's joint likelihood so they don't like it
What is 4 ounces
Write as a product
Given these prices
Summertime Will be hot for golf industry analysis
And the maximum likelihood estimate may be take this big loss figuratively
Saw for the ballast and maximizes you find that DMSO likely estimates of the provinces are
Biwi pretty much what you'd expect
Maybe it's just a fraction of spam emails
And the fire K given y = 1 is some
Well I write this on an indicator function notation
So that's the indicated function notation that writing out look through your chain
Find all the spam emails and of all the spam emails I
Examples of y equals one count of what fraction of them
Edward J in it
Charles Edward J appearing
You know it's just well all the spam emails
What fraction of them contain the word buy when what fraction of them had
Yeah XJ hose 1/4 of the word by
And so it turns out that if you implement the cerebellum
Well it will nearly where I guess
But this is now you pays for
Female Saint classification
I mentioned the one reason this and it turns out that was once Texas's album which was Hoffa
Kabobs on Wednesday this is actually
Pistachio not too horrible stand cause fire
It turns out that application YouTube better than this almost all the time
But this is a very efficient album because that's making these promises is counting and then computing power
So there's nothing about this very efficiently and also
Updating this small though even if you can you date a evens getting Yu-Gi-Oh uses his Marcus panel
Can you take an update this morning very efficiently but it turns out that
The biggest problem with his what happens if this is 0 open
Vito's in some of these equations on Wednesday
Good questions before you wrap up
Genesis learning algorithms will come back
Back on Wednesday and then about some more fine details how to make this right