the following content is provided under a Creative Commons license
your support will help MIT OpenCourseWare continue to offer
high-quality educational resources for free
to make a donation or view additional materials from 100 to them IT courses
visit MIT OpenCourseWare at 0 CW
that MIT DDD you
so
today our discussion of basin inference which we started last time
as issue probably so there's not a huge lot of concepts that we're introducing
at this point in terms of for specific skills of calculating probabilities
but rather it's more food
interpretation and setting up the framework so the framework in basin
estimation is that there is some
barometer which is not known but we have a prior distribution on it these are
beliefs about what these variable
might be and then we obtain some measurements
and the measurements are affected by the value of that parameter that we don't
know
and this effect the fact that X is affected by a theta
is captured by introducing the conditional probability distribution the
distribution of
X the Pens on say it's a conditional probability distribution
so we have four wheelers for these two
then said is the prior density in the conditional density
and given that we have these if we multiply them we can also get the joint
density of
X and theta so we have everything that there is to know in this setting
and now we observe the random variable X
given this random variable quit can we say about state to
well quite we can do is we can always cut relates the conditional distribution
of eight to
given X and to know that we have the specific value of
X we can plot this is a function of a to okay
and this is the complete answer to a basin inference problem this post area
distributions captures
everything there is to say abouts satyr that's what we know about state that
given the ex that we have observed
fate they still around to meet still unknown and it might be here there or
there
with several probabilities on the other hand if you want to report
a single value for a fate that then you do some extra work
you continue from here and you do some data processing on
X doing data processing means that you apply
a certain function on the data and this function is something that you design
it's the so-called estimator and ones that function is applied
it outputs and estimates of State Bank which we call say too hot
so okay so this is sort of the big picture of what's happening
now one thing to keep in mind is that even though I'm writing single letters
here
in general say to or exclude be vector
random variables so think I love this it could be
collection say that one fate that tooth 83
and maybe we obtain several measurements so this
X is really a vector x1 x2 up to expand
rate so know how do we choose
the if fate that you report there are various ways of doing it's one is
to look at the posterior distribution and report to the value of a to
it which the density or the PMF is
highest this is called the maximum a posteriori probability estimates so we
pick a value of a tougher
which the posterior is much movement we reported an alternative way
is to try to be optimal with respect to
mean squared error so quite is this if we have a specific estimate their G
this is the estimates its going to produce
this is the true value of a test so this is our estimation error
we look at the square over the estimation error and look at the average
value we would like this squared estimation ever to be as small as
possible
how can we design our estimator G to make that ever
as small as possible it turns out that the answer is
to produce as an estimates the conditional expectation of eight the
given X
so the conditional expectation is the best
estimates that you could produce if your objective
is to keep the mean squared error as small as possible
so the statement here is a statement of what happens in the average overall
state doesn't all
X is that the may happen in our experiment
actually this is a conditional expectation is an estimator has an even
stronger spring
stronger property not only talking while on the average
but it's also optimal given that you have made a specific
observation no matter what you observe let's say you observe the specific value
for
the random variable X after that point
if you're asked to produce a best estimates
fate that cuts that minimizes
these mean squared air or your best estimate would be the conditional
expectation given the specific value that you have observed
these 2s these two statements say almost to the same thing
but this one is a bit stronger this one tells you
no matter what's specific X happens
the conditional expectation is the best estimates this one tells you
on the average overall X is that may happen
a the conditional expectation is the best
the best estimator now this is really a consequence of this
if the conditional expectation is best for any specific X
then it's the best one even when X is left around opened your
averaging your ever over all possible X's
okay so now that we know quite is the optimal way of producing in estimates
let's do is simply example
to see how things work out so we
costarred with an unknown around a variable say to that which is uniformly
distributed between four and 10
and then we have an observation model thats tells us that
even divide your fate to X is going to be random variable that ranges between
say to -1 think the plus-one
so think of X is in noisy measurements of satyr
plus or plus some noise
ok which is between miners the London plus one
so really the model that we are using here is that
X is equal to say to plus you
who were you is uniform
on -1 and plus one so we have the true value of 80 but
X could be fatal -1 or it could be all the way up to fate the plus-one
and X is uniformly distributed on that interval that's the same as saying that
you
is uniformly distributed over over the sea interval
so now we have all the information that we need we can construct the joint
density
and the joint density is of course the prior density times
the conditional density we got both of these
both of these are constants so the joint density is also going to be a constant
16 times one cuff
this is one over twelve but it is a constant
not everywhere only on their names of possible
X's and faithless so theta can take any value between four and 10
so these are the values of a turn and for any given value Fator
X can't take values from theta -1
up to fate that plus one so here if you can imagine the line that goes with
slope
1 and then fate that X can take
that the value of eight a-plus blast were -1
so this service these that this object year
this is the set of possible X in state affairs
so the density is equal to 1 over twelve
over this sets and its 0
everywhere else solo outside here the density 0 the density only
applies at that point alright so no were asked to estimate fate
fate that in terms of X so we want to build an estimator
which is going to be a function from the ex's to the status
that's why I chose the axes this way X to be on the psych state on that axis
because the estimator we're building is a function of X
based on the observation that we obtained we want to estimate state
so we know that the optimal estimator is the conditional expectation
given the value of x so what is the conditional expectation
if you fix a particular value of pics
let's say in the strange so this is our
X then quite do we know about fate to
we know that say to lies in this range
fate that can only be somewhere between those two values
and what kind of distribution the state to have
what is the conditional distribution of they think given X
well remember how we build conditional distributions from joint distributions
the conditional distribution
is just this section of the
joint distribution applied's to the place where we're conditioning
so the joints is constant
so the conditional is also going to be constant density over this interval
so the posterior distribution of eight er is uniform
over these interval so if the posterior a fate is uniform over that interval
the expected value feder is going to be the midpoints
of that interval so the estimates
which you report if you observe that say to
is going to be this particular point here it's the midpoint
the same argument goes through even if you obtain an
X some cooler here
given this X say to
can take if fate that can take a value
between these two values
fate is going to have a uniform distribution over the sea in their vote
and the the conditional expectation of a to given X
is going to be the meat points over that interval
so know if we plot our estimate dealer
byee tracing meets points
in this diagram quite you're going to
to obtain is a curve that goes it starts like
vs then it's changes slope
so that keeps track of the meet point and then it goes like
that again so this blue curve here
is are geoVax which is the conditional expectation of faith a given that
X is equal to little X
so it's a curve the because it causes so
in our example it consists of three straight segments
but overall its nonlinear it's not a single line through this diagram
and that's how things are in general geoVax are
optimal estimate has no reason to be a linear function of exceeds in general is
going to be some complicated curve
forget so how good these are estimates
mean you reported your ex your estimate of
say to based on pics and your boss asks you
what kind of hair do you expect to get
having observed the particular value of X what you can report to your boss
is quite you think the mean squared air or is going to be
we observe the particular value for x so where conditioning
into our living in this universe given that we have made this observation
this is the true value Fator this is the estimate that we have produced
this the this is the expected squared
error given that we have made the particular observation
now in this conditional universe this is the expected value of fate that
given X so this is the expected value of these random variable
inside the conditional universe so when you take the mean squared off a random
variable minus the expected value
this is the same thing is the variance of that from the variable
except that he did the variance inside the conditional
universe having observed X
fate that is still around the variable
it's distributed according to the posterior distribution since it's around
available
it has a variance and that variance
is our mean squared error so this is the variance of the
posterior distribution of aid that given the observation that we have made
okay so quick he's the variance in our example
if X happens to be here
then fate that is uniform over this interval
and this interval cuz length to
Fator is uniformly distributed over in interval
of links to this is the posterior distribution of APE
what is the variance then you remember the formula for the variance 0 for
uniform random variable
it is the length of the interval squared
divided by 12 so this is one-third
so the variance of stator the mean squared error
is going to be one-third quit never this kind of picture
applies this picture applies Quinn
X is between 5&9 if X is less than five
then the picture is a little different and fate is going to be uniform
over a smaller interval and so the variance of fate that is going to be
smaller as well so let's start plotting our mean squared error
between 5&9 the variance of State Dun
the posterior variance is one-third
now when the X false in here
Fator is uniformly distributed over a smaller
interval this sizeof this interval changes Lee nearly over that range
and soak when we take this square over that
the square sizeof thats interval we get the quadratic function of how much we
have moved through that corner
so at that corner what is the variance of state
well if I observant X that's equal to 3
then I know with certainty that fate that is equal to 4
then me in a very good shape I know exactly what state is going to be
so with the variance in this case is going to be 0
if I observant X that's a little larger than say to is now around 'em
takes values on a little interval into the variance of 80 is going to be
proportional to the square
over the length love that little interval so we get a curve that starts
rising quadratically from here
and goes up stalwarts one-third
at the other end of the picture the same is true
if you observe in X which is 11
then fate that can only be equal to 10
and so the error in say that is equal to 0 this
0 error variance but as we hoped a nexus that they're slightly less than eleven
then the mean squared error again rises quadratically
so we end up with a plot like this
quite this plot tells us is that certain measurements
are better than others if you're lucky
and you see X equal to 3
then you're lucky because you know say to exactly what it is
if you see an X which is equal to 6%
then you're sort of unlike eat because he doesn't tell you think that with
great precision
say that could be anywhere in that interval and so the variance of fate
that
even after you have observed X is a certain number
one-third in our case so
the moral to keep out of that story is that
the the air or variance
or the the mean squared a ruler
defense all what particular observational you happens to obtain
some observations maybe very informative
ants ones you see a specific numbers and you know exactly what fate
is some observations might be less informative you observe your ex
but it could still leave a lot of uncertainty about state
so conditional expectations are really the cornerstone of the
basin estimation there particularly popular
especially in engineering contexts their
their use the lot in signal processing in communications control theory
so on so that makes it worse playing a little bit
with their theoretical properties and get some appreciation of
the few supple subtleties involved here
%uh no numa in reality Inc what we're going to do here
but it's going to be a good opportunity to practice
when you play Shiv conditional expectations
so let's look at the expected value
of the error over the estimation error that we obtain
so say to cut is our estimator is the conditional expectation
say to cut my newspaper is how what's kind of care or do we have
if they to cut before estimate is bigger than theta
then we have made a positive air if not
if it's on the other side you have made the negative error
then it turns out that though the average the errors consul
each other out on the average so let's do this calculation
let's calculate the expected value of them error
even X know by definition the
air or is expected value fate that cuts my nurse
fame to view them X we use linearity of expectations to break it up as
expected value fate the hut even X minus expected value fate that
given X
and no what are estimates
is maids on the basis of the data
over the X's if I tell you
X then you know what fate the cuts is
remember that the conditional expectation is a random variable
which is a function of the random variable own quit your conditioning
on if you know X then you know
the conditional expectation given X you know quite fayed
is going to be so think that that is a function of
X if it's a function of X then once a tell you
X you know quite fatal heart is going to be
so this conditional expectation is going to be fatal heart itself
here this is just by definition say perhaps
sent so we get equal twenty 20
so what we have proved is that the conditional no matter what
i have observed and given that they have observed something
on the average my error is going to be
0 this is a statement
involving equality of random variables
remember that conditional expectations are random variables
because the defendant the things your conditioning
on 0 is sort of a 3v around a variable
this tells you that this trend available is identically equal to the zero around
very more specifically its tells you that's no matter
what value for X you observe
the conditional expectation of the error is going to be 0
and this takes us to the statement here which is an equality between
numbers no matter what specific
value for capital X you have observed
your error on the average is going to be equal to 0
the so this is a less obstruct version
over the statement this is an equality between two numbers
it's true for every value of X so it's
through for in terms of these random variable being equal to that frontal
variable
because remember according to our definition this random variable
ease the random variable thats takes this specific value
when capital X happens to be equal to little X
know this doesn't mean that you're error is 0
it only means that your error is is likely in some sense to follow the
positive side this to follow the negative sides
so sometimes you're ever will be positive sometimes negative until the
average these things
Council out and give you is zero
on the average so this is a property that's a
s times giving the name we say that the
fate the hot is unbiased
so theta cut other estimates does not have a tendency to be on the high side
does not have a tendency to be on the low side
all the average its just it's just right
so let's do little more playing here
let's see Kyle theta a cow or
area is related to an arbitrary function of the data
let's do this in a conditional universe
of
and to look at this quantity
no when you know X
in a conditional universe where X is known then HO vikes
is known and so you can't boil it outside the expectation
in the conditional universe where the value of X is given
this quantity becomes just the constant
there's nothing random about it so you can point out the expectation
and right thinks the subway AFP
and we have just calculated that this quantity is
0 so this number turns out to be
0 as well okay
now who a having done this we can take expectations of both sides
and now let's use the law of iterated expectations
expectation of a conditional expectation gives us the
unconditional expectation
of of and this is also going to be
0 so sheer we use the law of
iterated expectations
of
ok
okay have why are we doing this we're doing this because I would like to
calculate
the covariance between faith a till the end fate the cot
that these ask the question is there a systematic relation between the
error and the estimate so to calculate the covariance
AFP AFP
we Hughes the property that we can calculate its covariance is by
calculating
the expected value for the product my news
the products over the expected values both
AFP and quite do we get
this is 0
because so quite we just proved
and
this is 0 because of what we proved
earlier that the expected value of the error
is equal to 0 so the covariance
between
the covariance
between the error and any function of
X is equal to 0 lets you use that's to the case where the function of
X who were considering is say to
had itself think that that is our estimates it's a function of pics
so this 0 results would still apply
and we get that this covariance is equal to 0
okay so
so that's what we proved
okay it's a quitter the morals to take
out of all these first is you should be
very confortable with this type of calculation involving conditional
expectations
the main two things that we're using other that when you condition on there
under variable
any functional that's random variable becomes a constant
and can be pulled out the conditional expectation
the other thing that we are using is the law of iterated expectations
so these are the skills involved now on the substance why is this result
interesting
this tells us that the air or is and correlated with the
estimate what would happen if that weren't quite sick hypothetical
situations where these would not happen
could ever say to how this positive
my a ruler tends to be negative
thats suppose that whenever if they to cut his big
then you say all my estimate is too big maybe the truth eva is on the lower side
so I expect my error to be negative that would be a situation that would violate
this condition this condition tells you that
no matter what fate of hot is you don't expect your
a road to be on the positive side or on the negative side your
there will still be 0 on the average so if you obtain a very high estimates
this is no reason for you to suspect
that the truth data is lower than your estimate
if you suspected that the true state that was lower than your estimate
you should have changed your say too hot it
if you make an estimate and after obtaining that estimates you say
I think my estimate is too big and so the error is negative
if you thought that way then that means that your estimate is not the optimal
one that your estimate
should have been corrected to be is smaller
sand that would mean that there's a better estimate
you want you used but the estimate that we are using here is the optimal one in
terms of mean squared error there's no way of improving it
and this is really captured in that statement
that is knowing fate the house doesn't give you a lot of information about
the error and gives you therefore no dole reason to
I just your estimate from quite it was
finally
consequence of all this this is the definition of the air or
send fatal to the side some faith that the lead to that side
you get to this relation that trooper I'm the true parameter
is composed of two quantities the estimates
and then the error that they goats with the mines sein
these two quantities are uncorrelated with each other
their covariances 0 and therefore the variance of these is the sum love the
variances
over these two quantities
so what's Watson interpretation love this
equality there is some inherent randomness
in there and available say to that we're trying to estimate
say tough cuts tries to estimate is tried to get close to it
and these if that the cuts always
she stays close to state that since they plays random
say the fat must also be quite random so it has uncertainty in it
and the more uncertain fate the hot is its
the more it moves together with a to so the more uncertainty it removes
from theta and this is the remaining uncertainty in theta
the uncertainty that's left after we've done you other
estimation so ideally to have a small
air or we won't this quantity to be small
which is the same as saying that this quantity should be big
in the ideal case fate that cuts is the same a state that
that's the best we could hope for that corresponds 20
a road and all the variance all the insurgent in theta is absorbed by the
insurgent in say two hot
interestingly this relation here is just another variation of the law of total
variance that we have seen at some point in the past
I loose keep that starvation but it's an interesting
it's an interesting fact and it can gives you an alternative interpretation
of the law of total variance
okay so no let's return to our example
in our example we obtain the optimal estimator
and we saw that it was a nonlinear
curve something like this I'm exaggerating the corner
little bits too so that its nonlinear this is the optimal estimator
it's a nonlinear function of x which means no mean you're generally means
complicated
I sometimes the conditional expectation is really hard to compute
because whenever you have to compute expectations you needs to
the do is to do some integrals
and if you have many random variables involved its bite corresponds to a
multi-dimensional
integration we don't like this can we come up
maybe with the simple simpler way of estimating theta
with coming up with the point estimates which still has some nice properties it
has some good motivation but
is simpler quite the simpler mean
perhaps eating your let's the
for theirselves in a straight jacket since and restrict ourselves to it
estimators
the third over these four my estimates
is constrained to be a lean year function of the ex's
so my estimates my estimate there is going to be a curve
a linear curve it could be this it could be that
the maybe it would want to be something like this
I want to choose the best possible linear function
what does that mean it means that they right may fade haircuts
in this form if I fix
is certain a and be I have fixed the functional form of way estimator
and this is the corresponding mean squared error
that's the error between the twID parameter in the estimate of that
parameter would take the square
this proof to
and no the optimal linear estimator is defined as one
fork which these mean squared error is smallest possible
overall choices of a and be
so we want to minimize this expression overall
A's and B's how do we do this minimization
well this is a square you can expanded
writes down all the terms in the expansion of the square
so you're going to get the term expected value fate the squared
you're going to get another term a squared expected value for x squared
another term which is B squared
and then you're going to get to there is crossed arms what you obtain cork what
you have here
is really a quadratic function of a and be
so think over this quantity that we're minimizing is some function
H of a and be and it happens to be quadratic
how do we minimize the quadratic function we set the derivative love this
function with respect to
aliens be 20 and then do the algebra
after you do the algebra you find
that the best choice for a is
this one so this is the coefficient next to
X this is the optimal a
and the optimal be corresponds with the constant
storms so this term and these times that
together are the optimal choices of be
so the algebra itself is not very
interesting what is really interesting is the nature of the results
that we get here if we were to
plots the results on this particular example
you would get the curve that's the
sort of something like vs
certificate goes through the through the media love these diagram and
is a little slanted in this example
X in theta are positively correlated
bigger values of ex generally correspond to bigger values of State
so in this example the covariance between X and theta is positive
and so are estimates is
consists it can be interpreted in the following way:
the expected value of a to is the estimates that you would come up with
if you didn't have any information about
about state if you don't make any observations
this is the best way of estimating fate
but they have made an observation X
and they need to take it into account I look at this difference
which is the piece of news contains
in X that's what X should be on the average
if I observant exc which is bigger think what they expected it to be
and since X with a to are positively correlated
this tells me that data should also be
bigger than its average value whenever I see in X that's
larger than its average value this gives me an indication that fate there should
also probably be
larger than its average value and so
I'm taking that difference and multiplying it by a positive
coefficients and that's what gives me a curve here that has opposed if slope
so a this increments the new information contained in X
as compared to the average value we expected a priori
that increments allows us to make a correction
to our prior your estimate docstater and
the amount of that correction is guided by the covariance of
X with eight if the covariance of X worth eight that were 0 that would mean
there's no systematic relation between the two
and in that case obtaining some information from X
doesn't give us a guide as to how to change the estimates of
fate to if that were 0 we would just stay
with this particular estimates were not able to make a correction
but when there's a non-zero covariance between X and theta
that covariance works as a guide for us
to obtain a better estimates better estimate of eight
the how about the resulting mean squared a ruler
in this context turns out that there's a very nice formula for the mean squared
ever
weird thing from the best linear estimates
I what's the story here these
the mean squared error that we have had something to do with the variance
of the or regional random variable
the more uncertain our original random variable is
the more error we're going to make on the other hand when you
the two variables are Co related we exploit that correlation to improve our
estimate
and the bigger that correlation is these are all here is the correlation
coefficient between the two random variables
when this correlation coefficient is larger this factor here becomes smaller
and they're mean squared error become smaller so think over the two extreme
cases
one extreme case is Quinn role is equal to
one so X in faith are perfectly correlated
when they're perfectly correlated its once I know
X then I also know theta
and the two random variables are million yearly related
in that case my estimate is right on the targets
and the mean squared error is going to be 0
the other extreme case is the for all is equal to 0
the two random variables are uncorrelated in that case
the measurement does not help me estimate
say to and the uncertainty that's left
the mean squared error is just the regional variants
of state so the uncertainty in faith that is not does not get reduced
so morals the estimation air or
is a reduced version of the or regional
amount of uncertainty in the random variable theta
and is the more the larger the correlation between those two random
variables
the better we can remove uncertainty from the original random variable
I didn't derive these four wheeler but this just
mother of algebraic manipulations we have the formula for state the hot
subtract vapor from that four wheeler take square take expectations and do a
few lines of algebra that you can read in the text
and you end up with this really neat and clean four wheeler
know the I mentioned in the beginning of the lecture that
we can do in for ins with a tan X is not just being single numbers but they could
be vector around all variables
so for example we might have multiple data that give us information about
X this discussion here
there's no there are no victors here so this discussion was for the case with a
to and X
were just scaler one-dimensional quantities
what do we do if we have multiple date suppose that state they still
a scaler its one-dimensional but we make several
observations and on the basis of these observations we want to estimate fate
the optimal least mean squares estimator
would be again the conditional expectation of fate that
given X that the optimal one
and in this case X is a vector
so in the the general estimator we would use would be
this one but if we want to keep things simple
and we want the estimated to have a simple functional form
we might restrict two estimators that they really near functions
of the data and then the story is exactly the same as we would
we discussed before I constrain myself to
estimating theta using a linear function of the data
so my signal processing books he's just
just applies a linear function and i'm looking for the best coefficients
the coefficients that they're going to result in the least forcible
squared error this is my squared error this is my estimates my nose
do think I'm trying to estimate squared and then taking the average
how do we do the same story as before
this quantity the X's in the state does get other edged out because you have an
expectation
whatever is left is just a function of the coefficients over the A's
and of be as before it turns out to be a quadratic function
then we set the derivatives over this function of
A's and B's with respect to the coefficients we set its 20
and this gives us a system of linear equations
it's a system of linear equations that satisfied by those goofy since it still
in your system because this is a quadratic function over those goofy
shins
so the to get closed form formulas
for in this particular case one would need to introduce vectors and matrices
and matrix inverses and so on
the particular formulas are not so much what interests us here
rather the interesting thing is that this is simply done
just the using straightforward solver so freely in your
equations the only thing to you need to do is to write down the correct
coefficients over those nonlinear equations
and the typical coefficient that you would get would be
what that's a typical coefficient would be
that's eight let's take it difficult to remove this quadratic when you expand
that you're going to get the terms such as
81 x1 times 82 x2
when you take expectations you're left with a 182
times expected value of x1 x2
so with typical okay so this would be
it would involve terms such as 81 squared expected value for x one squared
you would get storms such as 81 8 to
expected value x1 X to
and lots of other terms here
would have it too so
you get something that's quadratic in your coffee since
and the cone stuns that show up
in your system of equations or things that have to do with the expected values
of square so few random variables
or products if you're under variables
to writes down numerical values for these
the only thing you need to know are the means and variances
love your random variables if you know the mean and variance
then you know for this thing is and if you know the covariances as well
then you know quite this thing is so
in order to find the optimal lean year
estimator in the case of multiple data you do not
needs to know the entire probability distribution
the random variables that they were involved you only need to know
your means and covariances these are the only quantities that affect
the construction of your optimal estimator
we could see this already in these four wheeler
the form of my optimal estimator
is completely determined once I know
the means variance and covariance
over there and the variables in the in my model I do not need to know
the the details distribution
of the random variables that their involved here
game so as I said in general you find the form over the optimal estimator byee
using a linear equation solver their special examples Inc which you can get
closed form solutions
the nicest simplest estimation problem one can think of
is the following you have some uncertain parameter
and you make multiple measurements of that but I am at their
in the presence of noise so the W eyes
are noises I corresponds to your I experiment
so this is the most common situation that you encounter in the lab
if you're dealing with some crosses you're trying to measure something you
measure it over and over
each time your measurement has some random error and then you need to take
all your measurements together and come up with a single estimate
so the noises are this assumed to be independent of each other
and also to be independent from the value of the true parameter
without loss of generality we can assume that the noises have zero mean
and they have some variances that we assumed to be known
faith itself is a prior distribution with a certain mean
and the certain variants so the form of the optimal leader estimator
is really nice well maybe you cannot see to write the way
because this looks messy but what is it really
it's a linear combination of the X's
and the prior you're mean and it's actually
it waited upwards over the X's and the prior mean
here we collect all the coefficients
that we have at the top so
the the cold thing is basically a weighted
average
the CE the one over Sigma i squared is the weight that we give 2x
I E and in the denominator we have the some
of all the weights so in the end worst
we're dealing with the weighted average you from you was equal to 1
and all the excise were equal to one then
our estimate would also be equal to 1 now the form of the weights that we have
is interesting any given data points
is weighted inversely proportional to the variance what does that say
if my I data point has a lot of variance
if W I is very noisy
then X I is not very useful is not very reliable
so and giving its a small weights
large variance a lot of error in my
X I means that they should give it to smolder
wat if two data points have the same variance their
of comparable quality then I'm going to give them
equal weight the other interesting thing is that
the prior-year mean is treated the same way as the ex's
so it's treat this as an additional observation
so we're taking a weighted average over the prior-year mean
and over the measurements that we're making
the formula looks is easy for the prior mean was just
another data point serve that's the way to thinking about basin estimation
you have the your real data points the X is that you observe
you also had some prior information this plays a role similar to a data point
interesting note that if or random variables are normal
in this model the soap 2 million your estimator happens to be also the
conditional expectation that's a nice thing about
normal random variables that conditional expectations turn out to be linear
so the optimal estimates and the optimal linear estimates turn out to be the same
and that gives us another interpretation of in your estimation
leaders estimation is essentially the same is pretending
that all random variables are our normal
so that's a certificate sites points
though click to close with the Komen
the you do your measurements and you estimate state that on the basis of
X suppose that instead you have a measuring device that measures
XQ instead of measuring X you want to estimate state
are you going to get a different estimates
well X and XQ contains the same
information telling you X is the same as telling you the value of
XQ so the posterior distribution of faith a given X
is the same as the post your distribution if they think even XQ
sin so the mean so these posterior distributions
are going to be the same so doing transformations to your data
does not matter if you're doing optimal least squares estimation
on the other hand if you restrict yourself to doing year
estimation then using a linear function of
X is not the same as using a linear function of
XQ so this is a linear estimator
but where the data are the excuse
and we have a linear function over the data
I so this means that when you're using the in years
estimation you have some choices to make
lean your own what sometimes you want to plot your data
on the not ordinary scale and try to plot their
line through them sometimes you plot your date on the logarithmic scale
and try to plots a line through them which scale is the appropriate one
here it would be a cubic scale and you have to think about your particular
model to decide which version
would be a more appropriate one finally when we have multiple
data sometimes these multiple data
mites contains the same information
so X is one data point excludes another data point X cuba's is
nother data point the three of them contain the same information but you can
try to form a linear function
of them and then you obtain a leaner estimator that has a more general form
as a function of x so if you want to estimate your say to
as the cubic function of X for example
you can set up in your estimation model love this particular form
and find the optimal coefficients the A's ants
and to be alright so the last slide
just gives you the big picture of what's happening in the basin inference
for you to ponder basically we talked about three possible estimation methods
election opposed to your e
or per team who are you we mean squared terrorist Imation in the near mean
squared error estimation or at least squares estimation
and is a member of Standard examples that you will be seeing over and over
the station tutorial homework and so on cops on exams even
queer we take some nice try your son some unknown parameter we take some
nice models for the noise or the observations and then you need to work
out posterior distributions in the various estimates and compare them