welcome everybody to this webinar we're
going to be talking about numerical
issues and we Furr getting into that
well Robbie's been around for twenty ten
years and the teaming Grobe has over 20
years experience in optimization
problems and a numerical issues for sure
and actually we have built this
experience helping our customers now all
of them use Rob in a daily basis and and
the kind of applications that they solve
it is really wide from accounting to
programming to you really name it and
it's from this experience instead this
webinar is drawing on so let's go to the
global guidelines for numerical issues
or you may say why things go wrong and
more importantly how to avoid it so what
are numerical issues that's probably a
hard thing to say we may say that
actually people talk about numerical
issues whenever they encounter
unexpected results from their algorithms
or even they get inconsistent results
for the same problem and the source of
these are numerous however the most
common ones I will share these for these
are or can be more easily hampered the
first one is rounding which I'm going to
be talking about and then there is the
problem of the big gap between
theoretical numbers and what the
computers actually understand and
represent those numbers and coming from
that gap it's also a common thing for us
to forget that these numbers have limits
and those limits impose to what we can
ask for regarded on precision and
finally we're going to be talking out so
about it conditioning so let's get first
on trying to understand where these
issues come from the first and probably
the easiest one is rounding coefficient
probably this is best explained with an
example and this can really be very very
dangerous so this is a very simple
example I assume that you give the
solver
these two constraints and you may look
at them and you say well you know what
this really looked like the same
constraint and the numbers are
well close to be really one-third that
0.333 could be interpreted as one sir
however we always will interpret the
numbers that you gave the solver asked
the true number and if you do that and
you do the linear algebra on do the
substitution what you will realize is
essentially these two constraints are
really implying that X is one and Y is
zero and this comes from assuming that
the numbers you're creating the solver
are what you really want them to me so
if instead of using this kind of
approximate a fair number you have had
full precision let's say these two
equations a very different story will
have a Rison so essentially you will do
the algebra you do the substitutions and
then you end up with these expression
down here which is 2 times 10 to the
minus 16 times X plus 1 plus the same
coefficient and this is really really
really close to b1 and actually probably
will say well this is one and
essentially what we will do is that drop
one of those two constraints and except
anything that satisfies these linear
constraint as feasible solutions whereas
if you rounded as I showed before you
will be telling the solver only to care
for X 1 and y equals 0 which in most
cases is not what you really want it so
this is very easy to avoid and and and
one thing to look out for errors is
whenever you're feeling the numbers in
model be sure to use the full precision
allowed by the solver which is 16 digits
that's the standard precision I'm going
to get into that and actually probably a
good question to ask yourself well why 2
times 10 to the minus 16 what's so
special about that number why this
global use disregard those numbers and
that brings us to the next
that the real numbers are not really
real at least not in the computer or not
the way computers handle these numbers
so that's what I mean
and there is an easy check for that so
you can open up your Python interpreter
and type that expression and you will
see that you will get a true and look at
it and you say now that that cannot be
true but that's what the computer is
saying well you may think this is coming
from this is really tiny number so maybe
it's not so wrong to assume that is that
expression is almost one but surely it
is huge numbers that shouldn't happen
but that's not true either so if you do
these other experiments right now I have
10 to the 16 plus 1 I compare it to 10
to the 16 again Python will say that's
true that's really strange is those are
not really small numbers to make a long
story short this has nothing to do with
Python really this will be truly tested
in Excel in our in C in Java whatever
programming language that you use if you
use the standard provided floating-point
numbers underneath these expressions
will always hold true and it get worse
really so given these limitations I mean
these kind of comparisons then not even
the basic algebra rules that we learn in
school are true anymore so not even
associativity like here so if I add one
plus the tiny quantity and then again
the tiny quantity that's not the same as
first adding these two tiny quantities
and then one this is not people
so really numbers storing the computer
using they'd standard hardware it's not
what we expected or what we learned to
do with numbers in high school and I'm
sure there are ways around it just to
mention some of them gene UMP is one way
around it there are other tools but all
these tools rely on software base
implementations or handing out numbers
and
those can be very slow and I'm going to
be talking a bit about that and actually
you can expect where these limits come
from in Python you can say import system
and then print the floating information
and you'll see there is this quantity
down here epsilon that is 2 on all those
and the numbers times 10 to the minus 16
and what's that epsilon that's the
smallest number that you can add to one
and still distinguish it from one
anything below that and one won't be
able to register in the computer numbers
and we use and as I said there is no way
around this and that's what I mean by
real numbers none being real at least
not in the computer so that really means
that we have to be very careful on how
we do arithmetic cela computer now let
me explain a bit more about this so what
does this mean most peripherals provide
a cheap base implementation of these
floating-point numbers these kind of
approximation of real numbers and this
hardware is really really key to get
good performance I mean this using the
hardware base floating-point numbers
it's 10 100 and sometimes even a
thousand times faster then you use
seeing any of the standard extended
precision around there so if you really
care about performance you should deal
with these numbers under limit and
that's what GU Ruby does we use in
almost all code these double precision
64-bit numbers and this is not too bad
actually I'm going to show you that
although these numbers are limits if use
within reason they do provide meaningful
answers 99.999 percent of the time there
is also this last part which is very key
since these numbers don't really satisfy
so should he be adding up is not really
an exact operation
whenever you're testing if I mean
while their own equation holds you
cannot really test for equality of the
expressions you have to take into
account that there might be some
rounding error going on and what do we
do to handle that
well that's Torrance's and user scale
and this is key to understand so what
will guru be do we always as I said
before when we were talking about these
rounding issues we always trust numbers
that you keep so for example if somehow
in your model you write 1 plus 10 to the
minus 7 times X less than or equal to 1
with X being binary results may detect
this expression because it couldn't be
hidden and decided well that implies
that X must be 0 which is a correct
implication from what the input that you
gave the server is however I have to
make an emphasis that this may happen
and why may happen and it doesn't happen
all the time well because during resolve
or even by cutting planes we have to
limit how much time we invest in trying
to detect this kind of stuff so not all
the time we will be performing an
exhaustive search and all the
implications hidden in the model and
that's why it may be comforting to this
however when you solve the model and you
get a feasible solution you have to
check if that solution is feasible but
as I said before you cannot really check
for equality on the constraint and you
need to introduce the idea of tolerances
so whenever we have a feasible solution
produced by any race they go by solving
a linear system there are at least three
things that we have to check one is
deciding if a given solution is integer
or not and what we do is that we define
this tolerance called the inter-facility
tolerance to decide whether a given
quantity is interior enough the same
applies to
I'm a feasibility whenever we have to
check that an inequality or an equation
actually is satisfied we don't test for
equality here we always allow a little
wiggle room to say and again whenever
you are checking for a certificate of
dual optimality the same happens the
dual space and that is called the
optimality tolerance and those are three
different quantities that actually you
can move a bit so well does this mean
and this is very subtle so let me try to
be very careful with what I say first
the tolerances that I was describing for
there are absolute quantities they won't
change depending on the input numbers
that you give so if you change the scale
of a constrain or if a variable the
meaning in terms of these Torrance's we
change and you have to be careful about
that I'm going to talk a bit more about
it it also means that you may have a
feasible solution that satisfied the set
tolerance that I said before but even
having a solution that violates use
let's say by 10 to the minus 7 a
constrained pre-sold or even a cutting
plane may decide at some intermediate
state on the algorithm to cut it off and
to mark that solution as infeasible and
why that can be troublesome well because
it means that at least for models where
the numerix are challenging the optimal
value may change depending on the part
of the argument age so essentially what
this means is that these tolerances
especially for problems were is hard to
get accurate solutions imply that there
is a gray zone
there is a grey zone of solutions that
may be considered feasible in some
situations but in some other situations
may be considered infeasible where as
whatever is strictly feasible will
remain feasible always and then outside
that
torence region then things will be
considered infeasible so whenever the
problem has a big gray area that's the
case where you're observing numerical
issues with your mother so how bad is
really this because I mean up to now
this is sounding really really bad
however people use optimization very
often and it's not that common for them
to get into issues so how come there
must be something going on in here and
indeed there is
so first the fault tolerance that we use
in grub is 10 to the minus 6 for primal
and dual feasibility and for integral it
is 10 to the minus 5 and let's say that
you defined your model and your right
hand size are in the order of 10 to the
4 10 to the 3 it means that the gray so
the area where you may have some
solution to be treated sometimes
feasible and sometimes is infeasible
it's really tiny is use around 10 to the
minus 9 or 1 in a billion so that's
really really really small and for most
problems
that's just insignificant so that gray
area doesn't play any role and that's
why I said that the scaling is key
because you are the one who choose how
big is going to be the coefficients your
constraint or how big are the ranges in
your variables also this means since the
tolerance as I said before are absolute
and not relative if you have constraints
with very small active ranges the solver
may consider them juice to be an
equation and the changes that you're
expecting to see let's say if the
solutions moving the range of 10 to the
minus 9 and 10 to the minus 10 then that
is just not being seen by the solver so
you should also try to avoid
ranges for constraint are just too small
unless they are really equations
okay so what about variables because
this was talking about constraints so
the same really applies to variables
ranges for variables shouldn't also be
to be for the same reason because the
bounds are again primal constraints
although we treat them differently you
should also be careful not to have wide
range of primal variables and also for
interior variables I mean if you have an
entire variable with really wide ranges
let's say I'm million maybe you should
reconsider treating them as a linear
maybe their use continues especially if
that's kind of the range where they move
I mean just a unit change here in the
order of the millions won't change much
but you will be paying the price of
forcing this over to work harder to find
kind of an intercession for those ranges
what about the objective function so
again the same applies because remember
whenever you are looking at feasible
solutions you hold so how to check the
duality conditions really stay that the
giving help resolve is optimal so what I
recommend is to deal your problem so
that the optimal solution is in the
range of either minus 10 to the 4 or 10
to the 4 otherwise what you're doing is
really pushing the dual feasibility so
let's say you're talking about billions
of dollars a good idea will be to change
the units in your elective function to
talk about thousands or maybe 10
thousands of dollars instead of I know
using iron dollars or cents which will
make the objective function blow up and
make everything really harder also for
the same reasons that you don't want to
have constraints with very small ranges
you should also try to avoid objective
functions that are really tiny so if
your problem does have a meaningful
objective function what I will recommend
is to either make it strictly bigger
than 1 or relieve all of that minus 1
and
is to move away from zero again because
then there is a better handling of the
coefficients on the dual side and the
only exception for this is when your
problem is really a feasibility problem
in which case all the coefficients in
the objective can be zero and that won't
hurt performance at all actually and not
all these is not too hard to achieve
it's really just changing the scaling in
your roses change essentially changing
the units on how you measure the DVDs in
your rows and also in the columns and
also in objective function however as I
said before Ruby will not do these for
you
it's after the user really to define
what's the right range for all these
quantities so how did you get tight
ranges for constraints and variables
probably the first thing that you should
try is to use other information not
available in the model to impose tighter
bounds either in constraints or in
variables so let's say I know I'm trying
to plan providing a service of dispatch
and delivery I'm in principle I could
have any quantity of trucks however I
know because I know the business that I
am only considering three possible
locations to store the tracks and those
locations won't handle more than a
hundred trucks so you can give that
piece of information to the solver and
the server will use that information to
really prune both the search and make
everything much faster and if you think
about it a little bit if you don't give
that information solver the solver has
to consider the possibility of going
beyond although you won't have any use
for that solution if it's relevant so
the first thing and most important is
try to get whatever extra information
you have at hand to get tied bounds on
your variables after that
change units from towns 2,000 tons let's
say or from dollars to thousands of
dollars all the time try to choose the
proper units so that the range of
constraints and variables is not too
large were too large I mean hopefully
not beyond 10 to the 4 and finally there
is a common use case on where you can
get very big and small coefficients in
the objective function and that's what
people do when they are really trying to
do hierarchical multi objective
optimization they will blend all these
different objectives with huge
coefficients for the most important
objectives and then with smaller
coefficients for the less important
objectives so that will naturally two
huge clear pitch in the objective
function however now Grobe supports
internally this multi objective
optimization where each objective now
can have reasonable values reasonable
coefficients and everything believe me
will be much easier to follow so let me
try to stress why scaling is relevant
what I'm going to do is an experiment a
quick experiment actually so what I'm
going to take is like a problem and then
I'm going to randomly rescale the
columns of the problems so that the
optimal value in theory doesn't change
so I'm going to change the objective and
the coefficients here and then decrease
the bounds if they are finite okay I'm
going to do that within our range
coefficient and I'm going to run this
script you'll get this script so you can
look at it later on
this is our common problem and this
means that I'm going to scale the
columns with a factor up to of one that
it could be smaller randomly and then I
let grow be optimized the resulting
model and what you see is right away is
you get this warning I mean the
coefficients are to be actually here you
see here Tina's order of magnitude
difference between the smallest and the
largest
that's hardly nards
but Robbie goes on work a bit and he
finds the solution about a thousand
iterations and the value is 4.49 III
something like that okay it wasn't too
bad but if I now scale the problem with
a bigger coefficient so up to a thousand
well you see now the matrix coefficient
range and it grew now it's 18 orders of
magnitude you see again the warning and
now we are still getting the same
optimal value however and this is key it
took longer in numbers of iterations and
what if I push it even more well I can
push it now to let's say a million
I mean scaling coefficients for up to a
million the matrix range is really crazy
now Robbie will do sting and still is
able to get the optimal value to longer
but now you're getting these extra
warning what is saying is saying that
although he tried really hard to manage
violations when he unscaled
or when he finished the process of doing
depreciation he couldn't find a solution
with a small primer or do violations so
in this case it's really extreme still
got the objective function right took
longer in number of iterations but
couldn't find a nice solution with small
violations and it's no surprise I mean
you are scaling for up to a million what
if I push it even further well if I go
all the way to 100 millions or rescaling
columns now the problems really break
down so Rob is not able to solve them
all the carry to be invisible and this
has to do
that decided Roby we trust the numbers
that you were giving so in theory if the
numbers were pure scaling columns
shouldn't change anything
however since numbers are storing this
finite precision being when you rescale
the problem you're really changing the
definition of it and you are risking
even getting unfeasible solutions and
also please remember I mean as we were
increasing the scaling range essentially
making the problem harder to solve the
first time scrubby was able to solve
economy anyway but but the price that
you pay for that it's extra computing
time
so again rescaled your problem may
actually result in a faster optimization
so what else can you do to handle these
issues so one thing that you can do is
actually do a combination of row and
column scaling to get better
coordination so let me explain this with
an example so let's say that I have this
problem I have 10 to the minus 7 X plus
y and X + 3 Z less than or equal to 10
to the 4
what I can do is replace X with 10 to
the 4 X Prime and then I get this
constraint on top but later on what I
can also do I mean I can simplify this
constraint and instead of having 10 to
the 4 X prime here and then to the 3 Z
and then to before here I have this
simpler expression so by combining I
move from a world where I had 10 order
of magnitudes of difference in the
coefficient matrix to only 4 which is
better but not only that I mean since
the precision and the tolerance is
absolute as I was saying before actually
solving the system with this constraint
with scale it down constraint it's
easier than solving the same problem but
with the scale up constraint and why is
that it's because I will still be asking
for a 10 to the
- seeks tolerance in this case whereas
if I ask for a 10 to the -6 tolerance in
this constraint it will be in a sense
easier for the solver to find such X
Prime and Z don't lie to yourself so a
common guideline that we provide is the
range of coefficients in the rows and
colors should be smaller or not to be so
if you encounter this problem it's not
too hard to say oh where I can hide that
six orders of magnitude just having
these equations now everything looks
nice isn't it now instead of having 1
and 10 to the 6 I only have 1 and 10
however you didn't really change the
problem actually the same kind of
epsilon some feasible solutions still
persist in the model and actually Prasad
will detect this and we'll get back into
this original stuff so you're not
changing the problem by doing this trick
what you couldn't do instead of that is
do this so you can change Y by Y prime
where they are related in this form and
now what you will get is essentially if
you push these three orders of
magnitudes here in this coefficient and
you increase now the range of the
variable but the side gain from these is
that now if Y prime is epsilon violative
what you will get in X is really a very
small possible violation of only 10 to
the minus 3 which is really better so
this is improving the numeric I'm the
problem whereas the previous
substitution is not really helping what
about Big M constraints so in principle
is you rescale the meaning of your
constraint of your variables there
shouldn't be any big amps I mean because
the range of the variables will be
willing 10 to the 4 and then everything
will work fine however for some reason
you cannot really get away from those
big constants like here and you cannot
change that
with a by changing this scaling on the
meaning of X
what you can do that will improve
numerical accuracy is to define these
constraints as SOS constraints or or as
a general implication constraints which
was introduced in Robeson point five
however although you will get more
reliable solutions and actually this
implication of Y being zero then X being
0 will be always satisfied the price
that you will be paying is that extra
computing time so it will fix the
problem of numerical stability but the
price you will be paying is more
computing time so don't forget that
so in summary kind of the quick
takeaways variables shouldn't be in
general an arranged outside minus 10 to
the 4 and 10 to the 4 also the same
applies for the constraints the active
range of the constraints meaning where
the solutions really move and also the
same apply for the good solutions
it could solution shouldn't be too far
away from that range either also you
should not expect or try to compare
quantities that have two different
absolute values I mean if you try to
compare 10 to the minus 10 week one and
those quantities were computed true
linear algebra you're really asking for
trouble avoid those things and actually
if you change these in your current
models and you might get surprised
because it may give you a boost in
performance and and why is that well
because grow B will try really hard to
whenever we find numerical issues to
solve it by ourselves either by going
even to extra precision or by changing
the algorithm internally all bar turning
off some parts of the code that in
general help to solve it quicker and so
on and so forth so really selecting the
right ranges for everything may be
really key to even improve the
performance of your core amount so give
me the try
so how do you check if your given model
has some wreckage which is an important
question how we go about that so
probably the first thing to do is try to
separate whatever you're doing angel
code from whatever solver is doing and
an easy way to do that is to use this
parameter guru up our dump equal one if
you do that just before calling optimize
then optimize will generate two files
one is called guru B dot R EU and the
other is guru B dot PRM the first one is
an anonymized version of your model and
the second one is the set of all the
options non default values that you're
using just when you enter the
optimization column this is key because
essentially those parameters will manage
and affect which algorithms are used and
how they're used so to really be able to
reproduce whatever you're seeing you
need these two files so after getting
those what you can do it let's say try
to read the model back in Python and
inspect so you can do that easily by
printing the stats and that will give
you the matrix range the objective
coefficient range and everything and
that will let you see if the ranges of
these coefficients are within reason or
not also if everything looks fine there
you should read them into the model the
parameters that you were using and after
reading those parameters you can call
optimize and you definitely want to look
at the outer and the grubby outputs are
designed to provide as much information
as possible we will output in different
situations many warning messages the
most common one is these warning that
the matrix provisions looks bad or the
objective coefficients or whatever also
you see this warning mark with Torrance
tied into some given value that's really
signalling that simply is struggling to
find stable solutions and it's usually
indication
model indeed is having to medical issues
for sure if you see one and switching to
quad precision it means that Grobe not
only couldn't manage by changing the
algorithm but had to try on working on
extended floating-point precision it
talked a little bit about that later on
and of course you see numerical error or
numerical trouble encounter if you see
restart crossover that means that now
the barrier cruiser is having issues
with your model and suboptimal
termination is the same warning
variables drop from the basis is use
another warning saying that this simply
is struggling with your problem
landscape primal-dual violation all that
comes again from working in your
original model it means that Grubb is
not able to satisfy the tolerances that
you asked for for this given problem so
all those are signal strong signals that
you should think about looking in detail
to what's going on
also you look into the quality of the
solutions so you can always preen
what's the worst bound being violated
what's the worst constraint being
violated and what's the word integrity
condition being violated so in this
particular example bounds are very okay
ten to the minus eight that's pretty
good however in this problem we are
seeing a constraint violation of 10 to
the minus 4 and actually almost 10 to
the minus 3 that's really bad however
the integral it is I looks fine so these
will be signalling that my solution is
not really conforming and in and that
probably Robin is struggling to find a
solution confirming recurrence
you can also inspect the condition
number and I'm talking about that in a
little bit essentially the condition
number is also an indication actually if
it's really be that the solver may be
encountering numerical issues in your
model other symptoms so whenever you see
that I changed some parameter and now my
solution changes the problem goes from
infeasible to feasible or the optimal
the value changes all these are strong
symptoms showing that your problem has
numerical issues now most of these
problems are usually just America
meaning that they are not truly boxed in
Krabi would have a few but it's really
rare most of the time please believe me
come from numerical problems reading the
model one way to see if that's the case
is for example trying to solve your
model with tightening Torrance's let's
instead of asking 10 to the minus 6 you
may try 10 to the minus 8 it now the
solver becomes stable for the solutions
or with the answer that's a strong
indication that your problem is lacking
in good scaling and beware although
tidal torrents may solve the problem in
terms of stability again you will be
paying by extra numerical computation so
you are paying the price on running time
don't forget that now if you already
improve everything the scales look nice
and you're still getting strange answers
well I honestly paid and call us I mean
that's what we are for so that was about
detecting or assessing it
do you have numerical issues so what can
you do once you see that you have
numerical issues and there are some
parameters within Robbie that help
manage issues but again this will not
solve them it will just try to walk
around them and that's as I said many
times now you will be paying in running
time so let's go with this resolved
resolved can sometimes introduce some
issues one way to look at it is your if
your problem you now load the pre-salt
version and you can print the stats of
this Persol model if that start shows
that now your coefficients are looking
really strange then maybe resolve is the
culprit which is doesn't happen that
often but it may
if that's the case you can try to read
off all together with resolve equals
zero or you can say okay
aggregation shouldn't be used
aggregation essentially what we'll do is
remember that bad example of don't lie
to yourself of hiding ten to the six
with six constraints what aggregate will
essentially reweave those stuff and
we'll show implicit B coefficients back
into the models so you disable it
altogether
that won't show up in the constraint but
remember you still have those big
numbers whenever you're solving will be
anyway so be careful all you can also
disable the aggregation of Finland
essentially Grobe will try to strengthen
constraints and sometimes while doing so
may introduce kind of ugly coefficients
if that's the case then you can try by
disabling this piece and see if that
helps performance also think about using
the right in general barrier is the
fastest in general not always however
unfortunately
barri it's also more sensible to
numerical issues so you should try
either primal dual simplex and bar by
itself and see if that helps to get more
robust solutions for your model so it's
always worth a try
selecting the root node method you can
also try to make the algorithms less
sensitive to medical issues
the easiest way probably is to use the
parameter called numeric focus by
default is set to minus one which means
we Robbie will handle it internally and
decide what to do you can disable it
altogether setting it to zero and you
can increase it all the way to one two
and three what Robbie will do depending
on these levels is to be more
conservative on the Markowitz torrents
which is handling how this impacts
people's first like there he will also
select when to switch to quad precision
it will also limit gamma ray passes
which in some cases especially for an
American challenging problems may
introduce issues he married this the
number of
passes that we do on aggregation and so
on so this is a common thing for people
to try when they're encountering
numerical issues to change the Murray
focus to either one or even three you
can also try to change the way simplex
do its pricing by selecting different
norm adjust values if barrier is having
trouble you may try to use borrow
manginas parameter if also you can try a
different strategy for the second across
over babys and you can also try to
disable cuts altogether sometimes again
for numerically challenging problems
they may introduce more numerical issues
into your problem again you have to
remember all these is not really fixed
for the numerical issues this is a work
around them the real way to fix them is
really going back into the model and
trying to change how you pose it so this
is the last piece I have for this
presentation and it has to do with the
condition number so up to now what I say
these either problems come from rounding
input I mean not giving enough detail or
because essentially the limits of the
computer numbers imply that not
everything can be done in exact or
Eataly but not all the problems really
come from that some problems really come
from the geometry and that has to do
with the condition number what I was
showing before as Kappa so the usual
definition is while the condition number
of an optimization problem is trying to
capture the idea of how sensitive your
optimization problem is or the optimal
solution of it is to small changes in
different parts of the input a common
thing that is used as a reference for
this number for linear system at least
is known as Cavafy which is defined as
the norm of the matrix time the norm of
the inverse of the matrix but I
prefer really the geometry
interpretation which has to do with how
rounded the feasible region is and
provided sets this is really simple
essentially what you can try to imagine
is what's the ratio between the smallest
ball containing the feasible solution
and the largest ball contained in the
feasible solution so let me give you two
examples so you may think okay I can
work in optimizing something like behind
with many many many many linear
inequalities at the border so in this
situation the geometry of the condition
number will be the ratio between that
slightly gray circle outside and the
darker gray circle inside and that will
be fairly close to one and what the
condition number says is that well if
that ratio is small then small changes
in the input data should produce small
changes in the outer solution and indeed
that's the case so I have a script here
you'll get it later on that does exactly
that I define a certain weight I don't
know in this case a thousand
inequalities to find the circle and then
I move a little bit here and there some
coefficients and reoptimize and see how
much the primary smidgen changes and if
I do that I get these so out of 500
iterations changing small coefficients
on the order of ten to the minus six
only change the primary solution about
in the same order of magnitude one point
seven ten to the minus six so this
really chose that small changes as the
geometry idea was wind resolving small
changes in the optimal solution which
makes sense so these problems are easy
to solve but what if I try to optimize
over a very thin triangle like here so
if you imagine if I stretch this
triangle more and more the circle thing
we delete will get smaller and smaller
whereas a circuit that contains it
we'll grow more and more and the ratio
will get to be pretty bad and what I can
do is essentially define this
optimization problem tree constrain
solve it and then see what happen if I
change a little bit a little tiny bit
the objective function if I do that
right away I mean even changes in the
objective functions in the order of 10
to the minus 6 translate themselves into
changes in the primal solution of 16
order of magnitude and this is not
because it's always getting the wrong
answer actually these three numbers are
showing how big the primal-dual and all
the other bound violations are which is
essentially zero so it's always getting
the right solution for this horrible
problem the problem is that if you
change the tiny major objective function
you really may get very different
solutions from the primal side and this
is not to do with numerical issues is
really to do with the geometry which
again take us back to scaling
essentially if you have a good scaling a
problem that looks more round or square
if you wish then everything should
behave much better whereas if you have
very thin areas you are exposing
yourself to have being very unstable
solutions so remember that ok recap to
end up essentially in stability it's not
always coming from the numbers it also
comes from the geometry Robbie will
never second-guess your input will
really trust it and the best way to
handle these numerical issues is most of
the time think hard and rescale
everything and maybe even change the way
you formulate the problem
thank for your attention your time