so welcome everyone to yon Lukas plenary
lecture today it's really my pleasure to
welcome young and before I'll hand the
word over to him I'll say a few words
about his power his biography young Luca
is vice president and chief AI
scientists at Facebook and silver
professor at NYU affiliated with the
grant Institute of mathematical sciences
and to Center for data science he was
the founding director of Facebook AI
research end of the NYU Center for data
science
he received an engineering diploma from
a call superior for electronic
engineering in Paris and a PhD from
Sorbonne University after postdoc in
Toronto he joined AT&T Bell Labs in 1988
and AT&T Labs in 1996 as head of image
processing research
he joined NYU as a professor in 2003 and
Facebook in 2013 his interests include
AI machine learning computer perception
robotics and computational neuroscience
he's a member of the National Academy of
Engineering and a recipient of the 2018
ACM Turing award together with Geoffrey
Hinton and Joshua Ben geo for conceptual
and engineering breakthroughs that have
made deep neural networks a critical
component of computing today
Yann will speak about the deep learning
applied math connection and I very much
look forward to your talk young but
before I hand over to you and let me
just remind everyone that you can submit
your questions in the chat or the Q&A
box throughout the talk and questions
will then be handled at the end by
Geetha so with that I hand over to you
young thank you very much thank you very
much for the introduction and thank you
to Joel and the organizers for for
inviting me
I'm really glad that I can do this in
real time I prefer this to having to
record a talk in advance okay so I'm
going to talk about deep running
obviously and I've been trying for many
years to try to sort of connect the
applied math community to this to this
problem and it didn't happen until deep
learning sort of became very prominent
but now it's really happened okay so as
you probably all aware the the main
workhorse of machine learning and deep
learning at the moment and AI really is
supervised running where you train a
machine with examples and you heat the
machine with the correct answer and
machine learning consists essentially of
minimizing an energy function that our
last function an objective function I
should say that averaged over the
training samples minimizes the error
measuring measured in some way between
the output you want and yet what you get
and this works amazingly well for speech
recognition for image recognition for
even translation and things of that type
but it requires a lot of samples and you
know it seems that humans and animals
are able to learn with considerably
fewer samples so people have worked on
things like transfer running and things
like this but but none of those
solutions really kind of approach the
efficiency that we see in humans and
animals
so as you probably all know supervised
running but also all other forms of
running generally use a form of gradient
descent called stochastic gradient
descent which consists in basically
having a very noisy estimate of the
objective function and it's gradient
based on either a single sample or a
small number of samples and not
averaging over so that rolls out a lot
of sophisticated optimization methods
that people in this community and the
math community have worked on for many
decades and the property is the
convergence property of sarcastic
gradient of course have been studied
extensively in the context of adaptive
filters but also in the context of sort
of large-scale machine learning so this
is now the workhorse stochastic gradient
descent nobody even considers anything
else but they do consider refinements of
it where
you try to exploit the the the structure
of the objective function to define the
minimum now deep learning in this
business is the idea of essentially
building at least a simple form of deep
running is the idea building a learning
system by stacking multiple layers of
parameterize nonlinear functions
essentially a function of modules and
it's easy to compute the gradient of the
objective with respect to other
parameters in a chain like this by
basically applying chain rule as I'm
sure you're all aware of this is what in
neural net and machine learning people
call that propagation so a classical
neural net is really a sequence of
alternated linear and nonlinear
operation the linear operation is just a
matrix multiplication essentially or
perhaps a sparse matrix and then the
nonlinear operation is a point wise
non-linearity very often one of those
what people call real you which is
really a half wave rectifier so I see
identity when the argument is positive
and equal to zero and the argument is
negative and by stacking linear and
nonlinear operations of this type
multiple times you get a fairly powerful
parameterization of a lot of you know a
family of a lot of useful functions
so there are theorems I show that with
only two layers you can approximate any
function you want to score that you want
and so the question the theories might
ask is why would you need multiple
layers and it took a very long time for
some of us to convince the community
that actually indeed multiple layers
were useful the reason being that the
the world is essentially compositional
so a lot of the data that we're trying
to process with this bit images or other
sound or other natural signals tend to
be compositional they have a lot of
local correlations and so here
composition composition no signal there
is an advantage instead of processing
processing it hierarchically but
progressively extracting more and more
abstract representations of that signal
so that's the kind of sort of you know
conceptual philosophical necessity for
layers there's a more immediate one
which is that the family of functions
that seem to be interesting for us to
learn seems to be more efficiently
represented by multiple layers and by
just two layers even though you can
represent them with two layers most
functions require a ridiculously large
of nonlinearities in the middle layer
and so it's really impractical for
things like say image or speech
recognition you really need multiple
layers and what I've been puzzled by is
where it took so long for people to
really sort of come to terms with this
so one reason is that people in sort of
machine learning and optimization were
really sort of upset by the fact that
when you stack multiple layers the
objective function that becomes non
convex and of course there is no proof
of convergence we have non convex
objective function so a lot of people
objected to the fact that you don't know
what you're doing here optimizing a non
convex function now experimentally it's
been known for for decades that there is
essentially no problem of being stuck in
local minima in in in in multi-layer
neural nets as long as where they are
bigger than necessary for the problem at
hand so if you make the network much
bigger than necessary you know in the
end of function may require way fewer
elements than you put in your network
but if the network is much bigger than
necessary then the optimization problem
becomes relatively easy and your system
never gets stuck into a local minimum
because the or very rarely because the
the space of solutions is actually a big
chunk of the overall ambient space of
parameters and it's very easy to find a
local minimum the the mini Marv Airy
degenerate and so it becomes easier to
find it now when you tell this now to
start stations to tell you oh but that's
an abomination because now you're using
an enormous model with lots of
parameters you have absolutely no you
know no evidence that your system will
work outside of sample we generalize
because it's so over parametrized and
there something else happened which is a
bit of a theoretical mystery and seems a
bit of a bit like magic which is that
those systems
despite being widely over parameterize
seem to sorry seem to generalize
regardless and so you may have neural
nets with you know hundreds of millions
billions even hundreds of billions of
parameters and the and
generalizing you know fine with you know
where fewer training samples than they
have parameters so that's a bit of a
mystery I think there's a lot of very
interesting work a theoretical work that
sort of tries to explain this phenomenon
in terms of some sort of implicit
regularization so it seems that when you
stack linear layers ignoring the
nonlinearities for moment you seem to
get sort of a natural with reduction of
the rank of the activations in the
intermediate layers and there was a
initial paper by another F Cohen and
Sanjeev Arora about this and sort of a
follower paper bye-bye Nadav kind of
showing that there's some sort of
automatic rank reduction that might
explain some sort of implicit
regularization that occurs naturally as
a consequence of gradient descent
running there's other papers that sort
of get pretty much the same results in
sort of different ways so that's a bit
of a theoretical mystery but which I
think you know there's a lot to bring to
the table here from the applied math
community as well as from the
statistical physics community which in
the 90s was very interested in this
problem and sort of dropped at but now
they're coming back to it so I think
it's a fascinating set of developments
so back propagation is just a practical
application of chain rule for you know
multivariate functions essentially and
it says you know if you want to compute
the gradient of some objective function
with respect to some variables you just
multiply the jacobians of all of those
boxes with respect to their inputs and
parameters going backwards in the flow
of computation now this concept is
surprisingly powerful I'm sure a lot of
you are I've used automatic
differentiations in the past and this is
basically just like automatic
differentiation and in fact the root of
that algorithm can be traced back to
classical work in optimal control in the
50s and 60s where you know people use
you know the kilobytes an algorithm or
the country again extremum principle
which basically you know for channel
control treats this problem of kind of a
stack of functional modules that you
have to compute gradients over this is
called the adjoint stay tomorrow
in optimal control so the idea is very
old of course you know general even in
multi dimension is very old but the
realization that you can use this to
train the system through grid and decide
actually did not really pop up until
1980s so that said people have sort of
generalized this concept the the type of
architectures that people use nowadays
are not just chains of linear and
alternative linear and non-linear
point-wise nonlinearities it's more
they're more programs now so the the
type of deep learning systems the the
modern ones are basically you could
consist in writing a program a function
using a framework like Python or
tensorflow
let's say pi torch and what the system
gives you for free is the ability to
compute the gradient of the output of
that function with respect to serve
pretty much anything that goes on inside
so this is a very very powerful concept
which is sort of a generalization of
automatic differentiation if you want
it's very flexible and you know it runs
on GP using parallel don't all that
stuff so because this has been a huge
demand for this and so you can basically
write your model you know writing it was
basically a Python program and
automatically you get to differentiate
this with respect to any variable you
want and so that can be used for machine
learning can be used for tml control it
can be useful numerical analysis is
becoming a tool that a lot of
communities now are using because of
such interest in in deep learning so
that has come to be known as
differentiable programming you can think
of this more than just deep learning
it's you know programming in such a way
that there is support for automatic
differentiation in your programming
language okay so of course very often
you have to put some structure into the
architecture of the network for you to
process certain types of data
appropriately and there is the idea
accomplished on that that my name is
attached to which basically replaces the
full matrix of the linear operations by
convolutions which are another type of
linear operators with fewer parameters
so basically each layer is a is a filter
bank essentially
and then you're going to a non-linearity
and then there is another kind of
operation computing which basically is
subsampling
with some aggregation world which could
be an average or an LP norm or a max
it's properly properly in its most of
the time now it's called max the root of
the idea of this goes back to classic
work in neuroscience by Hubel and Wiesel
from it was Nobel Prize winning on the
architecture of the visual cortex and
it's really there really was inspiration
from neuroscience in other you can so
post hoc justify mathematically the
reason for why this should should be
done this way so comrades have been
incredibly successful for a lot of
applications you know every single
automated vision systems four four four
cars today you know the kids in lane and
detects pedestrians and brakes
automatically is based it's based on
commercial Nets most speech recognition
systems are better than commercial nets
most new medical image analysis systems
and image restoration systems are based
on complex on light so there is a you
know huge amount of applications for
this that have been deployed over the
last few years and the results in
computer vision are just amazing now you
know but you know back in 2012 people
could so barely with the best systems
you know recognized the dominant object
in an image with you know far from from
from perfect reliability and now what we
can do is basically train those neural
nets to essentially label every pixel in
an image with the object that it belongs
to right so all those pixels are labeled
as a person this one is a bicycle the
bottle at the bottom year is detected
that's another person another person etc
the persons are included they are still
detected and this is the idea for
panoptic segmentation we're not just
objects but also background areas are
are labeled like tree sky road etc and
this works really well with you know
hundreds of categories sometimes
thousands some at unity you can do pose
estimation of bodies and all of those
basically use the same trunk the same
backbone for the neural net with sort of
different heads that that produced the
different results so Facebook has a
system like this called electron which
is open source so you can just download
it and play with it it's free it's pre
train you can train it if you want of
course in very successfully medical
image analysis and I think the user
community in vehicle image image
analysis is it's so promising that it's
you know basically dominates a lot of
current conferences on on medical
imaging those networks use a very
interesting architecture that's
something called unit which includes a
accomplish on that that sort of
represents images hierarchically by
progressively reducing the spatial
resolution and increasing the number of
feature types and then there is sort of
a converse architecture which you could
call the decomp which on that that sort
of regenerates an image of the same
resolution as the input and this is the
kind of thing you want to use for image
segmentation at high resolution and like
in the case of medical imaging for
example these are pictures from a
collaboration between Facebook a
research and NYU on kind of speeding up
the data collection for MRI data where
you know you can sort of restore the
image with a fourth of the data that
would normally be necessary for the full
resolution image so applications of deep
learning and Congress on this that pop
up every day but I think the ones that
people will be more familiar with is for
autonomous driving or semi autonomous
driving or driving assistance and
medical imaging of course it's already
used a lot in you know online services
for content filtering and things of that
type people tell me that according to
some statistics the automated emergency
braking systems are now mounted in every
car in Europe as it's kind of a standard
feature it's kind of mandated by the
government in certain countries we use
the number of collisions by 40 percent
and that's enormous which means you know
deep running and coalition that save
lives
same for medical imaging now a very
interesting development that some of you
may be aware of is the fact that you can
define convolutions on graphs so if your
data doesn't look does not come to you
in the form of a sort of regular array
through to the 1 D 2 D 3 D 40
if he comes to you in the form of a
function on the graph you can still
define convolutions on graphs so the
idea is a completion of course is a
diagonal operator in Fourier space so if
you can could do before you transform of
your signal then in the Fourier space
you just do a point wise multiplication
with the Fourier transform of a
composition kernel and and then do the
inverse transform and you've done a
convolution now you can define the
Fourier transform in a of a function on
the graph and that basically consists in
transforming the function on the graph
into the eigen space of the graph
laplacian when I talk about this to
computer scientists every two weeks for
anyway all of this is but in applied
math a lot of people know what this what
this means and so you can define this on
on just about any network but of course
it's very expensive because there is no
equivalent of FFT for irregular graphs
so there's a lot of work on what's
called geometry deep learning or graph
neural Nets that consists in sort of
finding efficient ways to do the
equivalent of convolutions on functions
on a regular graph essentially sometimes
even dynamic graph so graphs that you
know you have regular graphs in which
you use on your commnets you have
irregular grass with a fixed structure
in which you can use those spectral
conditional nets and then you have
irregular graph whose structure changes
with any any new piece of data so for
example if you want to represent a
molecule as a graph a new molecule will
have a new graph so your network's needs
to be able to digest graphs of varying
structure and that's entirely possible
to define last year or couple years ago
I Corgan eyes the workshop at I Pam on
UT planning techniques and there's a
number of talks on this topic if you
really are interested in this I
recommend a series of talks by recent
talks by exactly but I saw who really
has you know given wonderful lectures
about this as well as a recent blog post
by Mickey Bronstein from Imperial
College which i think is on towards data
science blog
now again I think relevant to the
applied math community is that people
are starting to use accomplish on that
as a way to compute efficiently
approximate solutions of believes so PDE
solver you know an integrator is is
basically you know the state of your of
your system at twenty plus one is the
set of the system at time t plus you
know the the time increment times some
function of the previous state and and
you could think of the G function here
as being accomplished on that very often
the interactions are local in a PDE and
so you could think of this as the G
function is some sort of complex on that
and the question people are asking is
can I increase the size of the of the of
the grid that I used to serve my PDE and
both spatial and temporal and get the
same result not by actually solving a
PDE but by training a neural net to
basically predict the result of the
integration for several time steps with
a coarser grain and this seems like a
free lunch but it works and so people
are using this for things like lattice
QCD and in physics for fluid dynamics
predicting the you know hydrodynamic all
properties of solids for example and
perhaps optimizing them through gradient
descent and for cosmology or true as
astrophysics there's a very interesting
work that I mentioned in a bit now
sometimes those those grids are
irregular so we're back to this idea of
using graph convolutional nets where you
know your your your grid being a regular
you sort of have to deal with the
irregularity of it so that's an example
of a paper that appeared a little while
ago in PNAS a group from the the Federal
Institute in New York and they're
interested in simulating the early
moments of the universe at the scale of
the universe and it's essentially
impractical so what they do is they
train accomplish on that on the sub
domain by just running the you know the
equation that that they know how to
solve but on a relatively small domain
that generates training samples and with
this result they train accomplish on
that which is actually one of those
unites architectures that was used for
image segmentation they trained were
those neural nets to predict the result
of the integration for several time
steps and only using information from
coarser grain and once they have the
system is pretty efficient to run so now
they can run a simulation that the full
scale of the entire universe and get
some results that seem qualitatively
interesting there's a lot of really
interesting work in this which i think
is really fascinating so this is sort of
you know we're learning to integrate and
so this is the use of graph neural nets
for analyzing high energy physics
experiments okay okay so now a big
puzzle okay we're still far with our
learning methods from reproducing the
type of running that we observe in
animals and humans and and and the
question is how is it that humans and
animals learn so quickly and I think
it's really a very very important
question because we need to be able to
get machines to learn to let's say drive
a car without running in simulations for
millions of hours we need to be to have
machines are really robust for
performing computer vision or natural
language understanding we need machines
to have some level of common sense to
know how the world works basically
learning by observation and that's the
trick the humans and animals seem to
learn by observation without being
trained to a particular task but
basically to you know learn to
understand the world right so babies
around the age of two months you know
learn concepts like object permanence
the fact that you know I can I can hold
an object if I if I hide it behind me
you know it's still there and you know
babies run this really quickly they
learn whether objects can kind of stay
stable or will fall and they learn
things like gravity around nine months
so if you show a six month baby an
object that seems to float in the air
they're not surprised but if you show
that to a 10 month baby the baby will
look at it like like the world is
crumbling a little bit like this little
girl here and and the reason is that in
the mean time around nine months babies
learned that objects are not supported
are supposed to fall
tell sand that nobody gives them the
name of anything they just run this by
observation and they possibly learned
this by sort of training themselves to
predict okay so the rhesus hypothesis
has been around for a very long time
which is that what the brain essentially
does is a trend itself to predict all
the stuff that it cannot currently
observe and that it may be able to
observe by waiting or by moving so when
I move a little bit objects can move in
different ways and so the best way to
explain how do my view of the world
changes when I move my head is that the
world is filled emotional I don't need
to be
I don't need my brain to be hardwired to
know that the brain is really meant the
the world is really emotional if I move
my head and I predict what the world is
going to look like I will have to infer
the fact that the world is
three-dimensional that's the best
explanation for it so my guess is that
the my slogan if you want is that the
next revolution in machine learning and
AI will not be supervised it will not be
reinforced either in the sense that you
know there is this mode of learning
called reinforcement running it's even
more inefficient and then supervised
learning in many ways but I'm going to
go into this very much so deep learning
works well for perception as long as you
have data deep reinforcement learning
works well with for action generation
but you need way way too many
interactions which is why basically only
works for games where you can simulate
the world it doesn't really work in real
world if you want to train a like a
robot arm to do something you can do
this to some extent by sort of
transferring simulation to to the real
world but it still takes the equivalent
of tens of thousands of years of
training in simulation for a system to
learn to manipulate an object for
example so I think there's really three
challenges that the sort of machine
learning AI deep learning community
should should strive to solve the first
one is running with your label samples
or fewer trials and what I'm going to
propose here is the idea of supervised
learning the the second one is running
to reason so essentially you know
although the classical D planning
systems compute the output by just
propagating through a bunch of layers
but a lot of what we do as humans and a
lot of what some animals do is more
reflective right you you can
think about something before doing it
and that requires sort of you know a
satisfying set of constraints and so
this is a view of reasoning that
basically turns down to or comes down to
optimizing a function right the way you
reason is by satisfying a set of
constraints you can you can measure the
variation of those constraints as a set
of as an objective function an energy
function if you want a new sort of
minimize that energy function while you
do inference so reasoning might might be
optimization the same way learning is
optimization and then there's a third
problem which I'm not going to say
anything about because I have no idea
how to solve it
okay so what is a supervised learning
supervised learning is the idea of
filling the blanks I was giving you the
examples of you know babies kind of
learning to predict what's going to
happen in the world by observation and
this is really a circle the idea of
supervised learning take a piece of
video show a piece of video to the
system and ask it to predict what's
gonna happen next or take a piece of
text remove some of the words in the
middle of the text and ask the system to
predict the words that are missing or I
don't know remove the top of an image
and asked assistant to predict them the
missing part of the image so that's the
idea of supervised running pretend there
is a part of the input you don't know
and predict that now there are two kinds
of it one kind where you know in advance
which part is going to be missing it's
always gonna be the same one like like
video prediction and another one where
you never know which one it will be
that's slightly different so they sued
two users really for supervised running
the first one is running hierarchical
representations of the world where you
know supervised running basically is a
pre training that precedes a phase of
supervised or reinforcement learning
that well train the machine to actually
do a task whereas the pre-training just
trains it to represent the world and the
nice thing about this is that doesn't
require any label data so you can use
basically as much data as you want it's
actually free and then the second one is
to learn predictive models of the world
which would be very useful for model
pretty to be control and for for control
in general so the big technical question
were to solve here and this is again a
question to the applied math community
is how do we represent uncertainty and
multi-modality in the prediction so
here's a
here's here's the problem the problem is
if you're looking at you're looking at
me right now giving this talk and the
next thing I could do is move my head to
the right or move my head to the left
right so let's say you train a system
with initial segments of video of me
talking and you stop the video at some
point and you ask the system what's
going to happen next the system may have
no way of predicting whether I'm gonna
you know say this or or move my head to
the right or to the left which word I'm
gonna say you know how good move my my
lips etc so there is necessarily a
necessity of representing uncertainty in
the prediction you cannot make a point
prediction because the best thing the
system will do if you train a predictive
system like this would least-squares
that it will produce an average of all
the possible features like that can
happen
and so which is not a good prediction
right it's gonna be a blurry version of
overlapping versions of my face right so
so how do you represent uncertainty so
of course if you're a machine learning
person or probably least the first thing
you say was that station you say well
you you know don't run a point learner
distribution and this works in certain
cases so but it doesn't work in
continuous spaces because if you want to
predict frames in the video we basically
have no way of representing good
distributions on in the space of all
video frames this is a problem that we
just not don't know how we just don't
know how to do it and and all the things
that people are tribe is achill II don't
work very well so so one way to deal
with this problem is to abandon the idea
that we have to estimate distributions
and to basically focus on estimating
level sets of a distribution or
representing the log of a distribution
in a normalized manner but an energy
function so this is my favorite approach
which are called energy based models and
people have been you know using this for
proposing this in the past so it is not
necessary my own invention but I've been
pushing this a lot so an energy based
model is an energy function f of X Y
where X is an observed variable and Y is
a variable you want to predict and it
says it's called-- valued and the scalar
value that comes out of it is going to
be low if
x and y are comparable with each other
so if Y is a good prediction for X said
a good prediction full-of-shit your
video where X is the initial segment and
is and the output value is going to be
higher if if Y is not a good
continuation for X not a good completion
of X if you want so the inference
process in an energy based model which
is computing the output consists in
finding a Y which there may be multiple
that minimizes F of XY okay that
produces a low value for f of X Y and
then by D several and of course there
are several energy function that will
give you the same Y's okay there are two
examples here at the bottom two
different energy functions that give you
the same result you know this is X this
is y here so more generally you will
have a space it would be a space of x
and y or just a space of Y if your your
system is unconditional doesn't have an
observation and your data points are
those black dots and what you need to
learn when you train an energy based
model is a function represented by those
contours here that takes low value on
the data and higher values outside if
this function is continuous
differentiable relatively smooth you can
use gradient descent to do inference
right so I'll give you an X and you can
find backward in descent the value of y
that will minimize your energy so this
is basically replacing a feed-forward
function of a neural net by an implicit
function really and in fact you can
recast a lot of classical learning
algorithms in terms of sort of energy
based models so things like let's say
k-means for example so this is running
k-means on a set of samples that are
drawn from this view ellipse here in two
dimension and what is being plotted here
is the energy function the energy
surface after k-means has been trained
so each of those black areas are
basically energy wells centered around
prototypes and but but the way to
represent multi-modality in a energy
based model is probably to do it to do
something like like is shown here where
you take your X variable you run it
through a neural net or some
parameterize function that produces a
representation of
the observation that are called H and
then you can run this to a decoder that
makes a prediction and you can measure
the discrepancy the error the distance
between the prediction you you were
given the desired output you are given
and the prediction that the system
produces y-bar now if there are multiple
Y's that will be compatible with this X
you're only seeing one sample here what
you want is this to represent some sort
of manifold it may not be a manifold it
could be a set or some kind of all the
possible plausible predictions and the
way to parameterize this manifold is
through a latent variable so you have a
little variable Z here that goes also
into the decoder you can vary this
latent variable over a set and as you
vary this latent variable over a set
your prediction varies over this
manifold now again that's an energy
based model and what you can do is
minimize with respect to Z so if I give
you an X and a Y and I ask you what's
the energy of this pair of X Y what you
have to do is find the value of Z that
minimizes the the prediction error you
immediately see a problem which is that
if Z has too much capacity let's imagine
that Z has the same dimension as Y then
there's always going to be a Z that for
any Y is going to produce essentially a
zero reconstruction error imagine that Z
is the same dimension as Y and the
decoder happens to learn the identity
function or something similar to this
then for any Y there's gonna be a Z that
it's going to make the reconstruction
error the prediction error is zero and
so obviously that's not a good model if
you're good if your model you know
ignores the input data and gives you
zero energy everywhere you need a way to
ensure that the energy is small or zero
on the data points but you need a way to
ensure that it's higher outside the data
points and you have two options for
doing this one is called contrasting
methods and yellow one is called
architectural methods so the
architectural methods
consists in is actually limiting the
information capacity of Z but either
minimizing its dimension or the rank of
the the values values of Z or the
training set or by making it sparse or
by making it noisy I mean there's
various ways to do this but that is
essentially what it comes down to you
have to limit the capacity information
capacity using somehow
now inference when you have a little
variable model like like this with Z it
can be done in two ways that when I
explain is minimizing with respect to Z
to eliminate it so this is this another
way if you are probabilities marginalize
over Z and that comes down to basically
computing what physicists call a free
energy which is this little formula here
where beta is some arbitrary policy
parameter and this formula here which is
basically so integration over possible
values of Z reduces to the the one at
the top here when beta goes to infinity
which is the zero temperature limit in
physics so latent variable models and
this is a small illustration of a latent
variable model where imagine that your
data manifold is in the lips and then
the latent variable would be the angle
of a point on the ellipse and then when
I give you a data point why the energy
of that point is the distance to the
closest point on the ellipse so I have
to minimize that distance with respect
to the latent variable which is the
angle to find the energy right very
simple but of course in reality we don't
have this manifold where to run it okay
so how do we train an energy base model
you know two methods as I said
contrastive methods and confessori
methods consists in taking a data point
pushing down on it and then taking your
point outside and pushing up on it or
taking a bunch of points outside there
are many fold and pushing up on them and
that actually works quite well in the
context of image and speech recognition
at the moment but it's extremely
expensive computationally because in
high dimensional spaces you have to push
up on lots and lots and lots of
different places so but that's you know
it's so much successful categorical
method that's very popular at the moment
in computer vision and speech
recognition and then there are this
other class which I think in the long
run will probably win out but at the
moment they're not actually working very
well
those regularize architectural methods
that consists in explicitly regularizing
the capacity of the latent variable so
that you limit information capacity of
it so this is a bit of an eye chart but
it basically lists a lot of classical
machine learning algorithms as to
whether they are contrastive or
regularized or architectural so things
like metric learning siamese networks
which I'm going to talk about in a
minute are contrastive methods
your methods like ganz are this type as
well denoising auto-encoders ambassador
encoder that are incredibly successful
in context and natural language
processing also contrasted so those
really work really well those don't work
nearly as well but in my opinion in
longer and they're more the more
promising and this connects with a lot
of things that artifacts that the
applied math community has been obsessed
with for a very long time
think like sparse coding in sparse
modeling you know it's things like that
and then things are a more kind of
modern in the deep learning community
like variation rotary encoders sparse
auto-encoders vqv AE and things like
this
let me skip this so in contrast methods
there's various ways to define objective
functions and basically the basic design
principle for a good objective function
is is that your objective function will
have a term that's going to push down on
the energy of a data point XY it's going
to have another term that's going to
push up on the energy of some other
point that somehow has been generated to
be outside the manifold of data so this
is an example of a margin loss but there
is like a whole bunch of those lawsuits
that people have used in in various
contexts mostly not in the context of
supervised learning mostly in supervised
learning but but you have a kind of a
wide array of possible objective
functions if you insist on estimating
densities with you with your system so
if you if you insist that your energy
function be the logarithm of some
distribution then the only objective
function you can use is the negative log
likelihood of the data under your model
which is basically a particular type of
objective function that has a term that
pushes them on the energy of data points
and then a horrible intractable integral
that attempts to push up on the energy
of every point including the data points
but with a smaller force and so
probabilistic approaches basically force
you to use a particular type of loss
function that happens to have a
completely intractable term in it which
is where I am big advocate of those
energy based models gives you more
flexibility in the objective function
and you might be able to askew the
issues due to those normalization now
something that's really popular at the
moment is those
called contrasting methods which are
really sort of an application or what's
called Sammis networks and I'm not going
to go into the details of what the
objective function is but and I don't
seem to and here is the slide so people
call this contrast even betting and the
the way you do this is that you you take
a sample you distort this sample in some
way that does not change its nature and
those are your this is your pair x and y
you run those two images through two
identical neural nets they both could
use an H a representation okay and then
you compare those two representations so
those are vectors you just compare them
with some distance a Euclidean distance
or cosine similarity so that you
normalize norms and what you do is that
when the pair of images are semantically
identical then you force those two
vectors to be as close to each other as
possible that's one term in your in your
cost and then you have contrastive term
that where you pick a random sample from
your dataset or maybe not a random
sample but sample you know is different
and you feed those two images and then
you force those those two vectors to
basically move move away from each other
you can use one of those losses I was
talking about earlier people use
currently what's called this nce knows
contrastive estimation you know which is
some normalize exponential of
similarities this works really well so
the idea goes back a long time a paper
of mine from 93 where we used this for
signature verification and this sort of
more recent work I did with my students
in mid-2000 that was largely ignored and
more recently those methods have become
extremely successful for learning
features in images in fact the RISM more
recent paper from japan here that use
uses a somewhat similar idea although a
little different which works even better
and there's a whole crop of papers that
are being prepared that popping up on
archives that use those methods nowadays
let me back up a little bit so one thing
that's been incredibly successful is
contrasting methods in the form of
denoising auto-encoder so what is it
denoising auto-encoder you take a piece
of data why let's say a piece of text or
image let's say a piece of text you
corrupt it in some way so you remove
some of the information or you corrupt
it in the
kids over taxi will remove some of the
words and replace them by a blank marker
and then you run this to some big neuron
that and trendiest neuron that in the
case of NLP it's this neuron that is
usually what's called a transformer
which is a particular type of
architecture and then you minimize the
reconstruction error so you basically
train the system to predict the clean
version of the input only using the
corrupted version where some of the
input has been masked okay so basically
training the system to predict the words
that are missing in your text in the
process of doing so the system runs a
representation of text that actually
contains all the semantic and syntactic
information about about text so if you
see the sentence like the blank in the
house chases the mouse it will easily
infer that the missing word is cat
because it seems you know similar
similar examples before and what it
produces is a probability distribution
for every missing word and that
probability distribution is over all the
possible words in the dictionary so Sal
supervised learning in the context of
text works amazingly well I mean this
method of pre training at a natural
language understanding system is has now
become universal the the basic idea of
it I mean the basic form of this the
special form of this has come up about
two and a half years ago and it
completely revolutionized the way people
do NLP and now it's used absolutely
everywhere and those networks are
gigantic there are you know hundreds of
billions of parameters sometimes at
these billions of parameters so
incredibly successful use of a
supervisor on in there you know you can
you can have dialogue systems now that
almost sound he-man some researchers at
Facebook research in Paris you know in
France for shots no I've actually
trained one of those systems too as if
it were language translation to actually
solve differential equations so
symbolically this is not numerically
this is symbolically right so you give
an equation like this it's represented
as a strain which is turned into kind of
a tree representation in polish notation
to the input of the system so it's it
sees this really as a text and then you
train it by generating example backwards
it's ready to
just integrate this this differential
equations very simply and it works for
computing integrals symbolically solving
differential equations first-order and
second-order it works better than
whatever you know MATLAB people and
Mathematica gives you in many cases
which is amazing they have a similar
system now that is trying to translate a
program from one language to another
you know substituting the appropriate
idioms and and function calls and
everything it's it's it's frankly
completely amazing that it works at all
but it does ok let me skip this in the
interest of time so connecting this with
things that people know in applied math
is this idea of sparse coding ok so here
this is sparse coding space modeling
formality in terms of those energy based
models so you have a little variable
it's regularized by some sort of you
know l1 norm in the case of sparse
coding the decoder is a linear function
just multiplying by a matrix you have a
reconstruction and the reconstruction
error is just a quadratic distance and
the inference process of code consists
in finding the value of Z that minimizes
the reconstruction error and that's
basically sparse coding is fast modeling
consistent learning this matrix by by
gradient descent onto some constraints
on the norm now there is something
interesting about this twist that has
appeared over the last 10 years or so
which is sparse autoencoders and what
people have now come to call amortized
inference so instead of doing the
inference of this variable by doing the
East algorithm that optimizes this
function every time you have a new Y
what you do is that you train a neuron
that to make to predict whether the
solution of the optimization problem is
going to be so instead of solving an
optimization problem you're trying to
neural net to solve that optimization
problem and you hope that this neural
net will give you kind of a good
approximate solution which you might be
fine or not at the end and this form of
what's called regular risotto encoder is
very powerful so you can figure this
latent variable as being optimized or
marginalize over and you can think of
this guy as kind of predicting the
optimal value of the latent variable
there's a form of this column fashion
about two encoder which
and I gotta go into where basically the
disease are not points but there are
like little Gaussian fuzzy balls so you
know sparse coding works you you train
sparse coding on a list and you get you
know currents of the decoding matrix
that look like parts of characters and
if you try on that roll image patch is
you know it runs really quickly that
image patches in the approximated by a
weighted sum of a small number of edges
oriented edges and and you can actually
push the analogy even further so this is
a system that I could list are running
in try to shrink and thresholding
algorithm which consists in basically
viewing the East algorithm as which is
the optimization algorithm to solve this
l1 l2 problem as a recurrent neural net
and you're going to train the matrices
in that recurrent neuron that to give
you a good approximation to the
optimization problem as fast as possible
and you know it's it's amazing but it
works and it seems like a free lunch
like why would a train system give you a
better faster approximate solution than
actually running the optimization
algorithm in the first place and the
reason is that this system is only
trained to solve the problem for certain
types of inputs it's trained for example
with image patches or with audio signals
it's not trying to solve the problem for
any random signal which of course the
the full algorithm can do ok so let me
skip ahead so it's sort of various ways
of you know training the systems but I
want to I want to come to one particular
one which is using those kind of latent
variable models to do video prediction
and to do so in ways that can help for
example self-driving systems so let's
say you have a bird's eye view of your
car surrounded by other cars the green
cars are all the cars around you you are
the blue car and you observe whether the
car someone you have been doing so far
and what you'd like to do is being able
to predict what cars around you are
going to do so you can plan a good
trajectory ahead and avoid avoid
accidents so you can have you know with
train one of those energy based models
to kind of make the prediction and the
prediction would be multimodal so these
are examples I hope you can see the
video if you don't have anything
variable you just make your point
prediction you get the blurry prediction
that you see on the second column but if
you do use a late
variable and you change the way you
sample an enviable you get four
different types of predictions here that
are shown on the four columns on the
left the one on the on the right and
sorry
on the left is just a recorded you know
training example that actually occurs so
you can have multi modality with this
these latent viable models and this is
very important if you want to do model
predictive control which I'm sure many
of you are familiar with so you have a
forward model of the world that predicts
where the next stage of the world is
going to be as the function of previous
stage the action you're taking and
perhaps some latent variable that you
have to draw at random okay and if you
can unroll this model in time and you
can measure an objective function C of s
at every time step you could imagine
backward and decide optimizing the
sequence of actions that will minimize
your cost right this is classical model
predictive control and of course all of
those is differentiable your model has
been trained in advance so you can just
do this you know by automatic
differentiation using the same tools
that we used to train your minds now
another thing here instead of preferring
a sequence of action you use the
gradient of the cost over the sequence
with respect to the action to train a
policy let's go to policy Network so
it's a network that takes the state at
the previous time step and tries to
predict the action so again it said one
of those types of amortized inference
that was saying you about earlier and if
you train the system to drive itself in
the context of what I showed it actually
works quite well this is a trickier to
use to kind of prevent the system from
going into parts of the space that are
what the predictions are unreliable but
it works very well so this is an example
of the blue car driving itself
you know white dot indicates whether it
accelerates whether it breaks or whether
it turns and you have to realize that
here the green the green cars are
recorded the blue car is invisible to
them because it's a new car that we just
put in in this recorded video and so it
has to basically learn to avoid the
other cars about getting squeezed and
you know it works okay right so
conclusions conjectures and open
questions so supervised learning is
learning dependencies reasoning is
minimizing energies basically inferring
latent variables you could think of it
this way I didn't
much about this but this is what it
comes down to the main obstacles we are
to face is how to properly represent
uncertainty and my recommendation here
is to abandon the idea that we need to
launch distributions and learning
energies is simpler mathematically and
and in theory so I'm proposing to use
energy based models and you know there
are contrasting methods and regular
resolute and viable methods and my money
at the moment is on regularization
variable energy based models even though
the contrastive form is actually as of
today much more efficient now beyond
this good energy based supervisor
learning be the basis for machines to
acquire a common sense basically by
learning how the world works by watching
video and and you know not attracting
much like like baby humans and animals I
think it's an interesting question I
hope the answer to that question is yes
but I really don't know and just as a
side note a lot of people are talking
about artificial general intelligence
and are kind of hoping that by scaling
either supervisor or reinforcement
training methods that we currently have
with bigger hardware and more data will
reach something like human level
intelligence I think this is completely
hopeless I think to reach anywhere close
to even cat level intelligence we will
need to go through this type of learning
we're not gonna get there just with
reinforcement learning of supervised
learning there's been a lot of
discussions also about deep learning
whether it's an engineering science or
natural science whether it's you know
actually a science or more like alchemy
and the interesting thing about deep
learning is that it's an engineering
artifact that we need to study and that
we don't completely understand and it's
not because we don't understand it that
we should not use it right because in
the history of Technology and science a
lot of artifacts like the telescope the
steam engine electromagnets you know the
airplane etc have appeared before people
actually had the theory to explain how
they work right the theory was actually
developed as a consequence of those
artifacts being developed and being
useful and so it motivated Theory
theorists who can really understand what
what this was
so you know I think I think that's
really the the history with the work in
a kind of similar situation here where
we have an artifact you know deep
learning systems but we don't really
have sort of a general theory that
explained why they why they work so we
are like you know people who build steam
engines in the early 19th century or
late 18th century you know they had to
wait for you know roughly 100 years for
thermodynamics to really explain you
know the limits of it and and why it
works so you know the the government
Korea is basically to figure out what is
the equivalent of thermodynamics for
machine intelligence for intelligence in
general thank you very much yeah I think
much for a fascinating lecture and so
now we come to the question-and-answer
session and there are only dr. of
questions which came in so one question
is for instance by Bernhard got my heart
he asked currently the weakness of
reinforcement learning our strategic
decisions because they need to allege
rewards can you give some input about
how to solve this problem
right so okay there is three problems in
reinforcement learning the first problem
is your objective is not known it's not
it's not a module in your program it's
not a differentiable function whose
derivative you know it's it's something
you can only measure by taking an action
in the world and then observing the
result from the world so the world is
going to give you a scalar value
basically that tells you the action you
took was going bad and it's not going to
give you that value every time it's
going to give give it to you once in a
while okay so so basically it comes down
to trial and errors the second problem
is that the action you take will
determine the sample you see next okay
and so you know you're going to ride a
bike and you know if you do something
you're going to fall that will determine
what what sample you you learn from next
and so you cannot make the usual
hypothesis in machine learning theory
which is independence with samples right
you don't have independent samples so
those are the two main problems and the
third main problem I already sort of
alluded to which is the fact that the
environments only gives you back a
single scalar and so because it gives
you a single scalar it's it's horribly
inefficient you need many many many
trials to learn anything unless you
already have a good representation of
the world so one might ask what is it
that allows humans to learn to drive a
car in about 20 hours of training
without causing any accident mostly
whereas if you were to train a you know
some of the best reinforcement learning
systems at the moment at least the pure
ones that are model free it would take
millions of hours of training which of
course you could only do in simulations
and the car would destroy itself you
know hundreds of thousands of times if
not million so the difference is that
humans have this predictive model of the
world in their head and they also have a
good idea of the objective function and
because the subjective function is
computed by their brain is
differentiable I mean they they know it
right so you could imagine some way of
its ability ingredients that don't imply
interacting in the world that's pretty
much what I saw what I showed in my
little example of a surviving car it's
not really fast mentoring there nothing
is reinforcement running because
everything is differentiable everything
is is known and the system has a
complete model of the world which is
itself differential
which had learned through observations
with a supervisor on me
so what allows us to learn to drive a
car quickly is that we know that you
know if you drive next to a cliff next
to us if we if we turn the wheel to the
right the car is going to run off the
cliff and nothing good is going to come
out of it right or model of the of the
physics and our model of the objective
function which is that we shouldn't fall
down a cliff is such that we don't need
to try we know that the result is going
to be bad right so this idea of learning
predictive models crucially is what was
going to bring the sort of a new the
next revolution in reinforcement
learning as well now the problem is that
is the same problem that we had with
neural nets way back which is that from
the theoretical point of view it's very
hard to prove that model-based
reinforcement learning works better than
model free reinforcement running in fact
you can prove the opposite so in theory
model base is worse than model free but
in practice it runs way way faster if
you do it right so you know the proper
way to do this is not clear but it's the
future there's no question thank you so
another question is about energy based
models and the question is energy based
models look like free energy
minimization from chemistry and material
science to simulate materials properties
I mean are these ideas from both fields
do they use ideas from the other field
so can you comment on that dead
connection yeah absolutely I mean I I
use the same simple listen symbols as in
statistical physics basically right so
you know the minus 1 over beta log
integral e to the minus beta the energy
that's a free energy I mean you know up
to the minus 1 over beta but this is
like the you know there's a very famous
equation in thermodynamics which is a
free energy is equal to the average
energy minus the the temperature times
the entropy and the this term that I
talk about here is the minus temperature
times the entropy so basically it's a
free energy but you know divided by the
temperature multiplied by the
temperature so I mean clearly the the
math is the same you know people could
you know talk about variational
inference which
completely borrowed from from from
psychophysics when they don't use
variant inference to use my Monte Carlo
and Markov chain Monte Carlo inference
which is also derived from physics
so yes the the math the math is the same
the the concepts are the same some
people in machine learning actually get
annoyed by me because because I I stick
to the notation of physics and people in
machine learning you know I'd like to
talk about like you know objectives that
you maximize rather than me my's so and
and they always you know prefer to write
distributions instead of energy so so
they get annoyed by me because of that
but yes so another question is by Gantt
avi but he asked in many worlds people
trained to apply contrast if lost in the
latent spaces can you comment on when
the model trained was lost applied in
your Innerspace would surpass the
contrast if loss applied in the let in
space or in general what are the
weaknesses of contrasted contrast if
learning paradigm right so I don't know
if I can go back to the the slider I'll
try but it may take a little while yeah
so I mean the contrastive the siamese
network the sort of standard form of
contrastive learning that people use at
the moment actually does contrastive
learning in the representation but this
is a slide that I escaped that I want to
go to if I can which is another example
so this is another way of learning
training a latent variable energy based
model to do prediction which does not
require contrastive terms and so this is
a thing where you try to extract a
representation of a friend so let's say
those are video friends right so you
look at YT minus 1 YT and you're trying
to predict the next frame YT plus 1 and
one thing you could do is basically
train an autoencoder to encode this
frame YT of 1 and use the same motor
encoder to include the next frame so
what you get are two codes H for the two
frames and then have an internal model
that you train simultaneously that
attempts to predict where the next H is
going to be for the next frame and then
you decode and
reconstruction error I mean the
prediction error for the for the next
friend so ignore the the bottom part of
this at the moment okay so this is sort
of a predictive model it doesn't require
any contrastive learning or anything and
the reason it doesn't require any
constructive learning is because you
force the age representation here to
contain all the information by the input
because you force it to reconstruct the
input and so you know that no
information is going to get lost in the
encoding of a y2h because you force the
reconstruction that's an alternative to
the contrastive term that tries to make
sure that you know representations are
different for different inputs here they
must be different because the system
must be able to reconstruct the input so
you had those representations and then
you train a third network in kind of a
predictor network if you want to predict
the next the representation of the next
frame okay but it's quite possible that
there are a lot of things in the
prediction that are not predictable that
you know the information information is
insufficient in the past to predict the
future as I said you know the world is
not entirely predictable is some
stochasticity to it and so you can do is
add a latent variable to this H that
basically takes into account the all the
stuff that you cannot predict that may
happen in the in a model the diagram
here is not completely accurate for this
situation I guess the situation is more
akin to the one I showed for the the
driving system and you can think of this
encoder here as again implementing this
amortised inference idea of you know
predicting the optimal value of the
latent variable without having to do
kind of inference so yeah I mean I I'm a
big fan of this idea of eventually you
know essentially kind of forcing the
system to do prediction in latent space
but but I'm trying to eliminate the
contrasty phase because it's too
expensive computationally yeah thank you
very much so for the sake of time I will
now hand over again to powder thank you
so much young for a really fascinating
talk I very much enjoyed it and we had
an amazing turnout in terms of
participants just to emphasize this I
think at some point we had more than 500
just here and there
many more oh well thanks everyone for
showing up that's that's that's
wonderful to hear so I'll just conclude
this session by reminding everyone of
our next and final plenary talk that
will be given by Yuri Nesterov on
Tuesday this coming Tuesday same time on
in xx accelerated high order proximal
point methods so until then see you soon
thank you