I'm from between you about the
integration of the casa di intention to
the first of all why do you care if you
know density you probably also know
about your ass magician case for the
grass is an API that makes building
different models easier and faster so
it's a different tool box and it's all
about either fuse reducing complexity
regency reducing kind of those and each
you make learning easier to use what
happens is that you also making it
accessible to more people
the key idea with karen is to put the
planning into the hands of everyone we
believe that in the future differently
part of every developers to box it will
not be judged for experts and
researchers it will be into everyone
tense and this would happen with
platforms like dances through an API
care us to cast is better in the suit
not as a library or and as a codebase
it's more than a TI specs and it's nippy
I state that has several different
implementations there's the channel
implementation which was the original
and territories by already two years ago
residential implementation of course and
then there will be more implementations
in the future for instance microsoft is
working on scnt implementation cats and
two ND now the tensor for implementation
existed as separate conservatory with
the shooters and what we're doing now is
that we are bringing the canopy I
directly into the craft projects into
the tensile project and that really
that's a big step towards making
intensity and creating accessible to
everyone so it's concrete see happening
is that we are bringing into tensorflow
this Karis competitivity modules have
the terrace which is an implementation
of the Charis pecked has been from the
granddad's 410 secure and so as part of
this move
we're introducing into quotient rule new
data structures like layers which Martin
just introduced and with two models
which are containers or grasses layers
to reduce to be and carries a
distraction and they will be shared
between coordinates
through and it's cheaper carrots module
2 because this Care Act implementation
is being from the ground that on top of
the sherry destruction it is fully
compatible with all of the tensor
functionality that means it's also
compatible with suspense features that
says the experiment API that money
introduced so if you're accurate shooter
as what this means for you trust it
gives you access to more flexibility you
become able to very easy mix-and-match
Puritans of functionality which
recessionary you can be a lot more party
molecular models and what's more you can
access to really advance 3 battle
features through the idol wall
I'm sensor for training API that
might've mentioned you can access to
very quick and easy distributed training
forecast models you can access to
training LOL distributed hi perimeter
training which is really battle and you
can vary CD Projekt your cast models to
production we center serving the short
answer for user what does this change
means to you
basically it gives you access to the
full scope of the kereta API to make
your life easier without leaving your
existence of your work for you can just
start using the caddy I at know lots of
flexibility you don't have to adapt all
a scare as you can just add up the
layout you need for instance and also it
gives you access to any existing piece
of open source code against go and get
ad copy some business cards and drop
into your code base and by just changing
imports it will just work so it gives
you access to a lot of existing open
source code you can start leveraging
into intensive the field so to make
things a bit more concrete I would walk
you through an example of what your
workflow will look like when using
carrots with sensible and we will be
looking at a video question answering
model and we'll be using here at to
define model to solve this problem and
will be trained it using the high-level
intensively trained API in a distributed
sitting
this is a problem looking at we have
some videos I which is employed about
four French the second and rocky 10
seconds the video so we are about 40
friends the video and we are asking
questions by the video about you know
what's going on video but the content
like here we have this video of a woman
backing objects into cost you can ask
what is one doing
I was the color of the shirt and the
answer should be a single word so we are
depleting model that will take as input
the video at the sequence of frames
about 40 friends in order and the
question and the model should have put
one word the answer and so if your
discretion was the one doing his backing
it is actually interesting because if
you only take one frame and try to Train
comment for instance to answer the
question the prime yet just from one
frame she could also be impacting us
right
and the reason you know she's actually
packing is because the order of the
friends that if you get a second frame
the roads that said that got an extra
ink inside the car and we expect our
returning models to be able to leverage
this order to correctly answer the
question so needed city to determine the
difficult problems and just a few years
ago like three or four years ago before
10 to 24 carats this would only have
been accessible to a handful of large
companies are in a very well funded
research that especially if you wanted
to make your model train on a cluster of
GPUs and now we sense the floor as a
platform and with with characters API
we're making the tree difficult problem
accessible to anyone with just basic
items cooking abilities
we're putting the food power turning to
him the cerebellum the his model that
you will use to solve this problem at a
high level it has two branches there's
one branch that's meant to encode video
input into a single vector and one
branch meant to include the question the
sequence of words into a single vector
so we have one vector including
information about the entire video one
sector including permission by the
entire question and then we can get in
these two vectors
and give that one single vector that
anchors information about the entire
problem about the combination of the
video and the question and that's kind
of the beauty and the magic of the
planning is that you take the scepter
inputs like a video for instance you
take similar meaning and your am being
destitute input and your am being this
meaning into geometric space you're
turning meaning into vectors and then
you can learn interesting
transformations of the geometric spaces
that's really like the beauty of
computing and so in our case will be
taking this vector including and the
entire problem and will be training a
free clinic let's stay on top and it
will end up with South max over at
predefined vocabulary and so the world
with the highest probability and mr.
Kerry will be the answer to the question
now if you want to get the higher level
of detail how do we can video into a
single vector how do we turn a question
into a vector for the video we're
starting out with the video as a
sequence of friends to each frame is
just an image with LGBT has a height
width depth Odette's the other sequences
from you use a covenant to transform
each frame into a single vector
essentially you're using the
constitutional basis of a preacher
network and do something to extract 1
vector that encodes the visual content
other frame and so what you get out of
this is the video include a sequence of
vectors where each vector and code the
visual gardens of one frame and then you
dated sequence and you run it through an
STM you probably know what nme basically
it's a direction type of network that
can process sequences and takes into
account order right and this LSD my eyes
with a single vector and getting
information about the entire video and
so the question to turn a question into
a vector we are following a very thin
that process we represent your question
as a sequence of integers each integer
is standing forward and then we met each
where to web vector yeah and a process
called am building right so we get out
of this as a sequence of words extra
as a sequence of vectors we run it
through a different a team player which
will encode the entire question a single
vector so in chaos here the architecture
that we would use that implement this
term this model and it's literally just
a very simple translation of the
treasures graph so called the video
encoder basically we're starting from
the view as a 5d tensor the first axis
with that Texas then you have the time
axis then you have a tree density which
includes a frame and so we will apply an
inception industry network pre-training
imagenet to every frame of this file
denser to extract 1 vector one city of
vector friend and so after this we get a
sequence of visual vectors which will
run through an SMA and for the question
being part where simply using an
invading layer to match our question
through a sequence of integers into a
sequence of vectors and we run that
through an STNA finally we use it a
cattle operation to bring together these
two vectors and we staff you dense
layers and we end up with a softmax over
our are pretty fine vocabulary and
returning that with a target answer word
include is one hard rector so what does
the implementation new clutch
it's very simple this is a interesting
here in the video encoder inches 5 lines
during the first line you're just
specifying the shape of your video input
225 cancer with the shape argument you
do not actually attract the best shot
so the first axis said tune on the time
axis so it's set to not because we
wanted to be viable we want this model
to be able to process videos are of any
number of frames and then you have the
shape of a single friends is she your
friend is 150 x + 50 RGB image it's the
tents of the 2150 by three and then in
the next line which we are doing a
single line is associating an inception
victory network that will automatically
load pre-trained wait so trained on
image that
and we'll consider this network to be
doing visual feature extraction
essentially will not be including the
pleasure part of the search industry on
in the convolutional base and we
applying some average bring on top of
the bottle next year to just extract one
single vector their image input then the
next nine we are sitting at this
covenant to be non-training more to a
different is that during training we not
be updating the waist and why simply
because it's a pretty train model and if
you want to update its weight while
training on this new problem of question
assuming we would actually be wrecking
the representations that it's Mother's
original on imagine that you don't want
to do that you just read it and so what
the next step is to use a time
distributed layer to essentially take
this cabinet and apply it to every step
of the time axis of the video input the
standard security layer just take this
comments and apply to every frame and
what comes Adam to say is simply three
tensor which is a sequence of these
frames a sequence of different vectors
so is requesting for one friend and
finally we r n this sequence denser
through an SCM layer and it gives us one
simulator including at the entire video
and that you cannot is when we are
instant shitting disgusted a similar we
only specify one parameter which is
number two minutes in dsm layer and if
you think about it that's quite
remarkable because there are so many now
you know advanced configuration
parameters in India self-esteem layer
and you know there are a lot of this
practice is you have to follow when
using sem for instance you have to
remember that your current weight should
be initialized reasoning the Ottoman
installation and you have to remember
that the bias of the front gate should
be in this last one you know this type
of best practices that you really need
to follow if you want train to smoothie
and one key principle of Charis is at
best practices are included so
everything is here as well optimized
I'm default configuration that takes
into account all the deck box best
practices you can just read I can't read
supposed to be your default and you just
you know get your model to train out of
the box with that requiring you to in a
functioning every of these high
parameters the best practices included
and you should just you know care about
honest flying a few key parameters like
the number of units and everything else
should just work right out of the box so
the question encoder it's even think
that we just three times
first we specify the importance of the
question so every question we just be a
sequence of at most 100 integers so we
will we be limited to question that are
that are 100 words not and then we embed
every integer into web director of the
ids emitting layer and we note that we
use masking and disseminating layer
which means that for instance if we do
less siding on our questions which means
that if the questions are shorter than
100 words we were just nearby the rest
with zeros to get to 100 and description
mask which would be propagated two
directions layer right after that and so
we finally encode the sequence support
vectors into a single vector via another
lstm layer so note that when we are
using this pre-trained inception network
it's absolutely sentimental to be
loading disease between ways and because
you're dealing with a fairly small data
set and this is that alone would not
have enough data to allow you to learn
to extract interesting visual pictures
you really need to make this network and
actually trained you really need to be
leveraging mystery train ways and so
finally this is a high you end up pretty
easy answer word you are taking their
video vector and the question Victor you
are getting making them with just a
consideration and finally you are
applying a couple of density as
and you're ending up with a 1000 units
we have vocabulary as possible answer
the situation no one thousand different
worlds and here's the step where you
will find the training configuration of
your network so you just interesting a
model which is a container for
graphically and you're still shooting
them by just be trying whether the input
of the model
what does the output you can add any
number of inputs a number of output and
you are telling the model that he should
use at the academic advisor during train
and this last expression trees projects
like you cannot is that when defining
our classification yeah with 1000 units
we did not betray any accusation it's
actually purely in your lair the South
max accusation will be including
included with the last and so to sum up
this entire code is just about 59 to the
very short so we are essentially turning
is very complex architecture including
loading pre-trained wait introduce a
handful of France is really a reducing
complexity and by allowing you to rely
on before configurations for every layer
we're reducing contribute unit to care
about mass of the parameters of us him
later for instance and the best part of
it the most magic part is that because
this implementation of Charis is built
from the ground up for them to fuel its
really compatible with things like
estimators and experiments in just 19
you can instantiate and attention to
experiment and this gives you access to
distribute the training to train and
caramel and so on and also access to a
sort of serving in just a few times you
can start running your experiment as
shit with your question and sing model
reading your video data and questioned
data and answer data from at a data
frame you can start anytime access to
abusing just experienced
so what you should take away from this
talk is very simple we are taking the
Casey I we are bringing it into tens of
your it's a new implementation of Kara
switch from the grounded for tensile and
gives you attitude and more flexibility
you can start mixing and matching pure
sense of community with your class model
other tents of your users you access to
the food kereta API to make your life
easier and so you can expect discuss
competitivity modules to exist as the
country that casts yts 1.1 and finally
achieve the carrots in a sense of for
1.2 and so we believe this is a big step
towards making 10 through and deep
learning accessible to everyone that's
it for me i believe next we have a some
snacks
thank you
[Music]