mananas moon a star vehicle hacia poor
presentation look well maybe it'll thank
you very much for introducing me just
like to say that I leave the
cybersecurity laboratory and that
belongs to the BBVA a group where we do
many things that are very there are lots
of fun so what are we going to speak of
the title the heading is quite ambitious
because it's called advancements in
machine learning applied to
cybersecurity I have a now so what I'm
going to do is to try and do away with
the myth of what AI is not good for and
I'd like to tell you what it is good for
so I'm going to summarize how to attack
machine learning algorithms so we're
going to take our going to make a trip
around these things so what are we gonna
do this is a very quick introduction to
a I even though it's true that I speak a
lot about AI but in any case I've had to
learn some things on cybersecurity and
I'm gonna give you a very short
introduction regarding the context of
artificial intelligence and then I'm
gonna tell you about things you cannot
use it for that is limitations many of
which lie on a defensive security and
then I'm going to speak about offensive
security which is where we see there are
more advancements now formalized a
formalizing of artificial intelligence
from the point of your security and then
conclusions so what are we going to do
in the next are 58 minutes we're going
to try and do away with our myths
regarding artificial
and as I've applied to cyber security I
of any use well it is or as we will be
speaking about it from this slide we can
see the different techniques that are
learned like supervised learning non
supervised and reinforcement the
traditional things like image
classification recommendation systems
etc etc so here we have a large part of
domains where would be interesting to
use artificial intelligence and in fact
it's very interesting and this chart is
quite very well known and by the way
there is a multitude of companies that
are trying to apply AI to the own
domains because thus they can achieve an
advantage competitive edge if you look
at this chart chart the part on security
is very small there are many scenarios
where AI is very useful indeed
but in the security they are very but
there are very few companies
specializing in security some companies
are trying to apply it but there are not
many companies that specialize in during
the potential of these algorithms
applied to security and not let me move
well there are some areas that are of
great concern the regarding AI have
related to this this has to do to
whether these so this software can be
controlled by humans if they could reach
a stage where they would be harmful for
humans understand difficult after 1 min
take some time right now few pictures of
scientists who are working with
artificial intelligence where some of
them are well known
they teach in Stanford etc at least some
of them and one of the main problems is
right now we have to regulate the way in
which these algorithms are used to see
if we can cope or manage the moral
component in order to see whether these
algorithms are going to be good or not
there are many cases comportamiento how
I am iterator replicates the negative
attitudes of humans like racism etc and
this not only affect the net saving
safety but this is also related to
society and the economics and there are
many people that are proposing a
universal source of income so that by
the time that the AI is more developed
those people we won't be able to find a
job we'll be able to get an income via
taxes are paid by artificial
intelligence this sounds like something
out of a movie but nowadays they have we
have many other problems perhaps
tomorrow we will be murdered by a robot
but there may be other problems such as
wars between different cultures and the
problem is that we need artificial
intelligence in order to solve more real
problems that will affect us as a
species so obviously we can not do
without AI we need to develop it more
and more even though we run the risk of
reaching up where we won't be able to
control it and in fact when we speak
about the rapid of evolution
ái we tend to show just like this of
algorithms here we have several examples
here we have a game that top and we can
come up with specific algorithms that in
specific tasks will achieve better
outcomes or results than humans will
would reach when solving specific
problems this is the alphago zero and so
AI will be substituting humans in some
areas but the true thing is that this is
not all the way true and that we speak
about singularity this precise moment
where an artificial intelligence will be
able to perform better humans specific
tasks the problem is that there's a lot
of uncertainty nobody knows when this is
going to happen it is bound to help
happen but we don't know when the most
rigorous study that we found that tries
to establishes some possible scenarios
when this will happen and things that
would impact us as a society so what
this study says is that from now until
the 4th and from now till 40 years ahead
of us many things will be substituted by
a high here for instance the ability to
write a book as human would do it by the
year 2040 9 writing a book by 2050
different scenarios for instance
replicating the performance of physician
and other tasks in 45 years
AI supposedly will be able to solve some
problems that only a human being can so
now this is an opportunity but also a
risk
handle this adequately so these are
things that people technicians and
specialists in technologies are dealing
with right now we don't know how much
money major companies are spending and
developing this technology AI has
evolved a lot in the last decade because
there are many companies investing huge
amounts of money in this area and
there's something that I'd like
highlighting everything related to AI is
considered like the atom born bomb and
there's parallelism with the Manhattan
Project when the experiments were done
at the beginning with this energy that
they were afraid that they would burn
the atmosphere and the same is happening
with AI we've detected technology with a
huge potential but we still don't know
whether we'll be able to handle it as a
major powers investing in these
technologies because supposedly in the
next few years the economic social a
military impact is going to be huge and
this is related to you the introduction
of machine learning many people speak
about machine learning there are some
very good courses last year in cyber
camp there was a very interesting talk
talked on this issue and I have about 15
minutes ahead and I'd like to touch upon
some issues related to AI plus machine
learning and applying to the world of
cyber security the first slide has to do
with working without complex many people
here I'm sure have no idea had know how
to apply machine learning to cyber
security and possibly nobody knows how
to and there's
I really like and that is part of all
this marketing strategy around these
technologies applied to the field of
security says everyone talks about it
but nobody really knows how to do it
everyone thinks everyone everyone else
is doing it so everyone claims they are
doing it and there's a whole scenario
technology's contributing with some
differentials that are value so the next
few minutes I'm going to try and give a
few brushstrokes this issue about this
subject in order to see whether we can
solve this problem in fact there is
something we should reflect upon and
that is whether a may solve actual
cybersecurity problems let me read to
you a paragraph of the publisher of this
an agenda that says this is a world
where massive amounts of data are
replacing any other tool that can be
applied let's forget about the human
behavioral theories and linguistics or
sociology let's forget about taxonomy or
psychology who knows why people do what
they do the important thing is that they
do what they do and we can do a
follow-up and measure it with enough
data figures speak for themselves
and this is an opinion by the publisher
of this journal and I'm against this I
oppose this opinion I believe that
there's a human thing to this there
contributes with differential on added
value that AI won't be able to provide
or at least not in the next few decades
and there are many stories many articles
and papers that speak and justify the
existence of many scenarios where human
component provides or
contributes with an other component
where these AI systems are failed there
are other scenarios though where I can
simply not be applied imagine that in
cybersecurity we have the best possible
algorithm in order to identify a user or
an attacker but then there's a law
there's a rule that prevents us from
applying that procedure and doesn't
matter whether it's good or bad
we simply cannot apply and on this slide
I'm giving you an example in the case of
giving a loan a bank giving a loan
and here we cannot use machine learning
algorithm well why why because due to
the data protection law says that
there's decisions that may have an
effect on the life of citizens must be
reasoned and consents and revised this
means that if you know anything about AI
we have some environments where we get
where we are trained there are some
entries and outputs
and we don't know how this output has
taken place we don't know why so there
are scenarios where this is simply not
possible that is we need to be able to
negotiate there needs to be a
negotiation between an entry and exit in
terms of security privacy identification
we need to bear in mind all of these
things because even though algorithms
work we wouldn't be able to do anything
about this and this is the example that
I wanted to tell you about this is a
research we did I don't know three years
ago and this is a very specific example
this was a malware detection issues in
Android with AI in this case with
and so we started using AI in order to
detect everything in order to detect
malware and many what's going well well
characterized that worked half way well
but we needed more accuracy in order to
work like human analysts would so in the
case of Android applications we saw that
there were certain elements that were
differential advantage and made this
model work better there's this example
on the left and we realized that by
using natural language processing
techniques we realized that an attacker
when publishing Marwa in the Android
Market his or her description had a very
poor linguistic quality using very few
words
not many ejectives verbs per company to
the legitimate applications did better
descriptions that is longer descriptions
and much better written much richer
language so this proves that the human
component
still still makes a difference compare
wave AI which could be an addition a
compliment but by no means is it a
solution so what is actually being done
in machine learning applied to
cybersecurity well I think machine
intelligence is useful for defense
defense
no offense as shamea what i'm going to
show you right now has to do with public
information that's known from companies
and academic studies and part of this
top is justified by this person do you
guys know who this is
please raise your hands
that is the case this is Samir and one
of his last presentations or conferences
he said I think machine intelligence is
useful for defense no offense
this is sentence was very important for
me because
Shammi is possibly one of the most
brilliant people on earth I know I was
surprised by this because in the
presentation we are going to see that
offensive security is what has evolved
the most unlike a defensive security so
what is AI used for right now when
speaking about security or defence
security or defence traditionally
speaking normally it is used for
detecting fraud now where everything
related to analysis and monitoring and
variant and more and more in everything
related to authentic authentication a
control official Parowan i'ma chose out
the other scenarios where it would be
very interesting to use this like in the
development of software and this is the
normal scenario where all the major
companies want to contribute where their
own thing
using a this is the normal cycle that is
always
presented algorithms a training process
identification etc so what public
information exists in this field we go
to companies and more and more we start
seeing projects that are starting to
distribute tutorials and information
focused on defense security we have ml
SEC for instance where we see the work
by many researchers and most of these
proposals focus on two things basically
on the one hand traffic analysis and
network monitoring and everything that
has to do with malware detection or
preventing malware from exploiting a
serious certain characteristics as you
possibly know these are proposed these
proposals major suppliers major
technological companies or security
companies are trying to use these
algorithms with their products in many
of these cases as we shall see later
technology is not that disruptive these
are twenty-year-old
proposals with twenty-year-old
algorithms webpart which perhaps do you
make a difference somewhat but not their
associated Association associated to the
sort of marketing related to these
proposals this is an example of
companies that have greater potential
from the point of view of AI you can see
a small square in red where we can see
some of them focused on cyber security
and AI
let's hear something silence we see the
names of the companies plus a couple
more I don't know if you can see them
basically countries that work in this
area is the US UK and Israel Jews are
the three main three most important
countries that work in this tension in
this area and they basically work on
network and monitoring and in everything
related to preventing software from
exploiting certain behaviors so what
prevents us from applying real AI well
I'm going to summarize are these things
or the one we see the amount and the
quantity and quality of data if any of
you use as a are probably I asked you a
question how many data do we need in
order to train the system answering this
question is difficult because the answer
normally is the more the data the better
but that's no scientific answer what do
you mean by the more data the better so
I guess it depends it depends on the
quality of the data what does this mean
I mean how do you measure the quality of
the data these are abstract things and
if we apply this to cybersecurity this
makes things much more difficult and
prevents us from detecting as an attack
attack vectors for instance a threat
we've never seen what do we do we
normally do a profile of the normal
behavior and if it deviates from this it
means that this behavior is related to
human so from that point of view and
conceptually speaking we are still
working like we used to 20 30 years ago
so we haven't evolved beyond white or
black artists and there's another thing
that's very important that's related to
static training normally not the visual
I people trained in our systems and they
apply them but if we go underneath and
it will go deeper into this actual
training for real training is not an
ongoing training and by this I mean that
these algorithms would learn as they go
go with an important new ones and that
is that this training has to bear in
mind the presence of an active attacker
that is the actual proposals that are
being worked on is having an intelligent
that works about with the presence of an
attacker that will make you more room
succeed or fail more points that will be
developing more in-depth and that's
really important and everything that's
related to the privacy of algorithms and
models when we use AI we develop a
series of models and these models what
we do we publish these models can
somebody steal them can somebody
manipulate them which would affect the
security of our systems the answer is
yes and this is something we'll see very
quickly and a couple of issues in the
end and that is offense security exists
in AI that is today but it's possible to
manipulate AI algorithms that are used
in defense in order to provoke different
scenarios like I will show you but
normally these are targeted attacks or
mass attacks
and major openers institutions like
DARPA that willing to give lots of money
to those who can define AI systems that
will be able to learn in an ongoing way
but in the presence of an attacker and
the role of the attacker is to make this
system fail not work and if somebody has
the solution to this this company will
be delighted to give them lots of money
in order to you work on this so when we
speak about the real defense and offense
security we need to import we need to
understand what a generative adversarial
Network basically we have a
discriminator that acts like what the
charge of defense makes a decision we
have generated trick the system so what
degenerate does is to enter information
to the discrimination
so that generator is going to be
learning as a ghost and on the other
hand the this screaming nature based on
the sample she receives it learns so it
will learn more from the attacker with
AI where an attacker can attack a system
trying to beat it and in defense
security well this is a very complex
scenario for which there is no ideal
solution that many of the existing ones
are still being studied on a theoretical
level we don't have a lot of time I'm
just going to show you a couple of
examples in the area of defense security
which I believe are interesting apart
from the network monitoring once malware
detection and this is related to data
protection and I'm going to speak a
little bit about cryptograms
and I'm going to show you a research
done by Google a few years ago and the
headline said that they were able to use
a single use algorithms cryptographic
single use algorithms and we have this
system like the one I showed you earlier
whereby Alice and Bob want to encrypt
information in this communication we
have the presence of an attacker
she's a neural network so Alice and Bob
without saying anything or because of
the way that the network is working they
detect the presence of an algorithm
and in the algorithm is such that the
attacker could not decipher it a human
being wouldn't do absolutely anything
okay what is the problem about this well
a human being sees this this is
basically anyone a soliloquy told a
figure better tsunami an X or that is
what is being calculated it is a very
interesting proposal and it may entail a
significant change in cryptography here
is another example a quick simulation
we've made of this attack basically we
have a G a and scenario where one of the
neural networks is able to create
digital pictures that look very much
like pictures of celebrities for example
and in that process of deceiving the
detector it is hiding information behind
the pictures it also includes an
additional component that tries to
detect whether the information that's
been hidden is well hidden or not the
deduction of this study which is more
interesting than the one we showed
before is that with this gan system with
neural networks we can create new
algorithms which are more robust than
the ones that we have now if you have
the chance of taking a look at this
research and you know a little bit about
thin air stately and ography you will be
able to value the importance of this
study this is what we've seen about
defensive security now regarding
particular problems an issue here is the
privacy of models where do I publish
models and can attackers attack me this
is something we will understand in a few
minutes what are we trying to do in
order to protect that's an area well
crypto nets are being researched which
is the fact of creating ciphered a i
creating cryptography and apply it to
new
networks so if we are creating a
training model with Amazon and someone
is able to steal our model that model
would be coded that's how the attacker
wouldn't be wouldn't be able to draw our
knowledge from there only users with a
cryptographic key would be able to use
it if this and stop working in a more
professional way in the future
stealing training models will be an
issue that may be solved in the future
and what we say in the world we said in
the beginning one of the main problems
in cyber security is that sometimes we
try to detect some things where we have
very small samples of information from
that is the samples we have our scars
and they aren't enough and on the to
make an identification will we be able
to solve that problem we don't know
there are other domains of application
where a solution has been found for
example in robotics
there is one shot learning which is
applying models with just one sample
this is what can be done in certain
scenarios in robotics a robot sees just
one single example and can replicate
that behavior then for example in terms
of languages there is the cell soft
learning for example this is what Google
does language the system is able to
understand a new language without any
training without any samples learn and
replicate that language will we are we
able to do that in cybersecurity
not yet will we be able to do it well we
don't know yet these are challenges we
are currently working on when we focus
more on offensive security we will talk
more closely about machine learning and
how
software and hardware need to be
understood as one more element in our
industry and included in pen testing
software development etc in all the
words AI is one more element that needs
to be included and our industry well
this was a brief introduction there
we're going to focus on the advances in
terms of offensive cybersecurity this
year's where the greatest progress has
been made we need to tell you about four
different types of attacks attacks we
know of the classic attacks which is
using AI in order to do what we usually
do with all the tools or pen testing or
hacking techniques but is an improved
way then synthetic systems that attack
machine learning with proposals
another type of interesting proposals
which have to do with applying machine
learning to defensive security based on
machine learning and then a fourth
category where everything goes into we
will see how we can steal a I models and
how to introduce backdoors it's
important to focus on the picture we
have on the top right corner here if you
are into pen testing and black box pen
testing for example this is very similar
think about that
scheme and think what an attacker would
be able to do if an attacker can access
only the inputs the entrances what can
they do and if they can only access the
outlets would can they do using that
logic we can create attacker models and
build better systems in order to avoid
attacks we will now see some examples of
each when it comes to classic attacks he
have a couple of examples
the first one is passed Gann it's a tool
that's ready for use
imagine someone steals a password
dictionary from LinkedIn on a door a
different network then using neural
networks you send over that list of keys
and this proposal is able to create a
cracking rule a rule that will not only
match with the keys it sing but with
other similar keys as well so what is
this for
this proves that with this technique we
can hack passwords more efficiently and
quicker also this is the proof of that
because you can generalize the rules and
then apply them in future incidences
this is an example of the last D if
where neural networks were used in order
to create sequel injection attacks when
you have the rule already instead of
following your your own criterion or
using the brute force you can reduce
that database so this is the way of
improving traditional hacking tools with
neural networks second type of attacks
synthetic attacks this is very well
known it's been under the price many
times but we have to underline that it
is more and more complicated to tell
real information from unreal information
particularly in audio and video because
systems are getting more complicated and
they are able to falsify to forge system
use here somebody with a pair of
modified glasses can trick the systems
it can mislead faces here's another
example of how to manipulate a video in
real time here you can see how the
gestures of the actor on the screen can
be changed by using this program they've
used this with many celebrities
including Schwarzenegger so I encourage
you to take a look at it then there are
other programs to replicate voices with
just a sample of 30 seconds of
somebody's voice it is difficult to tell
from the human ear what the real speech
is and what the unreal or fake one is
and this is the kind of attacks that are
becoming more and more popular now the
third type are the offensive attacks
that I think are more interesting there
are three types of attacks evasion
attacks and poisoning attacks the first
one has to do with how to evade a
classification mechanism that's already
established for example if our antivirus
classifies between malware and good work
we can make the antivirus classify
malware as good we're a good where and
the other way around when it comes to
poisoning it is about introducing poison
into the systems so that they can
perform particular actions now talking
about offensive attacks here we have
another example of a defender and an
attacker competing at the same time
there is a lot of literature around this
many from the academia having to do with
mathematical formulae well if you're
interested we can delve into that but we
want it today but the goal my point is
that there are libraries that people can
download at models in an actual way
without
reading a lot of manuals and without
being John Nash so clever hands in deep
boning are the most developed ones
particularly clever hands which I will
show you some demos now and then we also
have deep deep pounding which is
presented as the Metasploit for machine
learning it was a very promising tool
for attacks but it's true that they
haven't updates it's their github for a
few months so the project isn't dead it
is close to death anyway now I will show
you some example of some significant
attacks the most representative ones
from clever hands and then you will see
some videos are showcasing a little bit
type two attacks I will try to simplify
it with human words as much as I can so
these attacks provide two scenarios
white box and black box in white box
attacks what the attacker needs is to
know the entire training model and all
the configuration parameters this may
not be very realistic but you need to
have all that information once you have
it you need to introduce inputs into the
model and get its outputs so you know
the configuration you add inputs and
obtain outputs and with that information
you're able to generate adversarial
examples that may lead the system to
miss classify this means two things
first the ability of performing massive
attacks imagine that a classification
system can detect numbers from 0 to 9 I
input 1 9 or 1 0 and I want the system
to classify it as something different
than 0 it doesn't matter where for
example in more where I don't want it to
classified as family but as a different
thing this is what we call a massive
attack on the other hand targeted
attacks are different I introduced a
zero and I always wanted to classify it
always as two or always as a nine so
depending on the system we will be more
interested in massive or in a targeted
attack box attacks they've been widely
developed in 2017 what do they enable
they have quite an important potential
because we don't need to know anything
about the artificial intelligence system
there is a system in Google enabling a I
and I don't need to know it I don't need
to know its configuration all I need is
to make some requests that is put in
some inputs and obtain some outputs
that's perfectly feasible and these
attacks are usually against Google for
example so what we create is a
replacement model it's as if I could
create a model at home without knowing
anything about the system I want to
attack once I have my model I can do the
tests you know that you know wood inputs
I need to put into the real system in
order to generate targeted or massive
attacks that's why black box attacks are
very useful and in practical terms can
be applied to any AI we don't have a lot
of time left so I will be pretty brief
here this is an attack from clever hands
this would be a white box attack
ASIMO londo serum this is the simulation
I would like to deal with it in more
depth but we don't have time here you
have a few pictures you can see a lot of
number running numbers running three on
the Left you have some noise and on the
right you have what the classification
system would see here are several
examples and the final table that's a
summary of it all and that's the
interesting part so this this mean what
is this table I do have a scenario where
the attacker knows the model and the
configuration parameters we have inputs
and outputs and our goal is no and what
we have to do at the input in order to
make the system react the way we want
this is what that would be like for
example on top we have ten columns from
0 to 9 so this silly example classifies
numbers imagine we input 0 and we want
the system to identify 0 so we introduce
our input and leave it as it is now
imagine we input a serial but we want
the system to classify it as a number 1
so if you look to your right you will
see that that 0 that we introduced will
incorporate several alterations in order
to classify to to make it be classified
as 1 or s 2 or 3 that is we can create
this table to know which input in which
perturbation is input in order to have a
particular output what the
this means in many cyber security
environments white box attacks and black
box attacks allow you to do this and
make the classification system fail so
this is a current challenge now in
offensive security the number of
requests that you have to do to real
system needs to be as reduced as
possible for example many techniques are
able to trick Google or Amazon's
classification systems with only 800
requests with that you can create your
own model train it and if you want to
perform a targeted attack you know the
table you need to use in order to yield
a particular results here are some other
examples with the other tool I was
telling you about that hasn't evolved
much but the basic idea is the same mode
for black box and white box attacks and
if you have time I would recommend you
to read this paper on how to perform
black box attacks on artificial
intelligence system it has a great
potential because you can generalize
this to any algorithm of a I type number
4 this is a little bit of everything and
I'm going to underline just to one that
has to do with model stealing and
another one has to do with backdoors
what's been proved is that in many cases
we either know everything or try to
approach it to inputs and outputs
creating our own model in these attacks
we just tried to steal the original
model that little or Amazon is using how
can we do this there are many ways I'm
going to give you an example please
don't be scared this is a very basic
formula which is the logistic regression
formula one of the multiple algorithms
we have you know the to create
artificial intelligence how can we
attack the system and steal the model
for example the model that Amazon is
using well in this case we need to
introduce queries and obtain the
responses X&Y
we just have to replace in the equation
X plus B and with this we will obtain
number of equations for example we
create 100 queries for 1 sorry 100
equations for 100 queries these are
normal equations equations we will learn
in high school
so once you solve the equation you would
have all the configuration parameters of
this area so with M petitions input
requests and with the outputs that would
be enough to copy replicate or steal
those models models I as I said from
Amazon or from Google here is more
information if you are interested in
knowing more and the last attack I
wanted to talk to you about was the
introduction of backdoors in AI
algorithms this starts from the idea
that people to share their AI algorithms
more and more in all the that all the
people can use it now technically we can
use backdoors in different ways would we
understand my black door in this
scenario for example if we work with
neural networks it is about introducing
more of neural networks network networks
so that the system continues working
normally but it now has an additional
channel that makes the system work
otherwise this is for example useful for
the systems that classify traffic signs
if this mechanism that controls traffic
signs had a back door we could use a
small sticker or a Mac so that the car
would stop instead of accelerating this
is really serious as you cannot control
it is an additional problem here which
is called
transfer learning I will give you an
example imagine a University of Research
Center develops an algorithm that is
able to identify objects in general they
publish it on the internet and somebody
takes that algorithm and what they do is
to make some minor changes so that
instead of detecting objects in general
it can detect lumps or chairs or tables
so that this is an ideal scenario where
everybody shares their knowledge but if
there is a back door in the original
original one thanks to the transfer
learning that back door continues to be
that the back door will remain there
so to summarize you need to be careful
whose models you are using and where you
are publishing your models with which
provider is because there is the
technical capability of introducing
backdoors to make your systems work
differently so now in summary machine
learning allows us to create better
hacking tools the identification
authorization can be deceit with or
without machine learning it is possible
to steal models so we need to take care
of our privacy we can perform targeted
or massive attacks we can replicate
models without knowing anything about
the system then we also talked about the
manipulation of backdoors and models and
what's very important is that these
attacks are more and more practical and
what I mean is that in many cases the
input information needs to be slightly
modified
the system yields the results you want
this ability is growing more and more
and in terms of digital applications we
can modify by two or three percent very
small amounts
after 600 700 or 800 requests to a
provider you can actually do it just a
couple of things regarding the
formalization of offensive security well
now everyone who wants to use these
technologies and technology or in
cybersecurity in particular they they
should apply the pen testing we are
currently applying you know the to check
how secure they are in terms of
cybersecurity and for that we have
several ways the traditional one more
related to academia creating better
algorithms but then there are others
that have to do with cryptography if you
know anything about cryptography hash
algorithms etc you will know that
certain functions for example m5 in hash
an attacker can provoke changes in
results so for this we can create either
that algorithms or use another
alternative which has to do with using
several algorithms at the same time
although they are insecure why because
using more than one algorithm as it
happens in many other scenarios you may
have two different inputs and achieve
that the system doesn't classify the way
it should be here you can see x and y
and the function may work differently
this may happen with correct algorithm
for example m5 and that is quite
difficult in artificial intelligence
this is the same thing we may have
model that may be attacked with
offensive attacks but with the
combination of both and using the black
box we can create white box and black
box attacks so that our system doesn't
classify it more work correctly and this
is what we are working on achieve
solution so these are now the recipes we
would need in order to make more robust
defensive security against these attacks
we are now defining the so called
rubbish class one of the usual problems
in classification is that whenever we
have an input there is an output within
a range of possible values we have a
system that identifies numbers from 0 to
10 so the output will always be a number
from 0 to 10 and in terms of security
that doesn't make a lot of sense
the ideal situation would be that 0
would be classified as 0 and if not we
would need to have a different class
going through that same way that's why
the rubbish class has been introduced so
that things that are a little bit off
the established path going to that class
we are trying to deceive the attacker
this is a classic solution and many
companies are currently doing that but
we need to continue working so that
attackers don't gain advantage over the
defender that's why we are using deep
learning now when it comes to privacy
here we have a lot of issues many of the
references I've given you allow you to
obtain models with 93 90% accuracy and
Google in any way as it happens in
security in general we need to
understand theirs as a global
architecture if we are introduced
singing artificial intelligence in order
to provide more security we need to
analyze the security of the element
itself and be aware that the fact of
introducing AI in order to obtain a
small percentage of improvement that may
be a global problem so we need to assess
the risk and see whether to introduce
artificial intelligence so if you work
and there's or if you see people or hear
people talking sometimes when we go to
the companies we ask them this kind of
questions and they don't know how to
answer because we need to convince them
that security needs to be embedded in
that cycle at least in the initial phase
of the algorithms with clever hands or
with all the tools in order to
understand that in a real situation the
attacker will try to deceit our a system
and if the AI system is aimed at
providing security well then we are in
trouble this is what I wanted to share I
wanted to insist on the idea that AI is
useful there are many domains we've been
using it in security for about 20 years
and more where detection and traffic
controlling it's not clear what the
solution is but what is clear is that
offensive security is clearly advancing
and we need to take decisions in order
to know what to do with algorithms and
how to make them more secure so thank
you very much for your attention I will
be hanging out here I will be on Twitter
on Facebook and I'll be very glad to
answer any questions you may have thank
you very much
[Applause]