so let's let's let's move forward
cryptographic protection of machine
learning models uh as large mentioned my
name is anastasi my nickname is
vixentail you can find me online in
twitter github facebook linkedin the
other other and i am secretary of
engineer with a background of software
development so i'm kind of in between
the world of software development
applied crypto security engineering i
work at kozak labs because the clubs is
um
a uk
ukraine data security company we a
product company we create tools and
solutions for data protection some of
our tools are public free and open
source you can find them on a github
some of our tools are paid and closed
source proprietary uh we take part in a
lot of cryptographic r d like research
with novel cryptography as well as use
pretty boring standard cryptography to
solve
problems of data leakage
and
complying to regulations
okay
typically we work with companies that
value data security a lot no matter of
their industry right critical
infrastructure fintech now banking
machine learning even just popular
mobile applications all of them are good
enough for cryptography if they uh take
care of
many like take care of sensitive data
right no matter what kind of data it is
rather user passports
or machine learning models and the
second one we will talk about today
first of all the disclaimer slides okay
this is the most important slide on this
presentation things that we
won't talk about
because i know i know i know no no
machine learning is nice to roam it's
like hype
artificial intelligence machine learning
la la so we won't talk about um
adversarial networks and adversarial
attacks so we won't talk how banana
sticker creates banana into toaster you
know things like that sorry if you're
interested in that follow the link
we won't talk about machine on learning
by the way someone in twitter suggested
this topic to me and i read the paper
this is fascinating so you can basically
make the model to unlearn some things um
i really like the idea from privacy
perspective but no no tackling needs
today
read the link
number three is a malware inside machine
learning models obviously you can kind
of put some custom layers and kind of
height malware in your model we won't
talk about it today and of course in use
of the latest days uh the bug in
tensorflow
in yaml civilization we won't talk about
that today yes sorry sorry not the
topics of the day this is a
cryptographic talk
no fancy things
what we will talk is cryptography a
little bit of tensorflow a little bit of
ip protection intellectual property
protection right
we will name things like application
level encryption we will name things
like
hpka hybrid public key encryption and we
will of course speak how to integrate
cryptography into you know real world
into other traditional security controls
right we will speak about um api
protection cloud security mobile
application security uh anti-reverse
engineering
and anti-fraud
okay so if you
wanna hear about that okay don't don't
like stay stay stand youtube if you
wanna read about fancy things just
follow the links and
obviously i will share my slides
afterwards so all these links will be
available for you to explore and i
really encourage you
exploring things
let's move
what is our case for today
imagine total artificial case no name
dropping no company names imagine that
you're a large company and you work in
machine learning space and to create
mobile applications android and ios
applications that work with this machine
learning with these machine learning
models
right now you have a huge
model on your back end right on your
backhand side that serves requests so
your mobile applications make requests
so your backend your backend use machine
learning to create response and send it
back to your mobile applications however
you have really really a lot of users
millions of downloads you're pretty uh
popular in the app store and
google play market
right so you would like to decrease a
load from your server side to your
client side basically to put some pieces
of this huge machine learning module to
the client side to the application side
but this
opens totally new security challenges
and threat factors
and the case we will discuss today is
how to protect these small pieces the
small machine learning models that you
basically put out of nowhere like you
put out of your trusted back end to
these tiny mobile applications hundreds
of them thousands of them installed uh
in your huge huge user base
so
what is as i mentioned what is was
before
before your mobile application sent a
request to your backend your backend has
access to this large
full machine learning model um it
executes uh this request it generates
something
and it gives response back to the client
side every time when user i don't know
wants to see a cat picture
with
cat ears right like use a photo with cat
ears they send requests to the back end
backhand needs to do these calculations
and send response a response image back
okay repeats every time of course when
you have like hundreds and millions of
users this is a little bit complicated
so you want to change that
uh you are changing the scheme
now application send request to your
backend instead of creating a ready
response your back-end creates small
individual
machine learning module
you know and send it back to the client
side and then client-side executes this
passionate model so instead of creating
um hundreds of images on the backend you
put this load on the client side now
your client side your mobile application
is responsible to create these nice
images of people with cat ears you know
what i mean right so and these
individual machine learning models we
will call them iml during the
presentation uh they are unique per user
so you have hundreds of downloads each
download is a device each device is the
kind of user and this user can have a
lot of these models
but you don't need to make requests
every time user wants to see a picture
of them with the cat's ear
with the cat ears to the back so sounds
good
what
kind of security risks uh it opens from
business perspective this switch opens
like advantages are obvious uh you
change the load from your back end to
the client side and instead of paying
for the cloud computation you make
mobile devices of your users to do these
calculations right all good however what
is risk obviously the leakage of ip
because when you put these small machine
learning models to the client side you
basically kind of cut your large model
and you put the sensitive bits of
information for
hundreds of applications that you have
right and uh things like uh competitor
advantage if someone if your competitors
will steal those small models and if
they understand how it works they can
recreate the same model and they can
move your application rating in the
stores so you will need to do something
to
change your model to to involve machine
learning engineers
uh and of course like things like broken
applications cloned applications api
fraud so when you have an api to
generate those small individual machine
learning models this api can be abused
and we don't want that because it means
abusive infrastructure potential denial
of service potential
loss of revenue and of course
losing leaking and abusing the ib
all the things are about money
so it makes sense to invest a little bit
to protect those tiny tiny individual
machine learning models
now
how to protect them let's dig into
cryptographic uh let's dig into
technical stack that we have uh as i
mentioned we have mobile applications
and in this case these are
native nice and native mobile
applications uh objective c swift and
core male framework on ios site kotlin
and tensorflow literature work on
android style nothing fancy very native
and we have a cloud side and back end
technically this technology like
technology i will speak about the
protection of models itself is cloud
agnostic we don't care what cloud is it
but for the sake of example i will use
google cloud platform so on the back end
we will have python tensorflow and
google cloud platform services like uh
gce computation engine like workers you
know like storage like google cloud kms
different kind of databases firebase all
the nice things that google cloud can
give you
aws has the same so cloud doesn't matter
we just
it just um
just the cloud
moving to architecture
this is more or less how very very
high-level architecture might look like
let's move from right to left on this
scheme let's start from the right so
obviously in your company you have this
main tensorflow architecture where you
have this large
machine learning models where you have
some training services training servers
and we don't care about this piece at
all because this is not a part of ip you
put on the clients this is something
that is trusted in your network right
you try to protect that
next how you generate these small imls
you have some um cloud workers some
servers that probably use outer scale so
they can spawn they can
exit and they generate these small
individual machine learning models based
on some user data
based on this large tension from model
and store put those small machine
learning models into into some storage
it could be google cloud platform
storage it could be a aws s3 bucket no
matter so you generate those individual
machine models you put them into the
storage your mobile application and you
do this on request
on request your mobile application send
request hey i need a machine learning
iml for my user
to the api api triggers cloud worker
cloud workers generate imls they put it
to the storage api provides a link
devices download
the model from a link very
straightforward and then devices will
execute the model on device site so from
in terms of ip protection in terms of
protecting imls we are really interested
only in this part of the data flow right
on the left
uh the the the places where iml is
and to put it in the nice table view we
can say that okay imail
is generated on a client on the cloud
site
and it is in the memory of this worker
right then it's get uh transited to the
storage and it is stored like data
address storage on the cloud side then
it is transited to the via api to the
mobile site
right and on the mobile side it is
stored somewhere in the mobile
application
in some like file system or cache and
then of course in the mobile site it
gets executed so this model appears in
memory so basically we have this machine
learning model like everywhere in memory
in transit and storage
sounds like a cryptographic task
let's talk a little bit about threat
modeling
very very simplified okay
what are the potential threats threat
vectors and attack surfaces in this uh
infrastructure
uh
how
these imls can be steeled obviously
using api uh we are leakage we are
abusing
obviously from the storage if you can
just go and pick a link you know all
these cases with aws three buckets when
buckets are public when links are public
and anyone who knows a link or who can
enumerate all their resources on these
buckets can get access so similar when
you start something on the cloud public
on a public cloud storage
this data can be found so it's very
important to take a look on access
controls
now transit yes active men in the middle
passive man in the middle uh active man
in the middle might be really
interesting imagine that you can
you can uh check
the um
there must like you can intercept and
tamper the machine learning model from
its way through the back end to the
mobile side and change it right it would
be really cool to change the model and
instead of i don't know putting cat ears
on the user faces you can put dog ears
okay like cat nose
or
all really bad things you can put really
bad things on this machine learning
model well so we really care about
men in the middle passive or active and
of course mobile applications are a huge
part of the tax office because the
machine learning models are getting
there
they are stored there so they're subject
to reverse engineering techniques
uh
you can imagine this some kind of
crowdsourcing right if your competitors
would like to
get a lot of those models they can ask
users to send those models to you and
you can also imagine different kind of
cloned applications a broken application
or like malware applications that are
available at 4pda
which
gives gives users like nice shiny
features however uh have some malicious
code that will send those models to
their back-ends instead right so a tax
office is quite large threat vectors are
numerous and if you're talking about
stride kind of model we are mostly
interested in spoofing tampering
disclosure and denial of service and of
course cryptography won't help you a lot
with the number of service but will
definitely help you with confidentiality
integrity and authenticity
of the data of these machine learning
models
this was introduction part
and i think now you're ready
to basically dig into crypto pod
yay
so before we start discussing encryption
of machine learning models let's
think
for a second what is the machine
learning model itself
and disclaimer
i am not an email engineer okay so if
you're an engineer you probably know
things about machine learning model
better than me i am security engineer so
sorry from security perspective um
machine learning model is just a file
okay okay this doesn't look a little bit
sophisticated file because it contains
some model data it contains some like
procedure algorithm how to get this data
right
uh but from security perspective this is
just a file with some structure uh in
our case it was layered structure with
weights
right so
the file with sensitive data the data
inside file like weights is sensitive
data very simple very straightforward
yes sorry no no exciting machine
learning things here for you from
cryptographic perspective when we will
take a look on the architecture scheme
back we will see that um we can use very
naive approach to encrypt data
the machine learning model is created on
the working side on the backend side
cool
we create the model we will encrypt the
model right after creation we will store
this model encrypted across all of the
ecosystem across all the infrastructure
we want to install it in the cloud
storage okay no problem we will store it
encrypted as a file right then when
mobile application will get a link to
download this model they don't know the
model and then they they decrypt
the model very very straightforward and
you as you can as you can see that
encryption and decryption points in this
infrastructure they are different
which means
that we can use
public key encryption cryptography here
[Music]
um but before we do that let's discuss
requirements of encryption layer because
why build things that don't satisfy our
requirements
as any requirements to cryptography it
should be fast smooth cool
it should work everywhere it should be
super easy in terms of cryptography and
super easy in terms of key management
come on we are active we like
application with machine learning and
artificial intelligence we don't know
how to build a proper pki how to build a
proper public infrastructure can we
please avoid that
right uh and obviously cryptographic
protection should should solve two main
challenges to minimize
lifetime of plain text models
so to store it encrypted as much as
possible first right and the second to
avoid accumulating a lot of those small
models which means to use different keys
different encryption keys across the
models because imagine if you have these
hundreds and millions of downloads
hundreds millions of users and even more
models if you use the same encryption
key across all the machine learning
modules
well your security is not so great so we
suggest obviously to use different
encryption keys to use unique encryption
keys per model
and in terms of key management key
management this might be tricky but
cryptography we know things how to build
key hierarchies
right which i will explain soon so
what we want from cryptography
encrypt model
right after generation decrypt it only
before usage use a nice traditional
fan no fancy ciphers like is the 56 gcm
like elliptic of different helmet
fast mode and cryptography
i use the femoral keys like temporary
keys to avoid building complicated
public infrastructure and of course use
cryptographic libraries that work across
all the platform keeping in mind all
future platform you want to build
we will use stems for this purpose
let's see how um how this
architecturally around how these
encryption and decryption points will
look like in a flow in a sequence on the
mobile side mobile application generates
keypair mobile keypad
it sends public key to the backend site
right this step of the key exchange on
the backend side
back-end application workers in this
example it will generate its own
server-side keypad
all right it will take public key from
application
own private key use elliptical difficult
magic to create to derive a shared key
right
then on the back end we will generate
random
deck deck means date encryption key
random data encryption key we will take
our individual machine learning model we
will encrypt it using this
newly generated deck using is the 562cm
encrypt version learning model into
encrypted machine learning model
then
what we will do with the key because we
can generate generate this key we will
encrypt the key
right we will encrypt data encryption
key with shared key we generated on the
step 2.
we will use same i as facing gcm
and this is what we got encrypted model
encrypted data encryption key and server
public key this is our package this is
our
data structure
we send this package we store it on the
cloud side and then we send it to this
mobile application
upon receiving mobile application takes
its own private key
which was a part of a key pair
generated on step one
so it takes its own private key and
server public key from the package to
derive to use the same elliptic code if
you have one to derive the same shared
key
right then using shared key it decrypts
encrypted data encryption key right
uh and using this data encryption key it
decrypts
the machine learning model here you are
simple straightforward um ies gcm
electrical dv helmet well of course in
cryptographic details probably after
deriving shared key we will need to
use h career for something like that to
derive stronger cryptographic keys but
this real cryptography details i don't
want to dig into let's look at the code
uh let's look at the uh format show so
this is how this is how modern machine
learning model might look like or like
api response might look like simple json
you have your your encrypted machine
learning model and encrypted model right
you have your encrypted key you have
public key of the backhand side
why the key is ephemeral
because these key pairs are temporary
and
because backhand won't store it keypair
it just rescue pair it uses keypair
um
this technology like their pro which is
called called ephemeral keys or throw
away keys we generated cryptographic
keys we use them we don't need them
anymore we don't need to store them
anymore right
encryption version because this is very
important to put encryption version on
any encryption systems that you build
because you will
want to rebuild this system soon
and maybe some additional encryption
inside your machine learning model later
later
uh this is how this is how the code
might look like and honestly this is a
more or less compatible python code i
noticed it looks like a pseudocode but
it's technically i think i believe you
can compile it
uh this is all cryptographic codes that
we need if we use stamis temps is a
cryptographic library i will talk about
it in two slides so
uh the encryption part the backend part
as we previously explained
generate compare generate
data encryption key temporary write
encrypt
the model
encrypt date encryption key and send
encrypted model encrypted key and own
public key
decryption part mobile part i used swift
an example here and again this code this
is a code it's not a civic code this is
a real code it's i believe it's
compilable
right and this is all encryption coding
you will need in this case
on mobile site generate keypair
take
own private keys server public key
decrypt
your date encryption key then decrypt a
model here very straightforward very
easy
the reason why we have only like 12
lines of cryptographic code is because
we use nice tools instead of spending
time fighting with openness ssl we can
use modern boring cryptography library
and in this case we use this disclaimer
like this is an open source library
maintained by caller claps team so we
use it pretty often but of course you
can use another boring crypto libraries
like lip sodium for example really nice
they just beware because if you have a
lot of platforms to support you might
want to find libraries that work on
these platforms right because many many
many problems with cryptography come
from different apis of cryptographic
libraries across different platforms
so with stemis it's not the case because
it's a single library that works across
many platforms and it hides
cryptographic details you see in this
code you don't see these uh keywords
like ios you don't see the keyword like
elliptic uh elliptical dv helmet you
don't know anything about init vector
about nouns about key lengths keep
padding
because if you're a software engineer
you don't
need this kind of details you just want
things to work
right
boring clips are not exciting at all
and a nice thing that stamis was
actually recommended by a wasp as a
library to use mobile devices when you
make cross platform application which is
kit and which is technically our use
case here
just to to introduce a new term probably
in your life the idea that they just
explained that back-end and mobile
applications are encrypting and
decrypting better you know and data
stays encrypted across
all the other pieces of your
infrastructure this idea is called
application level encryption
right so new term
new hype works in your vocabulary
basically it's just a marketing term
right basically application level
encryption means that this is encryption
um that is kind of built into
application any application uh triggered
by application part of the application
code right obviously it can be a library
in your code it can be a separate
service right while we distinguish
application level encryption just to
um
to tell
that
there could be data at rest encryption
in your database for example there could
be data in motion encryption like tls
right and there is application level
encryption and only and these three
different types of encryption they work
on different levels so you can have all
of them at the same time and using
application level encryption means that
you probably still use tls you probably
still use data trust encryption in your
database right but now your applications
are responsible to encrypt and decrypt
data of course it could be client-side
server sides or
and
the description is in fact application
level encryption
i don't want to talk more about
application level encryption
it has different um
risks and threats and provide different
like guarantees comparing to tls
comparing to file system encryption or
database encryption and of course
end-to-end just ultimate you know
ultimate um category of application
level encryption if you're curious to
learn more please follow the article
below at infaque we wrote huge huge
article about application of encryption
and different key management problem
related to having your applications
perform cryptographic code
another term that you might like and
another term to your cryptographic
vocabulary is um hybrid public
encryption
hpka so again link in the bottom
so um
what is happening here basically this
approach this idea is not new as many
things in cryptography it's ancient
uh just the name is new because rfc
that explains hpke approach uh is quite
new and rfc
explains like tries to um standard
to make a standard tries to describe
different hpk approaches try to you know
to to make a standard
so
maybe in two or three years when we will
speak about hpke
both of us will understand what it means
because right now
not many people understand what it means
and the rfc tries to make it you know a
standard
basically uh
hybrid public key encryption means that
you combine symmetrical cryptography and
asymmetric cryptography
as we did in our example we use
symmetric cipher in our example we use
symmetric cipher
authentication encryption with
additional data right ilgcm to encrypt
machine learning model itself and then
we use um public key encryption
electrical diffie-hellman drive
shared key at the other to encrypt the
key the data encryption key right and
combining symmetric symmetric encryption
is
an approach called hpk
the rfc describes different um
different formats different ways how to
do that you can combine different
ciphers you can combine different hash
functions
and as i mentioned tries to tries to
make it a standard
so neutron for you okay let's get back
to our cryptographic scheme what in this
scheme uh what we achieved in terms of
key management the key management
another example is super lightweight
it's super easy because we use this
ephemeral key pairs we don't need a
normal strong secure pki you know
building public infrastructure is a
struggle that many companies do
especially even in security right so
in our case we didn't uh we don't build
pki we kind of have protofikai because
to be able to pin our users we can store
the public keys in the database but
this very very lightweight pikai just
storing public user keys it's not a pica
itself right and obviously as our female
as our keepers are ephemeral we don't
actually care a lot about um
key management procedures explained in
nist
yes we care about degeneration we don't
care about key rotation mostly we don't
care about pure vacation because we
create cryptographic keys we use them
and we forget about them easy you don't
need to revolve keys that exist one
minute right
uh very nice very nice approach
obviously it won't work everywhere and
the main burden of key management goes
into mobile applications because on
mobile applications they need to store
encryption keys and they have luckily
they have um keychain and crystal to
store keys and uh on ios for example we
can even bind these keys
using
secure enclave right so build proper key
storage in our bio applications uh
unfortunately this approach won't solve
the authenticity problem
i remember our men in the middle example
if someone will steal the machine
learning model and
change it our client side almost doesn't
have a way to make sure that machine
learning model came from trusted
back-end but cryptography won't help
here
we can use other security like
traditional security things like
server-side attestation and taylor
spinning to make sure that device trusts
the back end
now let's make it even more complicated
because this is just a basically right
on device site
when device
received
individual machine learning model and
the group stage a device can re-encrypt
it like application can re-encrypt the
model before storage
very simple just generate another
symmetric key and encrypt this model
before storage
why
because in this case um
this device this application won't need
access to its own private key or backend
public key
right
if this model with style is there like
for four months for example uh the
application won't need to store
all the keys requires to to decrypt this
model if it uses another symmetric key
to encrypt to re-encrypt the model of
course it will mean that we will need to
store yet another key but come on we
have keystore and keychain we can we can
do that
bonus points if we use uh biometric
binding with keychair for example whisky
chain
now next
so re-encryption on device size
uh step two in our cryptographic defense
and depth adds more cryptography to
protects cryptography
you as you can see from this example
the lifetime of plain text individual
machine
machine learning model is quite limited
right
in plain text those models live on leave
only during generation
during this decryption on mobile site
under encryption likely
and during execution
and the execution part
where mobile application will execute
machine learning model this execution
part is
complicated because most frameworks like
core ml for example on ios it requires
the machine learning file
to be in plain text right when you use
api to load the model into memory the
file should be in plain text however we
can use nice quarter mail tricks
uh to
to add
encryption inside layers
of our model
right so
all the steps before we were talking
about machine learning encryption as a
file
now
remember that these models us have
structure and they have layers
so we can create custom layers that are
encrypted
right and we can decrypt them before
loading the model to the shader
which will means cpu
load on the mobile application or we can
go even further and we can create custom
shaders
to obfuscate
wait before
executing on the gpu before executing on
the shader unfortunately gpus don't have
a lot of memory so we cannot use real
encryption to
um
to
encrypt
weights in the shader itself however you
can use different kind of discussion
techniques source
and very lightweight cryptography also
apple has really nice dopes that explain
that you can in fact encrypt the file of
machine learning model and explains how
to do that
this is like a tricky bit uh a little
bit in from machine learning perspective
right so
model file itself and try to encrypt
layers inside the machine learning
file
obviously encryption means performance
penalties as i mentioned previously gpu
shaders
have quite limited cache memory we
cannot run
typical ciphers there so we either use
obfuscation or we use very very
lightweight cryptography
in our example we use normal standard
like traditional fast cryptography with
hardware acceleration like iodcm so
we don't have crypto won't take a lot of
time the thing here is
if your mobile devices can run caramel
models with quite high
fps right so without noticeable changes
for people
it usually means that your mobile
devices can do cryptography really fast
right so again no noticeability
exchanges
and generally speaking
it might be like on the back end in
generating these individual machine
learning models encrypting them sending
them and declaring on the client side
might be faster
than
exit than the previous approach this
company used executing models on the
backup
so what we did here we encrypted the
machine learning um the machine learning
model life cycle across
like machine learning file across its
life cycle and we did our best to cover
the whole data flow
but as i mentioned previously
cryptography it's kind of boring if you
don't combine it with other
sorry with other
security controls
so remember
now we have the scheme
where we already use uh crypto on the
bottom side we already store encrypted
data on the cloud and we use crypto on
mobile sites to decrypt data
and now let's put traditional security
control there plus ports authentication
api secrets application security frauds
and reverse engineering yada yada yada
because cryptography doesn't uh work in
vacuum and doesn't exist in vacuum you
still need these traditional security
controls you still need to log you still
need to monitor you still need to
understand whether users are malicious
or not
let me talk really quickly about some of
these uh security controls
which are especially efficient working
together with scripture for example
cloud storage security we've
mentioned this previously if you have
aws
s3 buckets or if you have google cloud
platform storage
you might want to check
a wasp has a special like a wasp as a
guide a guideline how not to make
configuration mistakes when you use uh
public cloud storage obviously the
the approaches are well known
and you try to limit
life cycle of the files that you store
so you don't forget to
clean those uh files to remove those
files physically right you don't forget
to use
ttl
on the urls like access urls so the url
itself will expire really quickly in
another example basically by the way we
use ttls really small like one minute or
so so the idea was that when mobile
application make request
backhand can generate all this
individual machine learning model put it
to the cloud storage and get it back in
response
under several seconds
right so ttl
with one minute lifetime was enough
for mobile application to download the
model
obviously authentication access control
and what is very very interesting that
many cloud providers have backup
functionality out of the box
so do your best try not to backup
sensitive data
now api protection well you know all the
things how to protect api
authentication api limits request
rolling firewalling yada yada yada yada
they just check who was fire squares and
you're good to go why this is important
in this case because api was a gateway
how mobile application will get their
machine learning models right so yes the
cryptography of machine learning models
might be nice but
if you have uh api without any
protections anyone can generate like
hundreds and thousands mathematic models
for them which defeats the purpose
of this exercise
now another nice thing here is
anti-fraud system
um
think about banking anti-fraud think
about that you
want to limit
apis you want to limit
these models
for users if they misbehave so you
basically want to cut malicious uses you
want to cut suspicious uses and how to
do that is to calculate user scores
right you gather events from mobile
applications from the backend and based
on these events you calculate scores
some of these events can be a stop
factors for example if you detect that
device is jailbroken or rooted and it's
important in your case you send this
event to the backend and this is stop
factor we won't send our nice and shiny
machine learning model to this device
because it's jam broken sorry no sorry
right
if uh the remote device attestation call
failed
sorry this is a stop factor no machine
learning model for you
if uh this device uses some honey
talking some honey pot right sorry this
is probably someone tries to
to attack our nice system no machine
learning model for you
and of course these top factors and this
rules really depends on your use case
and your exact infrastructure right so
uh things that i'm explaining right now
might be relevant for your case might be
not relevant at all
right uh stop factors are very
straightforward if event happened sorry
the user is malicious however sometimes
it's complicated to say that so we have
rules like scoring rules
if
this application tries to download url
and remember we have ttl url has expired
like ages ago
we track this error
and we increase malicious scoring of the
user
if the application gets reinstalled
really often
say five times a day we increase the
malicious score if application makes too
many requests
so for every user
we can calculate a scoring model
and change behavior depending on user
scores just block malicious users limit
suspicious users and provide a nice
service for
for normal for okay users
um let's mention really quick remote
device attestation both ios and android
provides api to do remote device
attestation which means to make sure
that application was installed in
legitimate way from the app store or
play market right check this apis apple
device check or android safety net
and of course let's not dive in into
anti-reverse engineering of mobile
application just open opposed by myspace
reverse engineering parts and follow all
these nice things
that why it is important to build in in
our system because again uh mobile
applications have these machine learning
models and if someone can easily steal
them from mobile application this is bad
if someone can write and like clone the
application that will steal this modules
and send it to their server this is bad
from a male perspective we can also do
some nice things like building
watermarks like create those layers
sophisticated layers or even bind
our model
into to our application only but this is
sophisticated machine learning
engineering again i'm not a machine
learning engineer
let's get back to our scheme
the message here is that cryptography is
nice we encrypted machine learning
models as fast as far as we can and then
we added traditional security defenses
four mobile applications for api for
cloud storage for cloud workers we put
this login monitoring and our actor
system is not a black box anymore
right so encryption is an ultimate
security control and then traditional
security controls to support to work in
parallel to work independently defense
in depths
right and this is probably the last
message for my talk today
just remember when you're creating uh
any security system when you're solving
any use case even as simple as
protecting machine learning models
um failure of security control itself
it's just a question of time like
failure of cryptography is a question of
time and your responsibility is to build
a system that has a lot of the security
controls that work together because this
the failure of the whole system is how
bad your design is like how good your
design is
um as usual i put a lot of links in my
slides
and some of them i will duplicate on
this slide some of them are new here
and my name is anastasiy please
if you have any questions
check my twitter check my website all
check i work as a club site and blog
posts or white papers you write really a
lot about cryptography
that's the end thank you
thank you nicest
for the great joke
for the very nice presentation for
firing up day 2 of no name con i hope we
will have
some audience feedback
do i see it
nothing yet but experience tells me that
yeah we need to give people yeah just
just put thumbs up if you like it like
thumbs up the stream on youtube so we
will know that you watch us
if you're still with us where are thumbs
up
oh yeah there are some
quite a few
okay
are there any questions
i have questions but i would like to
hear
someone else
asking questions there is peter peter
will be allowed to talk right now peter
please go on
the morning pistons
[Music]
my question is uh
how do you see this whole
development of quantum computing on the
matter of cryptography
and the way we uh encrypt nowaday
the
systems we have and
the data we have
yeah totally not about my talk however
as you know um
we already like nist already has um a
competition for post quantum
cryptography or how we name it quantum
safe cyphers right and we have a number
of cipher candidates so cypher is that
right now i believe to be
okay
to be strong enough not to be broken by
quantum computing
right and in terms of quantum computing
not everything will be broken so with is
for example what we will need to do is
just to increase the killings however
tls might be broken and that's why many
companies right now like google like
cloudflare they are looking to decipher
to to do this um key exchange
right that will be quantum safe and i
really recommend you to read a
cloudflare blog post from previous year
i believe where they did an experiment
of two they compared to different pos
quantum key exchange algorithms in tls
to check how fast and cool they are i
will try to find the link to this blog
post and send it to you so tia uh tldr
quantum computing will be here
in not in
near future but we already prepared
the competition for the best post
quantum cipher is happening right now we
will select the best cipher will change
the libraries and only systems that will
use tls 1.2 and 1.3 are scrooge yeah
okay
that's good to know
okay do we have any more questions
guys who moderate zoom and youtube do we
have any questions there
okay i see a lot of likes on youtube so
we are really sure that audience are
listening are watching and listening to
us okay no worries i will have this
slime
i see more likes than
watchers this makes me wonder how how it
may happen
they just log in on the different
accounts you know
yeah yeah we have a very creative and
knowledgeable audience
yeah there is a question practical one
how long did it take to implement your
solution into the application
not long because as you see it's um
let's define the implementation yes the
design
took a couple of months is because of
this risk modeling like a risk and
stress modeling right understanding like
threat vectors and understanding
defenses understanding cryptographic
protocol the implementation itself
developers did it in a day they test it
in a day and then they were doing all
this role out to make sure that users
are not affected so
you've seen amount of cryptographic
codes it's teeny tiny code uh
the main things were not related to
cryptography itself the main things were
related to as a whole approach of
changing large
ml model into this small individual
machine models right but again not my
area of responsibility i'm not emotional
engineer i'm sure the team did a great
job in that
cool
there is a question in q a section let's
start with the first one
[Music]
i hope i hope you will get it because
i don't do it
entirely so
is it safe to trust users
it looks like making a wider attack
vector
oh no we don't trust users in this
example at the title outline we don't
trust users at all and we do our best to
you know to make attack surface as
narrow as possible on the user devices
we use all these anteros engineering
measures so we don't trust users that
use jailbroken devices we don't trust
users that use rooted devices we even
don't trust users this reinstall
application multiple times a day right
so we don't trust users at all in this
example and we do our best to
not trust curious users
because crowd sourcing is a really nice
attack vector here imagine this a large
competitor and you ask users to crowd
source
to you know to um to find their machine
learning models and send it to you this
is cool we don't trust users at all
yeah who does
no one else
okay it seems that
it's a rather a suggestion but i think
uh you might have quite a view on that
so it seems that homomorphic encryption
will resolve all machine learning
problems what do you think about that
i'm not a fan of homomorphic encryption
i mean the idea is called the
implementation
um is quite far from being
industry ready however you know recently
google has um
published yet another open source
repository for fully homomorphic
encryption and it's very interesting to
begin
but
yeah let's wait i'm honestly i think
that personally from my personal opinion
that post quantum cryptography ready to
use
will be here
faster than ready to use homomorphic
encryption by like fully homework
encryption but fight me this is just my
opinion
uh the question from me will it be ever
useful by mere mortals what do you think
i mean uh
it's quite a complicated
math
to move to make it usable uh fully yeah
there are some machine machine learning
models around that for instance uh
network vendors one of them we can name
because it's our sponsor cisco for
instance they have very interesting
security detection solutions for
encrypted traffic because uh it's not
like fully homomorphic
you know but there are like correlations
between how malicious traffic looks in
clear attacks and encrypted
yeah so you can like
derive some uh
knowledge and insight
from that and then
basically extrapolate this knowledge to
something that you do not know directly
and entirely but you can guess
yeah some details so it's of course it
looks more like
art than science or technology but
it seemed to work
you know
so yeah will we be there
i think what he explained is more like
um
like a traffic inspection and dlp than
homomorphic encryption itself right
so how do you find you make some you
make some derivations you know you
derive some uh properties of traffic not
decrypting it you know
so i i think it looks like something
very similar but i i provide this
example just in order to show that
it's not homomorphic encryption at all
but uh it's some kind of thermomorphic
themed
technology which uh is a dirty hack okay
let's let's let's call it the name yeah
it's a dirty hack but will we be able in
the near future to use our morphic
encryption for something provable
i think that we will stay in this dirty
hack area for a long time because these
dirty hugs already exist for searchable
encryption right so it's possible to
build such a correction without food or
more encryption again based on dirty
hugs and
one of them is this blind indexes
approach when you take this query you
calculate a blind index which is hash
and you encrypt the data and you search
by hashes right so the data stays
encrypted but you search from it but you
search using hashes so dirty hack
naughty homomorphic encryption at all
it's not a dirty first because
you know many dirty hacks have mad proof
okay
uh
another one
how fast will we have quantum computers
don't we have them already
it's not the same as quantum character
we have but
um
yeah i think alexander what's you trying
to say that quantum same quantum safe
crypto we believe that quantum safe
crypto will be good enough that quantum
computers won't be broken well not in
the nearest future yeah but right when
we will have these quantum computers we
do have them already
uh
check out ibm for example however
however the trick is that right now they
we have really small amount of qubits in
these quantum computers and many
cryptographers many researchers believe
that we need
much much larger amounts of cubies right
right now we have hundreds i believe and
we need like 100 thousands like millions
of qubits to be able to to decrypt
cryptography i really recommend you
watching a talk by jean-philippe omazon
at nonemcon last year
he was speaking about quantum computing
and post-quantum crypto quantum safe
crypto and you can find this talk on
youtube and you can find his slides and
he did a great job explaining
where we will have it well what cyphers
will suffer and what problems will
remain
good and strong
okay
thank you i think we will have the last
question
uh i have i think i have a right to ask
you one more question
uh
when we meet aliens
what kind of computers they will have
quantum computers or something similar
to to ours like from them and
i believe they're gonna have this
silicon uh life forms so they might be
computers themselves right
let's talk about artificial intelligence
let's talk about um singularity right
maybe we are building aliens right now
with all these shiny things happening in
google cloud platform with all these
codes it gets pushed there
uh you know feeding
the learning algorithms
yeah yeah
okay okay thank you again for
um
for
the great talk
and for being
the part of no name con in a new role uh
it was a great debut
congratulations thank you
thank you my pleasure have a great day
thank you everyone thank you for your
questions and let's continue with talks
bye bye of course of course thank you