see thank you so much for being here
this is a this is a fantastic panel and
I think this is the classics I content
panel because when saikhan started 10
years ago it was really the only
conference that was coming out and
talking about cyber conflict you know
not not cybersecurity or information
security but specifically to talk about
these areas of warfare of the role of
militaries in societies so to me it is
the perfect place and the perfect panel
to come together and talk about what do
we think we know about cyber conflict
and how do we know that it's a topic
which fascinates me and I've really
tried to spend a lot of time on working
now with my colleague Bob Jervis at
Columbia with a Minerva grant from the
DoD to look at the dynamics of cyber
conflict because I think it's it's so
important to look at the things that we
think we know right if you you know at
the beginning of almost every paper on
cyber conflict every book every talk
well a lot of times start out with what
we think we know and if we and if we
were gonna go around the room here to
show what we think we know we would hear
things like network speed forever
changing for a while you're here
difficulty of attribution or attack or
advantage and these are the things that
we tell ourselves about this field and
some of them are undoubtedly true like
network speed as matter fact network
speed is one of those that's probably
like physics right we can't imagine an
Internet that doesn't move at the speed
of light or that's a scale-free network
others when we say well they're
borderless or it's attacker advantage
well those are things that could change
right they may only be true about the
internet or cyber conflict today but it
might not be true in five years or 10
years or 15 years you know we've seen an
attacker advantage I'm sorry we've seen
difficulty of attribution start to
change over the last couple of years
where that thing that we thought we knew
was true is now less true and we're
having to change our theories our
operations our practices as we go around
that others are just about our state of
mind right when we say well difficult
definitions or it's it's difficult
authorities
well that has nothing to do intrinsic
with the field that's just the way that
we approach it in the way that we think
about it so with the cyber conflict
Studies Association we have an annual
state of the field conference that a
workshop that looks at these and so I
think we've really got a fantastic panel
that's going to look at some of these
issues all they'll be going through
several will be using slides although
the papers are in the book so if you
want you can flip through and read the
abstract and follow along if that is
useful to you so for the next 70 minutes
or so first we'll do the panelists are
going to introduce themselves then we'll
go for ten minutes
for each of the speakers going as the
order that they're seated and then we'll
come and have a discussion amongst
themselves as we do that I'll start
looking for hands for people to start
calling on and that's how we will we'll
finish off that with the Q&A session
I'll try and do that earlier rather than
later and so with that let's go on to
the introductions Christmas hi my name
is Christopher White I'm an assistant
professor in the Department of Homeland
Security emergency preparedness at
Virginia Commonwealth University in the
United States
I run the cybersecurity curriculum for
that school I teach broadly on cyber
conflict issues as well as international
security more broadly writ and political
risk strategies I'm I'm undertaking a
lot of research that is cyber oriented
but my main areas of interest I I
publish our research and publish on
cyber enabled information warfare some
of what I'm doing right now is focused
actually on assessing the impact of
Russian bought based activities over the
last few years in Europe and North
America as well as the role of non-state
actors in cyber conflict and more
recently applications of artificial
intelligence and I'm gonna wear in the
introductions I'm not supposed to
moderate during the day I'm
introductions but like this is this
connection of cyber backed information
operations are in the influence right
this is one of the things that we was
core to our field 20 years ago and we
moved away from it and so what we know
and how do we know it now we're having
to knit that back in it was one of the
things that we dropped and now we're
having to fold it back in commander um
my my name is commander dr. lobokoff I'm
working
cybersecurity center of the federal
armed forces my disclaimer no in the
beginning I'm not talking for the
federal armed forces here only from a
research a spective so I do my research
at the University of the federal armed
forces in Munich and also giving
lectures at the University of Bonn
working in the area of cyber security
and from a military point of view I'm
building up penetration testing
capabilities at the Forces at the moment
hi my name is Martin libecki I teach
Midshipmen that makes me a government
worker which means the usual caveats
apply and you can google everything else
all right
with you okay great well thank you as
well thank you all for being here I'm
presenting paper that can be found in
conference proceedings I will likely
brush over a number of things and so
please do go ahead and read the paper
itself and should also know it upfront
that this is part of a collaboration so
I have three co-authors on the paper
Brandon valeriano who's sitting right
there please direct complaints to him
Benjamin Jensen they're both professors
at Marine Corps University and Ryan
manis who is the Naval Postgraduate Air
California the lucky devil and so this
paper is a part of a broader set of
projects that were engaged and focusing
on developing better resources for
scholarly research into cyber conflict
issues so the paper we've written there
the paper we've written for this venue
rests quite simply on two sets of
concerns that we have about the conduct
of cyber conflict research as it happens
right now and his concerns I'm sure
they're chaired by many of you here
first and foremost what we're concerned
about is a seeming lack of general data
resources and opportunities available
for performing research for scholars and
security practitioners that are
interested in developing better
understanding of cyber conflicts the
utility of different forms of cyber
aggression and so on so I don't want to
kind of start by overstating things it's
true that you know two or three decades
ago social scientists weren't doing too
much in cyber conflict the past 10 to 15
years that's not being so much the case
there's been a lot of excellent research
a lot of new programs and a lot of new
resources have been made available to
I'm political scientist so political
scientists particularly wanting to study
cyber conflict but at this time what
we're concerned about is the fact that
few comprehensive data resources
actually exist the use of which might
allow researchers quantitatively at
least to generate inference in theory
about when and why cyber conflict occurs
and I'm saying few resources and the
truth of the matter is that incident
data event data which is the you know
bread-and-butter of a lot of political
science research on international
security issues incident data on cyber
conflict really doesn't exist is found
in a few places but even in those places
the most prominent major databases the
study conflict issues broadly read they
tend to leave out major cyber incidents
and they certainly don't do things like
link different parts of sophisticated
8pt campaigns over a series of years in
fact to our knowledge this is a tad
self-serving but there's only one data
set that's really emerged over the past
half decade or so this really looked at
cyber conflict in a detailed and
replicable fashion and that's the the
dyadic cyber incident dispute data set
which full disclaimer was developed by
brandonandryan two of my co-authors on
this piece but even with that data set
as my co-authors freely admit that
effort was designed to be an initial
foray and it specifically selects on
certain kinds of rival of rival s
relationships and so we're still left at
this point in time basically being a
field that lacks the kind of data
resources that are commonly found in
other conflict and crime research
programs like on terrorism extremism
piracy militarized interstate dispute
and more so why is this the case why
don't we have these resources right now
you've seen this kind of slide a million
times I know the reason we don't have
these data resources is largely because
of the second set of concerns we have
and these concerns are frankly just the
most common feature of methodological
debates on cyber complet issues so
simply put we face significant
ontological and epistemological
challenges I'm not gonna go any deeper
on that really but if our findings are
our findings and research are constantly
coming from limited observations how can
we ever really create generalizable
knowledge about this subject and even if
we solve problems about how to get good
data if the data is
in any way then how can we actually
ensure that we know what we know how can
we guarantee our descriptions of when
and why cyber conflict occurs and so
these challenges as you know - well
manifests in unique ways when it comes
to cyber conflict research mostly the
attribution of cyber events is not
almost feasible and it's true in a
couple of fronts so the method is covert
is the main one I think worth noting it
simply isn't possible we'll always be
able to observe every cyber event every
output of a cyber conflict action often
events go unnoticed by their victims
attempt to actually intrude fail and
don't get noticed by the victim and of
course victims and stakeholders that
might give us good data are often
motivated by reputation or financial or
political costs - they fail to report
that attack has taken place as a result
and moreover the way that the domain
actually functions again as you all know
means that we can't easily project the
frequency of otherwise invisible events
in the way that scholars in terrorism
studies or extremism studies might so
even where indicators suggest the
existence of a cyber conflict incident
again validation usually requires the
cooperation of either victims or
infrastructural stakeholders commercial
vendors backbone operators web service
providers etc and so adding to all of
these are due challenges of bounding
scope and controlling for negative cases
we talk about this more extensively in
paper I'll generally put that to one
side for now but briefly successful
cyber operations might include thousands
of individual intrusions how can we
bound those how can we think about those
events without the benefit of for
knowledge and with negative cases what
we have to do is acknowledge the fact
that only successful intrusions or
significant attempts tend to get
reported in the public sphere so how can
we as scholars actually assess the full
scope of the domain if we're not
possessed of the full breadth of
information that should be made
available to us or we'd like to be made
available to us so our effort here is
partly an effort to address these
pervasive concerns about what's possible
cyber conflict research and partly an
effort to demonstrate the efficacy of
large-scale data collection large-scale
approaches to parsing and curating
usable OSINT information open source
information of cyber conflict from the
volumes of information we have before
in the public sphere so our main
criticism that we levy in this paper of
those who argue that much covert action
just fails to generate observable data
points is that this problem
fundamentally misunderstands what we're
talking about when we're talking about
covert action in cyberspace covert
action simply implies difficulty in
determining responsibility or the
existence of an action but not that an
action actually took place and so beyond
this basic rebuttal we make a few other
arguments that I just want to go through
very quickly about why why researchers
should attempt to undertake open-sourced
collection of information and cyber
conflict trends so the first of these is
quite simply that the field is in dire
need of an extensive base of core
knowledge and assumptions or theoretical
knowledge about how cyber conflicts
manifests and what it looks like there's
a whole political science and social
science thing that go into here but the
development of research programs there's
a whole hot set of debates over it but
the general idea and the general point
of consensus by social scientists is
their research programs are layered
basis of knowledge with peripheral
hypotheses on the outside linked to core
assumptions and we assess these
peripheral hypotheses with the aim of
advancing the state of a given field we
don't actually have that much yet with
cyber conflict studies this is a field
still in its infancy
I mean arguably maybe just out of its
infancy but the condition of general
core knowledge at the heart of the
research program remains remarkably
unclear we also argue that the use of
public facing information abayas to
attempt to construct comprehensive data
on cyber conflict issues is necessary
because otherwise we're going to
struggle to link knowledge of cyber
incidents to broader socio-political
context this is particularly as Jay just
mentioned with the link between cyber
conflicts and information warfare
important now when we're starting to
acknowledge once again the cyber
conflict is not a discrete phenomenon it
is all other forms of conflict
manifesting in unique and new ways and
we also regardless of the criticism that
some actions won't be captured we argue
that there are clear benefits from
public facing data collection on cyber
conflicts in a large scale specifically
because it's being collected at scale so
shifting to machine code
schema the way that terrorism studies
and extremism studies scholars have that
collects disparate sources from all
manner of source so social media
traditional news media reporting
government and industry documentation
and so on
is good for our analytic purposes these
combined observations even if the remain
imperfect our orders of magnitude better
than any one case reporting line so and
very briefly excite I want to go over my
time here to demonstrate the utility of
an approach that we're suggesting here
we supplement our arguments in the paper
by applying a comma Patrick - it's a
calm as I commonly said you said of
Python programs the basically treat text
produces event data - and we apply this
to a corpus of several hundred thousand
news articles that were gathered and
curated to focus on cyber conflict
issues in the United States over a
two-year period and the idea for us here
was essentially to try and replicate
past efforts in in this case we were
trying to replicate the dynamic the dyad
excuse me cyber incident data set that
brandonandryan had previously produced
just very briefly our results we focused
on American experiences with cyber
conflict in 2013 and 2014 and our
results were quite promising so just
briefly let me walk you through a couple
of these what you're seeing here is the
raw count of incidents we found
involving the United States
over that two year period so 512 major
cyber incidents that we found 279
directly involved two United States 213
of those that we specifically had the
mission code involved the government
directly in some fashion this as many of
you know actually this busy period right
here between marsh and Alliance 2014
this contains several well-known
incidents that involve US government
agencies and major companies this is the
target hack the OPM hack and so on our
approach also automatically treats text
to annotate specific cyber incidents for
different techniques used used and
different perpetrators so we can kind of
see here is that while a lot of what is
reported on the public sphere ends up
being labeled as a cyber attack we find
most evidence of incidents actually
taking place when it's your
porta Don in the form of state agencies
attributing an attack or trying to
arrest a suspect that itself I think is
an interesting point worthy of further
discussion on the data collection front
we also on the attribution side of
things naturally find that substantial
number of incidents we found they're not
linked when you're coding in this way to
a particular aggressor but we also kind
of under surprisingly find a lot of what
happens or a lot at least what a lot of
what happened in this period we're
linked to Russia China and well known
through countries Moldova Malaysia and
so on we also see evidence of actors
specific usage of cyber inch from
instruments in line with established
knowledge so Russia for instance is most
often linked with malware employments
rather than directed degra davut acts
and most importantly just finally what
we did was we compared our own effort to
that of this data set that came before
us in the same period of time that data
set found 21 major cyber campaigns and
we found 14 of those so that's about a
65 percent success rate and so given
that this was a relatively small
demonstration on our part we think this
is pretty good clearly promising
particularly because several of these
that we didn't catch information on them
was only released the year following
that we had information input
information for and so let me just
finish up right there I would finish up
just by saying that obviously no data
collection approach is perfect I'm
looking forward to discussing it there
are remarkable issues even with using
the tools that we're suggesting
political scientists social scientists
need to start using the main drawback to
using what we've advocated for here is
that a lot of these tools are dumb right
they're remarkable they're very useful
but it still relies on on appropriate
research design on others not trying to
are not baking bias into design and the
implementation processes and so given
the kind of data we need to produce
better context in this field I think we
got work cut out for us over the next
few years right thank you very much
[Applause]
ok when I was thinking about the tense
event of the successful conference last
year I thought it's time to have a look
at what happened in the cyber domain and
what can we see and how to interpret it
so everybody very well knows the origin
the DDoS attacks on Estonia in 2007 that
eventually led to the foundation of the
CCD Co II and this conference and that's
quite a time ago DDoS is a very trivial
attack mean but having a look nearly 10
years later so in 2016 for example the
DDoS attacks on doing were able to show
nearly same consequence and so hundreds
of vendors were hit by that indirectly
because it was on the an attack on the
DNS system not the vendors itself but
because of that millions of users had
access to for example Amazon or Twitter
or other services in the United States
having a look also a little bit on the
net losses that's very difficult to
estimate but for example a recent study
by McAfee said that we have about 600
billions of dollars worldwide of net
loss and losses based on cybercrime so
it seems that there is not a really good
improvement in the security area there
so we took a look on some significant
cyber incident and of course is quite
important to talk about what is the
significant cyber incident because there
are a lot of statistics out there where
even a ping or a scan of a system is
counted as an attack on a system and
that's not really helpful you can't work
with the data so if you have a look for
example to the statistics from the
Center for strategy and international
status
they offered that the first time I think
in 2006 I started with 2007 because of
these 10 years and you can see that the
significant cyber incidents are
definitely on a big rise especially
during the past years so we took a look
a little bit closer to why our
cybersecurity measures obviously still
fail and what's going on there in the
cyber domain in the heart of the
publications so please read the paper we
have this timeline we picked up a lot of
these incidents and identified different
and different relations and conclusions
for example
what about reveal attacks sophisticated
attacks or other things like for example
preparing the battlefield I will talk
about that in a minute and what are the
connections so definitely not the time
to talk about everything that we
identified but giving you hopefully a
little bit appetite and to read the
paper when talking about what we know of
course it's quite important to realize
what we don't know in the area of cyber
security very well-known are the common
vulnerabilities and exposures so a
database where a lot of the known
vulnerabilities are in so in real world
yo kanga always scan your system for
this world winner abilities close these
vulnerabilities and hopefully you have a
much better security level but the
interesting question is how good is this
security level and the real world these
examples are the vulnerabilities for the
Microsoft Windows operating systems over
a couple of years but if you go into the
details of the things that is within
these statistics you can identify that
the publication's of vulnerabilities
strongly depend on the research focus of
some companies and some single persons
in the research field of the security
area so only having a look in
the statistics doesn't give you a
picture or a good good mindset on the
security of your system because you
can't close down everything that's found
and in the in the database but you
haven't any idea what's really left with
vulnerabilities in some of the systems
one of the things we identified
I call it preparation of the battlefield
and I'm highlighting only a few aspects
during the past 10 years not going of
course into the details some of these
events we could see a quite obvious a
preparation of the battlefield others
are not so please have a look to the
publication we describe why we are the
mind that this is all was in the
preparation of the battlefield only
highlighting of you so starting with
configure everybody will remember this
quite famous warm because it was very
aggressive and quite difficult to catch
it has a number of versions and really
learning from the defenders how to
improve the attack but the important
thing about that only the last version
that deleted itself after about one
month had a payload with it so you can
see ok someone more tried out something
if it's really working in the real world
and of course having a look at the
events in the Ukraine a few years later
so the blackouts in December 2015 and
2016 that's even more than the course so
especially the second one in December
2016 where the power outage only
affected another part of Kiev it's quite
obvious that somebody tried out if
something is working but it was not
planned as the real broad attack that
should disable the complete power grid
the chances with malicious software were
quite good to do that but you can really
say ok some players are at the moment
using the Ukraine as a test field to
identify what's possible and what's not
so we collected a few other things and
of course very important for the
preparation of the battlefield are some
actions of hacking industrial control
systems and critical infrastructures not
only in the UK but throughout Europe and
in some of these events we could find
installed backdoors and something like
that so where the attackers prepared two
systems for later access because there
are a lot of bad news I included a slide
with good news during the past years
already and was a outlook to the near
future there are quite some technologies
which can be used to improve the
cybersecurity arena a lot but as always
there is no Holy Grail so sometimes
especially blockchain during the past
two years it's named everywhere but like
in every case you have to identify where
can I use it and what for it's not the
holy grail for everything and improves
the security of everything
they are challenges for example also on
the quantum area these computers can do
if they have finally arrived a lot for
cybersecurity but not everything so you
really have to go to the details how to
affect the cyber arena was that so
having a look to the next 10 years what
might happen the super exponential
growth rate of IT improvements that's
definitely the case there are some good
revelations highlighting that and that's
an important case because what we are
seeing nowadays that's not what
happening in the labs that is the
technology from the past 10 years that's
now arriving on on the mass market but
there's much more coming on and we have
to deal with that for example we will
have practically unlimited storage quite
soon for example you can store about 250
in petabyte of data within one cubic
millimeter of the
and if you follow the news Microsoft
announced that they will prevent some
sorry provide some DNA storage by 2020
in the other cloud so that's only two
years ago but will not be a full-blown
system more a demonstrator to show the
capabilities but you see how much and
how fast of these new technologies are
arriving Quan and supremacy that can be
quite soon Google announced the new
processor were only a few weeks ago and
within that context they said they may
reach quantum supremacy so when we have
machines in that area that are more
powerful than the most powerful
traditional systems in that area so
solving new problems that may arrive
even in a few months and of course with
a look at the military forces a lot of
systems are completely dependent on IT
nowadays and that will improve top
approaches for example did great work on
excel exoskeletons or a hair for example
look at erasing memories of traumas from
the brain so something like men in black
you come with the flashing device and
erase something there interesting
research works out there for that or the
other way around
matrix loading some knowledge up to the
brain also there are some amazing Java
programs running at the moment so in a
few years you will have these
technologies and that will severely have
implications for our thinking about
cyber cyber domain self.x technologists
and the power of I am NOT going into the
details with that now so in the end also
some recommendations especially from a
military point of view to be to have
assertiveness on the battlefield
nowadays we are forced to use
network-centric warfare and autonomous
systems and we have the complexity rep
that was also mentioned quite well in
the morning so
my recommendation was in that context is
not to focus on the billion-dollar very
heavy and running for decayed systems we
have nowadays but building
cost-effective systems specifically for
a mission which are produced on time by
additive product and production methods
for example sorry and 3d and 4d printing
using AI and one behavior of course
there are a lot of risks too as you
surely we well aware when you visited
the respective session today for weapon
systems in crypto we need verifiable
secure designs which are already there
but in a lot of cases we don't apply
them which is quite a risk for example
there are verified micro plans out there
but often you don't find them but ballon
full-blown operating systems even in our
military systems and and very important
aspect that was also challenged by Bruce
Schneier and morning the preparation of
the battlefield is happening at the
moment the backdoors are included
introduced in the systems at the moment
so it's very important to do more for
the security about the supply chain and
a last aspect for disruptive
technologies we still don't know if the
quantum computer finally will work the
general one chances are very good but
future systems must be very very
adaptive because we are working since a
few years on post quantum cryptic and
cryptography and things like that
but this is a very very young field so
even there are good proposals for
algorithms which should resist quantum
technologies we don't know if they
really will so future systems must be
able to adapt very fast you must be able
to exchange algorithms in a very fast
mean and of course we need quite
flexible structures and organization to
realize that and that's quite
challenging especially for government
institutions
there are a lot of more details in the
paper so I will be very happy if you
have a look to that and come back with
any questions about the topics in there
later to me thank you very much good
afternoon we've had two days of fairly
heavy discussions about cyber things we
had two excellent presentations this
morning that if correctly understood
should give you fairly depressed so what
I want to do is lighten things up a bit
and start my talk with an excursion into
thermonuclear warfare take the wayback
machine go ahead it has fans
so take the wayback machine to 1983 a
time of high tension between the United
States and Soviet Union and the Soviet
military officer named stanislav petrov
was manning the nuclear desk at the time
and all of a sudden he was getting an
incoming message that the soviet union
was under cyberattack from the United
States best I understand the satellite
had reportedly picked up a number of
missiles coming in to Soviet Union well
the standard course of events in that
case was to fire everything back at the
United States whereupon the United
States would have fired its surviving
weapons and to the Soviet Union and
let's say that things wouldn't have
ended well but the reason that were
alive today is because officer Petrov
took a look at the what he was getting
his input and he said I don't think this
is a true indication of an attack I
don't think the satellite is reading it
correctly and one of the reasons he came
to the conclusion was because the Soviet
military doctrine had taught him that if
the United States was Garan carry out a
thermonuclear attack it was gonna come
at the Soviet Union with lots and lots
of stuff the satellite had only
indicated a mere handful of stuff okay
so he came to the conclusion that in
fact if there Moe nuclear war had not
begun that it was a false alarm
the Soviet Union then didn't do anything
today that didn't do anything the night
they said no anything and we had a happy
ending in so far until of course you get
to the cyber conference and then we're
depressed all over again but in any case
one of the aspects of kinetic world is
that if you have thousands of targets
you need thousands of weapons because
this is sort of an aegis thing going on
here and if you're looking at a
situation in which you don't have
thousands of weapons chances are they're
not going for thousands of targets it's
probably not an existential issue as you
can probably figure it out about ten
seconds ahead of me this isn't true in
cyberspace
it isn't true in cyberspace for a lot of
various reasons one reason isn't true is
because often in military operations and
a lot of different operations one piece
of information feeds thousands of
decisions if you can corrupt the one
piece of information you can also
corrupt the thousands of decisions
another reason which was noted by one of
the morning speakers is that if there's
something wrong with a particular item
you own because it's wrong in its coding
if there's a vulnerability in that item
there is a vulnerability in every single
item of that type so that a single piece
of malware which by the way is also
capable of very rapid replication can in
fact take down everything at once and
that would normally make people very
very nervous I mean just to give to
illustration let us say Oh random
profession that you're running a nuclear
centrifuge farm and somebody says hey we
got problems
I discovered a piece of malware and you
don't look at the guy and you say what
do you mean you discovered a piece of
malware that's one piece of malware well
you have thousands of centrifuges what
are you worried about what clearly it
doesn't work that way and because it
doesn't work that way in many cases when
you have a cyber threat to your military
systems you're in a position to be a lot
twitch here then if you're operating in
the kinetic world and you need thousands
of discrete threats to your systems in
order to take it seriously and I'm gonna
switch gears ever so slightly
and essentially talk about a canonical
challenge in kinetic operations and that
is there's a there's a scenario a fairly
common planning scenario that basically
says if you're going to have a
large-scale war it's going to start with
a cyber bolt out of the blue which is
going to basically make your systems
much harder to use make them unusable
makes them unreliable and then the
initial period of confusion is one in
which there's going to be a great deal
of advantage to the attacker to the
cyber attacker because the fed offenses
will be D fuddled okay now let's take
that as a notion and I'm gonna get to
the key question here the key question
here is what if I find something in my
system that has somebody else's
fingerprints on it one of the attributes
of cyberspace that makes us a unique
problem is that at most stages of
discovery an intrusion for the purpose
of cyber espionage is indistinguishable
from an intrusion for the purpose of a
cyber attack so if you discover
something in your system that comes from
a hostile other party what do you assume
about the party's intentions okay
conversely if you're a country trying to
carry out cyber espionage on another
country's military systems what do you
do to make the other side not react in
an unfavorable way okay here's an
instability problem that is exacerbated
not uniquely in a cyberspace but
exacerbated by the particular
characteristics of cyberspace I don't
have a good answer to the question but
like a good academic I've got six
alternatives so let's go through them
and order first alternative ignore the
problem okay this is called the null
hypothesis in real life it's the one
chosen most of the time okay even though
it never has the best cost-benefit
analysis it's almost always chosen by
default
okay so let's forget that one
Oh item number to do cyber-espionage
better so you don't get caught that's a
wonderful piece of advice it comes into
the category of nice work if you can get
it presumably
your folks are doing their
cyber-espionage as efficiently and as
stealthily as possible but there's a big
difference between hoping you don't get
caught and being certain that you don't
get caught so yeah you want to do that
but that's not necessarily gonna solve
your problem
option number three don't do any cyber
espionage at all okay and in fact I've
given you a reason to be very very
cautious about doing cyber espionage
against military systems for fear that
they might be misconstrued okay and now
I'm gonna say something that some of you
may not expect cyber espionage is often
a stabilizing element you will hear a
lot of people from the intelligence
community and there's a history to
support it that knowledge of the other
side is an important element in keeping
your own site from going off half-cocked
because you think there's a missile gap
or a bomb or gap or all that sort of
stuff more often than not the more you
know about the other side the easier it
is to suppress worst-case situations and
in fact there's another element here
that's stabilizing one of the elements
of instability in the international
system is surprise attack okay if the
other side thinks that you have eyeballs
into a system the other side may think
that they are not capable of carrying
out a surprise attack if they're not
capable of carrying out surprise attack
that in and of itself is stabilizing and
in fact if you want to ask me some
nuclear questions are a couple of
interesting ramifications here but I
continue item number four and at this
particular case I am speculating
emphasis on the word speculating here
okay try to carry out your cyber
espionage in ways that don't make it
look like preparations for a cyber
attack and two of the speculative
suggestions I have is that stick to the
communications elements don't try to go
after clients and servers you don't need
a present
son but make sure that if people
discover it they can tell themselves oh
he's just eavesdropping he's not trying
to make things go you know work funny
and the second item is to try to steal
credentials rather than using persistent
malware okay but like I said these are
speculative item number five a lot
easier to do in theory than in practice
one of the problems with discovering an
implant in your system is knowing when
the implant was put in because that
allows you to make inferences about the
purpose of the implant okay here's the
paradox if I'm in a crisis
I am apt to scrub my systems more
diligently the more diligently I scrub
my systems the more likely it is I'll
find something that I just as soon not
want to see right and the more likely it
is therefore that those discoveries of
implants will be correlated with crises
and if the correlated with crises
they're going to be meant as specific to
the crisis so paradoxically if you put
an implant into somebody systems you
want some sort of indication about when
you put in the implant as a way of
calming the inferences that other people
are going to make about your system okay
and I finally going to end with a
counterintuitive notion for how to deal
with the problem if the problem is that
cyber espionage is going to be inferred
as leading to cyber attack do a lot more
cyber espionage that does not culminate
in cyber attack and after a while people
will get used to it and they won't panic
great thank you very much Martin at all
and thanks for keeping clear time I'm
gonna pick up on that last point just
because it's the one that you you left
us on because it's one of these
differences you know in getting back to
the themes what do we know and how do we
know it I had seen your folks at it Fort
Meade say to me well espionage can't be
escalatory because everybody does it and
therefore it's something that we don't
have to worry about because they because
it won't be destabilizing rescue Latorre
but i but as you pointed out right as
since as general Hayden puts that attack
is a lesser included form of espionage
then right then it can be this confusing
and and if so if we have these habits of
saying or espionage can't be that
escalatory it might lead us into these
dangerous kinds of conclusions so I'm
really curious about your thoughts on
the panel about what are some of these
areas where maybe our instincts are of
where cyber conflict and conflict are
diverging and maybe it gets us into
trouble but I also don't mind if one
where we think they are more different
that cyber is oh it's so crazy different
when it actually is more similar to
conflict well let me make two
observations here about the Cuban
Missile Crisis one of the reasons the
Cuban Missile Crisis is wrapped up the
way it was that the both Kennedy and
Khrushchev thought that something would
get out of hand and there would be an
incident that would get out of hand
there were three incidents two of them
were about spy planes one of them when a
US spy plane looked like was gonna
wander in the Siberian may in fact have
wandered in in Siberia and the second
one when a spy plane was shot down so
this notion that spying doesn't
implicate you in conflict isn't even
true in the nuclear realm much less the
cyber realm
yeah and I'm very cautious that this is
this area of miscalculation especially
and we're going to talk about this in a
one of our future questions Cyber
Command just issued their new fusion
where they talked about persistent
engagement and that they need more
agility to engage with the adversaries
and so I'm really interested the
president might be talking with a
counter partner in a sensitive
negotiation and Cyber Command is off
doing things like Cuban Missile Crisis
drop
depth charges practice depth charges on
Soviet submarines that's sending one
signal that might be very different from
one the president is expecting and so
I'm really curious about this
miscalculation and I don't know if
either of you have any of these points
of where we might be closer to conflict
than we think or the places where we
might be farther farther away from our
instincts so with just one brief
clarifying point I would add to that is
you have another dynamic that you have
with say cyber weapon or that you don't
have with nuclear weapons which is that
authorization need not necessarily come
from on high right executive
authorization or even high-level
military authorization is not actually
consistently required for every
individual intrusion particularly if
intrusions are automated after a certain
point in time in - that just frustrates
the problem even further where if you
can't distinguish between espionage and
attack because of course when we're
talking about cyberattack we're
inevitably I mean just think of the kill
chain whether or not you're a fan of it
most points in there are exploited of
points right up to kind of complete the
mission do whatever the actual attack
function is if they're interesting we
should will
the fact that it could be somebody in a
control room authorizing what they think
to be a low-level interaction because
let's say let's say you are in a set of
negotiations or potentially a standoff
with another country you may think it's
actually the norm that you that your
opponent is expecting you to attack
their assistant for reconnaissance
purposes at an increased rate and so you
may be more willing again not at a high
level - okay these operations mm-hmm is
really addressing anything well that
operations can be quite similar to a
regular operations and but we just have
to learn not to demystify the cyber
arena a little bit but I think there are
also quite big challenges because other
domains are strongly dependent on cyber
nowadays and you can do a cyberattack
well in advance even not doing any
hands-on when real conflict arrives so
for example you can put in some sleeper
functionality a navigational system
would be a perfect example that just
disable system when it's going through a
specific geographical point and then you
don't have to do any hands-on that can
be detected or recognized so that's an
second aspect that is completely
different on the other hand now both
especially you Robert you had but when
you're looking your timeline you had it
in an arrow and the arrow was going up
at an angle and you might have done that
just because that fit on the slide more
easily but and Chris's was similar but I
mean they have been getting or I mean it
seems that every year has been getting
worse than then each year and other
speakers have hinted at that and that's
been my experience the one I'm curious
to all the panelists agree that each
year in general seems to have been
getting worse that the scale and scope
of incidents year by year has been
getting worse anyone disagree with that
spams getting better yes in some areas
fam has been getting better the but the
the intensity of cyber conflicts today
seems to be blowing past places that we
thought we would have been at an
adversity of cyber conflict as well
mercy of conflict that we're not calling
cyber but that an actual five looks like
something else right yeah the and so why
do we think this is getting worse you
know in many other kinds of conflict you
can see trends right you can see
terrorism might be getting you know we
can see the trends go down as well as up
so what is it about cyber conflict is
this just a period of cyber conflict
wonderin where it's where it's getting
worse and worse and worse I think was
Thomas ed in the morning was his great
speech is a key aspect because we have a
lot of new tools out there which are
really helpful and can improve the
security in the area where we were much
but on the other hand economic driven
reasons putting much more risk on a much
faster pace into the cyber domain as
vice versa you know I'm not sure I agree
with the assertion it could be a bit of
balderdash but what I wanted to note is
there's a tendency sometimes for
problems to go from acute to chronic
if you're just doing cow accounting it
looks like it's growing bigger because
there's more chronic than there is acute
yeah okay but it's entirely possible
that looking back from the vantage point
of ten years ago we will conclude that
the peak for cyberwar was in fact around
2010 okay because you had operation
orchard in which cyberwar techniques
were used to disable a Naiad system you
had Stuxnet you had a DDoS of this
lovely country okay which on a nation
scale has not been replicated since you
have a lot of people who were
experimenting with various notions of
how to use cyber and took a look at the
experiments and say now okay now it may
be too soon so if it in ten years from
now I'm totally wrong and you didn't
hear it from me and you gonna bomb right
you did here you had Stuxnet even though
it hadn't been discovered yet sis mm
yeah yeah yeah that's very that's an
interesting cake any other thoughts on
maybe why it's I'm I'm tempted as well
to say that your zero is I think someone
called it in 2010 or 2011 might we might
look back and see that as being the the
most intense year of cyber conflict in
history you know looking back 10 years
from now I do think that we've seen
again this diversification of what we're
calling cyber conflicts cyber-enabled
conflict of some other form and I think
the reason for that and this is not
necessarily the same thing as saying the
cyber conflict is getting worse in fact
not at all I think the reason for that
might be that you know we're certainly
not you know I'm gonna quote somebody
from King's College here I'm sure
they've said in this conference already
the nature of war isn't changing the
character of war is constantly changing
at this point in time and in actual fact
maybe I was thinking that cyber conflict
is getting worse is us simply seeing the
character of war changing in a much more
fluid fashion over the past decade then
we have up this point I think that the
degree to which information technologies
are embedded in societal functions
throughout history as well I mean you
can go back and look at you know the
printing press or telephones or
Telegraph's or what-have-you but the
degree to which ICT get a
bedded in societal functions determine
the degree to which ICT altered the
nature of warfare and what we've seen
over the past decade is the
proliferation of different forms of
networking naval technologies across the
side of functions that's why this most
recent information revolution and then
the kind of miniature ones you know the
development of social media platforms
and their adoption across the globe have
been so meaningful in changing the
character of war in a much more
consistent fashion over the past 10
years or so yeah and I think there's
really fascinating in both and I've
really seen it this week where I saw one
very senior European General say no war
is always about terrain terrain terrain
terrain and you couldn't get another
word you know it was about terrain and
but we also saw general Nellore here
General Petraeus into this earlier in
the week of you might be seeing a
fundamental nature change in the nature
of war and you know generals don't you
know talking about a fundamental change
in the nature of war is they don't use
that lightly you know especially
especially the Commandant of the Marine
the Marine Corps and so I think that
would be very interesting to see are we
seeing just slight changes or more
massive changes all right I'm gonna
start looking around for hands ok yeah
we've got Melissa here and then we're
gonna go a second here and third here I
don't yeah Melissa and Kim you put your
hand up yep hi my name is Melissa
Griffith I'm from UC Berkeley in
California and my question goes back to
the first presentation we had around the
types of data we collect and the kind of
reliability of the inferences we draw
from that data but we also know that
researchers face a pretty serious
problem with non public facing data
specifically because of how close in
proximity we are to the changes were
observing and have a question about as
researchers how cautious we should be
about the types of inference as we draw
around certain types of questions on
cyber conflict and if you worry about
whether or not our results be
substantively different on certain types
of inquiries if we had access to this
whole other space of data
yeah absolutely we should be extremely
cautious about the inference we draw
from data the likes of which we're kind
of advocating needs to be generated this
field because of the attributes of the
domain is so unique that I actually and
I've advocated this in other published
work that I actually think the the
institutional format the collaborative
field level community level scholarly
community level that is format should be
different here as opposed to and than
what it is in other areas for one thing
I think we need to adopt a micro
foundational approach in order to
validating the assumptions we generate
from these kinds of datasets if we have
a book and I actually show for Brandon's
book here that he just launched a book
cyber strategy that uses this kind of
data to come to certain conclusions they
have certain findings we should not
accept that as as taken we should adopt
a rigorous process and I'd like it if we
had journals in the field that were
devoted to this to be honest with you
where we replicated across a number of
different formats those results because
you're absolutely right we need to be
extremely cautious about data generated
in this fashion it's I think we can
generate good data it will not be
representative data obviously though and
I also think with regards to the high
side of things the classified side of
things again I think we would do well in
the field to have a kind of
collaborative representation that does
not exist in other fields such that we
can actually have conversations with
those in the IC those with your IC
counterparts in Europe to validate our
inferences with what they're actually
coming up with with frankly probably
better data resources and we can be
useful to them as well here there's a
lot that the IC and governments could
actually be said to do if they were able
to use more robust more replicable and
more interesting frankly data resources
that exist in the public sphere that
generate results that they found on the
high side but because of classification
issues they just can't talk about right
they could warn us about certain things
perhaps yeah and it really strikes me
the other way
that were starting to ask the
interesting questions like you know the
work that Jacqui Schneider has been
doing and others of saying alright let's
use site let's use cyber exercises as a
way to generate testable proposition you
know we've got these kind of natural
experiments and we can use them and mine
interesting information from them maybe
not de you know maybe not data but
anyone else want to pick up I'm you know
2013 all the professionals would say you
never see the Russians therefore we know
they're really good and you see the
Chinese all the time server you know
they're not so good right well the
reductio ad absurdum is Luxembourg must
be really on top of the game because you
never saw any cyber espionage for their
fingerprints on he might be here yeah it
might they might well be here you never
know right and that's a problem because
most of the activity in cyberspace
is cyber espionage and the whole point
is not to get caught
yeah it really struck me in when we
seemed to have data coming in from
mandiant saying that the Obama XI
agreement was being heeded to by the
Chinese that you saw this didn't reduce
to zero but you had the significant
reduction that how much that data was
argued that he was first it's incorrect
there has been no decreased and when the
decrease was was verified john carlin at
justice confirmed then people were
saying it's there but it doesn't matter
and i think there was just that built-in
bias of agreement suck this if we were
done this by deterrence then it would
have been more trustworthy and it was
very interesting to see the policymakers
with practitioners go that way if
there's one i it surprised me when I did
my book and how much the you know I had
the people on the the cleared side look
at it and it was certainly made me feel
better that they that they liked that
they thought that it carried through
from the classified verses the
unclassified I'll just I'll just put out
there is one more thought I mean you
might not think this given the
presentation I just Kate I'm I'm an
extremely conservative thinker when it
comes to the data that we can use on
cyber issues I simply think that
researchers in political science
particularly need to be doing more and
that's why it's certainly high
evolved with this type of project mmm
okay and I had called the second you
know I've got one over here there had
been one right okay well I don't see so
the second will be here there it'll be
the MIT thank you my name is coach
Patrice I work at Norwegian Institute of
International Affairs um yeah the
workers are security from a social
science point of view and they really
recognize you know that the challenge we
have because how do we study it with we
can see what politicians and Generals
say or we can read the strategies so we
can read media but it's you know it's
kind of far away from the real thing
and it typically inflated or you know
it's not really secure so I we really
follow you it's difficult to kind of
find good good good imperiex so to speak
but at the same time and you said
yourself it's it's in the context it's
always it conflicts in the context um to
cyber tools that has one many tools any
care to Ukraine in other words we know
it's going on and it's not so difficult
to know you can look at that specific
context specific incident if it's a big
political crisis but my question one I
want to bring up is that what we rely on
less they don't have access to
classified material is typical private
sector companies like the semantics and
the PI's who provide very good
information for us about what happened
but of course they have a there are not
objective you know information providers
necessarily there have an agenda and
they want to sell into might inflate the
factors well I don't know but I would
like your thoughts about that how we are
social scientists relied so much on them
on the private sector on this thank you
mm-hmm
I mean it's a methodological something
came to you but we also heard from
Robert for bits I don't know I've I've
had my brief list I've had my hand in in
several of those data parts and I've am
currently actually working with a couple
those companies I like the tools that
they offer offer us as social scientists
means something you can do with like
semantic data for example is you can
look at you know global malware
distribution actually controlling for
the number of users that Symantec has in
a given country right so you can you can
actually work on this representation
problem that we have with just broad
scope ocean style data collection I am
that said though I'm extremely leery of
the idea that you can infer about
sort of kind of global question the you
know the school questions the IRS
scholars consistently I seek to answer
just from working with vendors as I say
I'm kind of currently involved in a
couple of these projects and I don't
think I've convinced myself yet that I
can do more than focus on very very
tailored and specific region specific
questions you have using this data from
my point of view there for example I
mentioned that also in my paper the data
breach investigation reports from
Verizon a good example they did some
analysis on the cases they saw in 2015
and 16 and during the ovulation there
were also some technical conclusions and
some researchers put them up in the end
and made a very detailed analysis about
them and they found out that it's
absolutely not possible that these are
the real reasons for these breaches so I
think yeah you can see a little bit
because that that were the cases they
had seen and they analyzed but the
analysis itself was very weak so they
only did present these technical details
in these two years nowadays not so also
a good solution and just put the data
not in the report but often the the data
and these reports is very biased so you
really have to be careful with these
stuff it really struck me when you're
putting the costs up because when I was
doing my book I came across a quote from
cliff Stoll and he was talking about the
Morris worm and he's talking it looks
like the cost from Morris worm were
between 10,000 where between $100,000
and 10 million dollars from the Morris
worm 1988 so within two orders he
couldn't get it within two orders of
magnitude he couldn't get it pinned down
and you still see that today of you know
okay it's a billion it's ten trillion
dollars of different like we're still
like two orders of magnitude off and
trying to figure out the costs and that
and that was and that was 1988 okay over
here
hi I'm Emily click guard from the u.s.
Naval Academy my question is for dr.
libecki in regards to the recent cyber
conflicts between the United States and
other countries such as Russia China and
North Korea you mentioned several
options with regards to dealing with
cyber espionage
I was just wondering which option do you
think what the present conflicts were
experiencing today is the best course of
option for the United States to approach
with these countries do you think we
should approach them all the same or
should we approach each nation
differently especially with the vast
differences in the leadership of these
nations that's a that's a really good
question and a lot depends on the sort
of inferences they're likely to make
unfortunately if you actually work the
logic it turns out that you treat the
most reasonable people in the worst way
because they're the got least likely to
make the bad inferences right of the
three countries I think China's clearly
the most level-headed least likely to
fly off the handle under its current
regime but unfortunately you walk the
dog that means well we really ought to
be doing a lot of spying against China
precisely because they're not gonna
overreact which is not exactly a way of
rewarding there they're calm demeanor I
would be I would be very keen in general
I'd be extremely careful you know I'll
do the sums and check them twice for any
sort of cyber espionage against these
countries oddly enough because the North
Koreans having read the New York Times
may believe that we carried out a cyber
attack against their missile system they
may be the least sensitive to it but I'm
just guessing okay so it really struck
so I think the most important document
that's come out the the biggest change
in our doctrine and our thinking almost
since I've been in this business was
earlier this year when US Cyber Command
came out with their vision statement
that did a couple of things one it how
many folks have read that the Cyber
Command vision or the paper by by
Richard by Richard harken it
because it said one we recognize that
this that nations are in this state of
persistent engagement right it's all
something called constant contact right
there's not a period of peace and that
period of war it's not even a gray zone
it's this but it's this persistent of
the the the cyber forces of nations up
against one another in their throwing
elbows all the time
and and I thought that was such a one
they said and therefore we need to
defend forward with more agility so
there was a descriptive and then a
prescriptive part of that and to me they
many of us had been writing about that
and talking about that that it wasn't
near as necessarily discrete but I think
Richard harken it's paper and in Emily
Goldman really helped bring us into this
new mindset and it was interesting
because it was one of the first paradigm
changes that I personally felt and I'm
curious if am I out on a limb on that or
does it change the way that was freaking
we might look at date or the way that
you think about history or what might be
coming up next I mean after this I'm
gonna do I'm very curious about the next
ten years are we gonna see
discontinuities and Roberts already
picked up on that that'll be the next
question it's interesting that you talk
about persistent engagement and then and
then catch it in the language of states
fighting each other these are two
different ways of thinking about the
nature of cyber threats right their
language and and this is the this is
where the paradigm shifts you describe
anything comes in for me the interesting
thing is language over the past five
years six years or so has consistently
moved to thinking about cyber threats to
national security less as some sort of
warfare style of threat and we almost
talk about it more like pollution
something like that it's something in
the environment that we have to
consistently consider when making
strategic decisions and it's no longer
really about making strategic decisions
of a particular kind of a cyber warfare
flavor
it's about couching our actions in a
strategic environment that involves this
constant engagement and that creates
unique as Martin was talking about
unique institutional psychological
challenges for us yeah but I mean EPA is
not saying that they want to go in and
read
you know and take down polluting Chinese
factories right I mean but that that is
what Cyber Command is asking for right I
mean they're wanting to go into gray in
red space to directly affect adversary
command and control because it because
it's persistent engagement right I mean
so it really is you know normally
Russian and you know us warriors going
into in disrupting Russian warriors we
would you know we would can we wouldn't
mind hanging terms warfare around that
right on so I mean this this is really
is this a difference for example in the
way that we think about war that now the
adversary nations are constantly
engaging one another even or though
there's not bloodshed I certainly think
it has institutional ramifications
insofar as you know the question of the
nature of the cyber threat prompts you
to simply ask who are the stakeholders
that are you know necessary who are the
ones we actually need to empower in
order to mitigate cyber threats and in
particular in context of cyber command
obviously that is you know that is their
mission set they are absolutely right to
want to be able to well maybe not but I
think you want to be able to engage in
in those conflict areas to disrupt
disruptive activities and the likes but
you know the institutional question more
broadly particularly considering that
we've seen this diversification of what
cyber conflict is is should we cede that
degree of responsibility on you know
cyber on that cyber capacity with the US
government for example entirely to DoD
anymore yeah yeah I really like the
picture of the pollution with was having
a look at severe challenges in the cyber
arena sometimes it's really difficult to
differentiate was it just by by bad
programming or something like that or
did really some powerful actor influence
a process so I think sometimes it really
can be difficult to differentiate but at
least I think that the nations that have
the capabilities doing hard at the
moment to put some stuff that can be
useful for themselves later into the
systems
which is challenging because everybody
uses the same stuff and you can have the
boomerang effect but if it does strike
mean for both of your papers then we'll
go to Martin and then we'll do the the
final comments about about the future in
that we're used to in political science
counting definable events but if it's
persistent engagement than the
individual events matter significantly
less than then the set because because
it's more constant sure or that you know
it struck me in the way that I'd have
done like you've done like you've got
the line and you put the things on the
line but when we do that we're missing
this back and forth between adversaries
over time but I read a dissertation once
that tried to say that there was no
escalatory effects from Stuxnet there
was no tit for tat or anything that came
from it because they they stopped their
timeline at like 2011 right so they
didn't include the Iranian DDoS so they
said there were there's no impact from
this and so I think it is interesting
and I worry how we tell our stories and
how we draw our lines and how and how we
count yeah absolutely now I just put a
shill for my you know our suggested
person that's pay there again I promote
my approach in this paper again and just
saying that part of the reason I think
that political scientists actually need
to just adopt the idea the public facing
information can be useful for a research
is not because we want to talk about the
content of cyber incidents we need to
talk we need to gather metadata about
the context in which it occurs mm-hmm
that is as often more important that's a
great that's a great point all right I
promise that we were gonna cover what's
coming up in ten years it turns out that
we don't have that because we're running
out of time so Martin libecki is gonna
get the final word yeah and it better
not be balderdash
there's a story here I know I took a
look at the Cybercom thing and I don't
know how to evaluate it because they're
talking about things that they're not
really talking about and so you had to
make a guess my sense is that Cybercom
is trying to raise the cost of adversary
actions but as an economist I always
have to ask the question one
is it costing us more to raise their
cost than it's worth and number two is
something called the fallacy of the last
move right when you do something and you
say wow this is successful you know I've
won until the other guy comes up with a
countermeasure and then you think you
know maybe my investment wasn't such a
smart idea after all something I don't
know yeah the last word is gonna be I
don't know it's actually not a bad way
to leave a panel on what do we do and
how do we know it of I don't know
because the open questions are so broad
and like it brought up in the keynotes
this morning that Microsoft brought up
John Frank love saying look we are in
the first steps of a new digital
civilization right we are at the very
beginning edge of this and we've got a
long way to go so with that we don't
have a long way to go with this panel so
please join me in thanking our
[Applause]