okay great so we'll get going with the
south
it's my pleasure to introduce captain
core
who's currently PhD student pasty Canada
of computer and information technology
at Purdue University he's earned a
bachelor's and master's degree in
government and politics from st. John's
University masters and networking
communication management and an MBA from
Keller Graduate School he currently
serves as a captain a United States Army
and has worked in the information
technology field for over 24 years his
research interests include information
security cyber security forensics risk
analysis cyber resiliency and
information assurance ethics he'll be
talking to a city about the application
of natural language processing in
advanced persistent threat domain so
captain holser it's all yours thank you
very much yes so it's a long title I got
it but being in the military used to
acronyms and the one thing we're always
told to spell out the acronyms the first
time you use them so this what this is
this is my dissertation work and i'm
looking at natural language processing
of open source documents out in the
internet so there's nothing in here and
obviously it's classified or sensitive
but what it also means is it's also
nothing that's proprietary and that's so
so what I've done is I started actually
and I'll actually plug although none of
none of her students here but the dr.
darks information security problem
challenges class and this actually
started out in her course I did a
project on cyber resiliency and looking
at the advanced persistent threat and
that kind of spring boarded me into
doing some of the some of this work
about the advanced persistent threat
domain which is a relatively new domain
and just to define advanced persistent
threat if you haven't heard of it before
advanced obviously being adversaries
that are employing tactics that cover a
full spectrum this isn't a drive-by
attack on a website this isn't just
installing some malware and and doing
something that's pretty innocuous this
is actually a very strategically
designed attack from beginning to end
when it's which you'll see later on I'm
going to actually go through the cyber
kill chain
but it's it's all the way from planning
execution staying within the system
which is where your persistence comes
from and then ultimately a extra waiting
in a in a lowkey fashion until you find
the data or the system that you're
looking for and then exfiltrating and
executing your final attack so as we
continue with this there's actually
excuse me I apologize i'm a bit of a dry
mouth there's the term apt was actually
coined by the air force about ten years
ago when they wanted to describe this
kind of low and slow attack that they
were seeing across the networks and
across government networks but they
wanted to be able to discuss this with
with organisations with academia that
didn't necessarily have the the
classification or the authority to see a
classified information so they came up
with the term advanced persistent threat
because as I showed them the less
sliding as i'm going to show in the next
this in the next slide this really
defines not only the actor that's
executing this kind of attack but it
also describes the attack itself now as
an actor these are adversaries that will
employ or what right easy for me to say
wide range of tactics from the use of
fishing to the use of malware to how
they're going to actually sell the data
and make make a profit on it or turn
data into money into goods and then back
into money again so that they basically
do like the old organized crime and
clean their money they're well-funded
and well-organized and they're patient
they can stay in a system from six
months to 18 months there's actually
been some reports on the extreme side
that some of these apt attacks have been
extended over two years so it's not
something that they're just trying to
deface a website and get out of there as
I said before they can infiltrate a
network and remain hidden as I said and
their goal is stealthy execution they
don't want you to know they're there
because the longer they can stay there
the more information and the more
valuable information they can obtain and
it's the challenge here is that as the
as the actor he only has to find one
flaw in the system whereas the defender
has to defend every floor we've heard
this before and other talks previously
and and their priority is this attack
whereas the defender also has to worry
about business processes and and other
things that they have to what risks
they're going to accept so it's a bit
it's a bit more challenging for the
defender as an attack it involves
multiple methods tools and techniques
using a sophisticated and complex manner
a single apt can include social
engineering like i said before fishing
campaign as well as malware and maybe
even what's really interesting is once
they get past the initial phase of the
apt attack they can start using
traditional packages that you and I
might use SSH SFTP other things like
that that would actually get go through
a firewall or security measures and
would probably be considered just normal
traffic and probably ignored for that
reason so here-here's and I apologize
that this looks like a bit of an eye
chart but this is the cyber kill chain
that you're seeing here and includes
reconnaissance weaponization which are
the first two phases excuse me then you
have delivery exploitation installation
command and control which is maintaining
communications with those systems and
then actions on the objective I know it
sounds very military and probably that's
my fault but actually it's the term that
Lockheed Martin uses and defined this
kill chain because this is when they
find the data they want how do they get
it out of there so going back to
reconnaissance this is if you've ever
heard of the term footprinting that's
what this is this is finding out
information finding your flaws finding
the flaws in the system finding the the
internal person that you might be able
to take advantage of whether it's a
spearfishing a well in campaign or just
the person who might have some interest
that you could send them that email
and just get them through a phishing
attack you have weaponization and here
the actor puts together codes used to
compromise the target system it involves
the use of existing improving code that
they've adapted that they will adapt to
their need so again this excuse me goes
back to the complexity thing they can go
out there and actually I apologize they
can go out there and they can actually
hire someone to write a piece of code or
take another piece of code that might be
out there on the internet and then
weaponize it for some kind of for their
attack there's delivery and this is the
most common of course are going to be
email attachments website malicious
websites removable media all the same
things that we hear about in general
cyber security but this doesn't have to
be that's a sufficient doesn't need to
be that specific excuse me sophisticated
it is the overall complexity of the
attack that makes it a sophisticated
attack exploitation is it doesn't work
unless the person clicks on it so how do
you exploit the user and and get him to
actually trigger the next step which is
installation installation we often
include a remote administration tool or
a rat which or some kind of backdoor
that the apt can use to gain control the
target system once triggered the
malicious code reaches back to its
commanding control servers which I'll
get back which and they provides the apt
actor with useful information and usable
information that they can use in later
ap stages of the attack whether it's
usernames passwords IP addresses IP
address ranges and the like once
installed the rat can also lay dormant
and can be turned off now normally what
will happen in a much more simple attack
is remote access tool will be used get
the information they want delete it off
the system and get and basically clear
the wipe away their tracks but here what
they do is they let the right lie
dormant in the system and again this
goes back to that persistence aspect so
they get bits of information at a time
then
I only get a username the first time
they connect to this computer then maybe
they'll try to figure out how elevated
permissions work and that might be three
months later again that low and slow
approach because the more activity they
show between the rat and the command and
control servers the more likely it is
that that it will trip some sort of
sensor whether it's an IDs or an IPS
system that runs on the network so by
doing it in small doses it's a lot
harder for the defenders to the tech and
the defending systems to detect
malicious activity command and control
phase starts when that when that rat
starts beaconing back to to the command
and control servers or it could be a
botnet I mean it they'll use a variety
of different things so that that it so
that no two apts are exactly alike
they'll also install additional software
they might roll their own encryption
they might again like I said earlier use
what would be generally accepted
programs like FileZilla or or SFTP or
ssh to to actually get themselves in
transition system and then allow them to
extract the data because of course over
an encrypted pipe it's a lot harder to
detect what actual information is going
out and then the active actions on
objective we're talking about actively
going after the data they're looking for
actively going after and possibly
damaging systems that they want to do
because apts aren't just about getting
data out Stuxnet is actually classified
as an apt and that the goal of Stuxnet
was actually to define to identify SCADA
systems and and and close issues and
nuclear reactors so we can see how the
apt can not only be about data theft but
it can also be have a malicious intent
cost of apt attacks of just quickly I
mean a study by the pond amount
Institute the researchers estimated they
closed up to 161 dollars for each record
lost and in in an event think about it I
RSA here they estimated was 66 million
dollars us to undo the damage caused by
the apt attack now there's
isn't just in the data they lost but
this was also providing services to the
compromised individuals and compromised
organizations and having to basically
fix the damage now so that that's this
is the that's the area that I was doing
my research about but what I'm trying to
do now is actually use open source
documentation using natural language
processing to learn key terms within
this domain that I can build tube in
order to build an ontology that can be
used by anyone who's working with apts
and understand as dr. Ruston was talking
about last week it's it's partial partek
taxonomy and also understanding more
details about and standardizing the way
terminology is used so open source
technology has been used for years used
by law enforcement intelligence as a way
to gain information that's actually
publicly available Facebook would be
considered open source obviously not
applicable for my work here but but
definitely is considered open source and
skip that why is this significant well
part of the yes part of this comes from
my proposal companies are facing this
challenge all the time a pt's late as I
said are they take this approach that it
keeps them obscure and hidden imagine if
if well the imagine a company like
Target if an apt infected their system
and was slowly siphoning off credit
cards over years and years of time
that's that that damage and the impact
of that company could be significant
okay I'm going to skip this so here's
here's the methodology and the approach
that i've been using or i've been using
in my research I started out and I disc
I found excuse me
I found to open-source knowledge bases
about the advanced persistent threat I
took them I call a to them I
deduplicated because there was a lot of
repeat documents in there and I ended up
actually the numbers a little high I
actually ended up with a few few more
doc inside and removed but 425 documents
which I felt was a significant enough
base to start with developing an
ontology i selected 22 of those
documents for manual processing and i'll
get i'll get into that a little bit all
that was was actually me going through
these documents and reading them
identifying key terms and and and key
definitions because i had to be able to
use that the what i found in those
documents to actually build not only my
draft ontology but but also a co
reference library that the language the
natural language processing software
will will use to ideally filter out and
understand ambiguous references within a
document i then took that I'm then
taking that draft ontology and the the
actual same pool of 425 documents using
using natural language processing to
identify key terms within those
documents and then going to and
developing a protege an ontology excuse
me using protege which is an which is a
recognized piece of software and then
ultimately the final product is an owl
too compliant which is an industry
standard developed by the World Wide Web
Consortium for an ontology so it is
natural language processing natural
language processing is just that it
takes naturally found a text in
documents like these slides like an
article like a blog and will actually
read through it and analyze it and and
break it down into the tokens and the
words and the terms and identify and
categorize those terms it looks at
things syntactically and semantically
natural language processing can actually
deal with multiple languages I had to
exclude foreign languages from my
research only because of my own
inability to work with those languages
so I had to actually that's why actually
part of my document size dropped a bit
was because not only were there
duplicates but there were also ones that
were in Chinese and Cyrillic and Arabic
that I just couldn't process so I had to
accept at least for the for this work
that I couldn't use that so some basic
terms that you'll hear when you hear
about natural language processing you
have a domain which is just the area
we're discussing apt in this case you
have the corpus or the corpora which is
the all the documents you're using so my
corpus currently consists of 425
documents you have a lemma which if you
remember ACTU Basic English classes in
high school and grammar school it's your
root word so as the example was here I
have the word capture and you variations
captured captures capturing and then all
documents are broken down into tokens
and I'll talk a little bit more about
them later but primarily anything
between white spaces are considered
tokens so they can be words they can
also be punctuation or lat which
actually became a bit of a challenge for
me as I was trying to to sort through
this NLP s will identify parts of speech
tagging they'll identify things as verbs
verb phrases nouns adjectives and the
like they'll also use named entity
recognition or any are and what that is
is it sees the word microsoft and
recognizes it as an organization it sees
the term States or a country like India
and they'll recognize that as a location
or Baghdad as a city which is also a
location and there's actually about 13
common any ours that are used and part
of my work was to actually try to
improve that a little bit or refine it
because for example United could be an
organization as in United Nations or it
can be a
location is in the United States the
role of an ontology I've already spoken
about so I'm actually escaped here here
and I hope you can see this a little bit
clearer than I can on my small screen
ontology associate terms vertically and
horizontally vertically as in if you
think as I have in the example here we
have communications underneath
communications we have encrypted and
unencrypted and then below encrypted for
example you have things like unencrypted
you'll have file transfer protocol HTTP
you'll have telnet under encrypted you
have ssh SFTP secure HTTP and as you can
see there as I've kind of done it in the
tree diagram here the the horizontal
would be your the items at the same
level so encrypted or unencrypted would
be at the same level because of the
broadness of the two topics and the more
specific examples or the identities are
actually at that lowest level now when I
when I took these documents actually one
of the interesting things as I said I
started with about twenty two documents
and try to actually do it myself I think
of from that experience if I had tried
to go all through all 425 and then I
probably would it would have taken me
years so and to see that to see this
natural language processing software
only take maybe four days on on my
personal computer to process through
four hundred documents actually is very
helpful the only challenge is it doesn't
go all the way it it doesn't just spit
out a list of useful terms it gives you
everything back that's in the document
so the 425 documents broke down into a
four almost 4.5 million tokens to be to
be used now a lot of those and I don't I
don't I don't want to trivialize it some
of them were meaningless some of them
were actually parentheses other symbols
that were just useless wouldn't wouldn't
provide anything useful for the ontology
development but then you also have
common words and this is actually where
what I actually did I had this wasn't in
my original plan but I adapted on the
fly I actually went out and used the
America the contemporary American
English 5,000 most common words and i
actually just filtered those out of the
documents now I only did that in terms
of identifying important terms for the
ontology I did not I I couldn't just
delete them because obviously that would
potentially destroy context of what I
was trying to work on in finding
definitions but almost half of the the
tokens were or half of the words that we
found 1.25 million was actually those
common words you can see how much I mean
how quickly we were getting down now
another 450 another half a million were
punctuation some kinds of punctuation so
I could get rid of those at least as far
as identifying terms for terminology and
then another 85,000 were just numbers
and by that you know whether it's 1024
64 whatever is which weren't going to
meaningful term stream fine ontology and
I ultimately came down to about 600,000
meet tokens that I could actually say
some ontological terms could come from
these now one thing I discovered and
this is actually getting back to I'll
get to this like these are the district
the chart on the Left shows the
distribution of how many tokens per
document and you can see that actually
most of them fell in the 0 to 30,000
range with that one little outlier or
too little outliers that were between
eighty and ninety thousand and those
were actually full blown books that I
had or most several chapter books I
think they're more like dissertations
but most of these documents as you can
see really falling that really the one
the 0 to 10 thousand range because a lot
of these or articles are on websites
short papers that are done by things
like kaspersky papers done by kaspersky
or mcafee or things like that so they're
actually designed to be to be a lot
shorter now one of the things I quickly
again
you won't you only realize as you're
going through your research is that I
was finding many very like I showed you
the example of captured before the word
capture before a lot of my I've seen
variations of the same word I'm like
well how many times I had to ask it made
me ask myself how many times should this
word actually appear in the ontology
well theoretically only once it should
appear in the ontology so I started
looking at the unique lemons which is
why I show that here and the average
document had only between zero and five
hundred unique lemma that makes that out
starts to make my identification of
ontological terms a lot a lot easier
than then it might have been when I was
looking at 2.5 million tokens then this
is just student this is just a quick
breakdown of the distribution now the
one thing this doesn't show is the total
tokens but the the first the first two
are just punctuation xin digits which
usually fell out now you can the third
one is the actually the interesting one
there that's that's where the unique
lemma falls and you can see how tight of
a distribution that is in that box plot
and that's basically what I knew I
needed to focus on excuse me that's what
I need to focus on in order to find my
ontological terms and this the one the
second from the right is actually those
common words as 1.2 million tokens that
I was talking about 25 tokens I was
talking about earlier so this is this is
actually some statistical analysis that
I did of some terms now you can see the
third one there is the apostrophe s
possessive that's not an ontological
term but you can see how many times it
appears over 6,000 times in 338 of the
documents it in and of itself as a token
is not useful now the possessive the
word that's before is of course
important but you can see where that's
where again this shows where even though
the the natural language processing does
a lot to choose me help me break down
these documents there's still a lot of
manual
lifting that has to be done in my part
and where I first had thought I said
well the words that I'm going to want to
use are going to be those terms if they
appear a lot in throughout the whole
corpus but if you look at CID which yes
has 20,000 appearances but it actually
only appears in 28 documents and
actually of the of the whole it's only
three point six percent so that we I
quickly realized well where's my where's
the sweet spot where do I need to if i'm
going to use statistics to try to
identify some of my terminology and it
came down to appearances poor corpus now
that appearances / corpus is the number
of appearances averaged out over the 425
documents though the column to its to
the left of that the appearances per
document is actually documents it
appears in so that's what explains the
difference in those numbers now it's our
if these are only the top 15 terms
there's actually about the table ends up
for lemma ends up being about 70,000 so
I'm not going to there's not going to be
slides for everything obviously but it's
actually ironic here that if you look
the term apt actually isn't but by this
by the designation that I've use
actually comes up as number 36 so
there's a lot of terms in here that you
wouldn't necessarily think of or I did
that I wouldn't have thought of right
away as being corpus terms but hear
things like server attackers C 2 and C
and C are actually both shorthand for
command and control so that's the one
thing that I will say that would take
probably a lot more time to work on is
actually going through because again
this goes back to why we're looking for
an ontology in the first place C 2 and C
and C or I if I said c2 to somebody and
they only knew it is C&C they wouldn't
necessarily know what I'm talking about
and that's the value of the two ontology
now here's the here's a a simplified
overview of basically the the ontology
as I'm developing it you can see I have
the
the tables or the portions that include
focus on the organization and then the
the parts that focus on the attack so as
I go to the organization we have as an
apt organization it's going to have a
name most of them right now or just apt
one apt to apt 30 is a common one if you
actually google it some but not all will
have state sponsorship sponsorships by
motivation i was looking at things like
mice deep some some are looking to close
some most or criminal activities
stealing information others are looking
to do malicious things along lines of
Stuxnet then we go to the organization
and how its structured remember I said
this is a complex thing you have things
like code writers virus writers web
designers system administrators and
these are terms we probably all could
come up with a definition pretty quickly
but finding this kind of information and
seeing how complex these organizations
are shows how important it is to have
these come this ontology development
nation now as I move down to the attack
obviously this is a lot more detailed
you have things like the name of an
attack and you'll hear things like
Stuxnet ice fog trying to think the are
the RSA attack I'm drawing a blank on
the name but I mean you have a lot of
and they'll get reused and they might
get renamed but there could be a
relationship attributed organizations
the attribution is one of our toughest
things to do but sometimes there are
patterns and there there's information
embedded within the code that actually
tells us the organization not too bright
but it's what it is and then when it
first appeared and again some apt
attacks may get reused by other
organizations or by the same
organization later on with a different
different set of targets now again like
I said apt name here's some examples
shady right night dragon ice fog
arachnophobia and that's actually one
thing i don't i'm not sure if you can
see clearly but the there's a link here
between an apt attack and the malware
that it uses
and bitter bug which I've used here is
actually a sub component of the
arachnophobia apt I guess that makes
sense using with the similar names and
here we have the types of communications
that are used which again you I've
mentioned earlier and that actually
pretty much covers this this the
ontology is still in work in progress
and I've got to get it done pretty quick
because I'm graduating here and in
December I'm about to end i'm scheduled
to defend here but the first 22
documents produced almost a thousand
terms that I personally could identify
using some of the statistical analysis
and the larger set of documents I'm
already seeing that list growing to well
over 3,000 if I would turn that ontology
and now start to use it as a search tool
to search the web because I now have
terms that I can use I would expect that
we continue to expand and that's
actually one of the future aspects of
research and using a potentially link
analysis as a means of understanding if
if two if two different apts for example
use the same kind of same means of
communication other other attributes
that are similar are there are there
ways that we can apply detection methods
from one to the other because of the
similarities that they share and that's
that's where the application comes in
here too for cyber security and for us
as the defenders not only having the
common terminology but potentially
starting to mine data big data and find
improving our detection defense method
methodology so I'll open up the floor to
questions
I was interested in what software you
used for a natural language processing
i'm using oh that's good i'm sorry i
didn't mention that there's actually a
few out there there's a couple of open
source ones i actually looked at the
natural language toolkit which is python
but i ended up on Stanford's core NLP
and there's actually an extension that
somebody developed and I'm drawing a
blank in his name so i apologize i want
to give him credit he created an app
called book NLP which is just an
extension of Stanford's product which
was meant to deal with large documents
entire books versus just sentences or
paragraphs or short blogs it's called
book NLP okay look is the extension now
the one the one thing I will say if
anybody looking into natural language
processing Lisa's far there Coronel
Coronel P product actually they have
there have been a lot of extensions that
have been developed by people doing
their own research I didn't personally
have the time because of my timeline
here to develop my own the only thing
the only challenge that I've seen and
also with protege as an ontology tools
people use it they develop it for as
much as they need and then it doesn't
quite either get continue to be
developed because they've run out of
funding or just they finish their
project or it doesn't quite hit the mark
that you might need but there are a lot
of there was a lot of tools and a lot of
resources out there as even as dr.
Raskin was talking about two weeks ago
sure the software that you used what
format was your corpus in and did you
get your corpus off the internet or oh
I'm sorry I thought I okay yes I found
two knowledge bases online they were
actually both on github and they were in
PDF format what I ended up having to do
though was using a Python tool to
actually extract the text from those
documents so that they could be
SS by the NLP software now some some
some pdfs can be processed natively
through the NLP but i was getting too
many errors and I because it was kind of
strolling my progress I made the
sacrifice of losing images and and maybe
graphically design PDFs accepting that I
would lose those uh and I just extracted
whatever text I could get from them and
process that any other questions I have
a question yum you mentioned text
categorization do you use any machine
learning algorithm to categorize text uh
their core NOP well the natural as a
natural language processing tool some of
them have some algorithms I didn't I
used what came out of the box I I used
it as it came I again this is that that
ability to extend these packages that's
where you could add algorithms like that
I did I did and it was part of the
reason for using the 22 documents was to
try to train their algorithm it didn't
prove and I'll honestly say it could be
it could be on me because I was actually
trying to learn this as I was going my
training of the tool didn't seem to
change my results very much but you can
actually train these NLP software to
recognize regular expressions that they
might be coming you can use regular
expressions to train it in algorithms to
train its a yes they are possible any
other question
well thank you very much that does it
it's weird being on this side of the
screen off
you