Okay, great.
So we'll get going with this afternoon's session.
It's my pleasure to introduce Captain Corey Holzer, who's currently a Ph.D.
student, a Ph.D.
candidate of Computer and Information Technology at Purdue University.
He's earned a Bachelor's and Master's degree in Government and Politics from St.
John's University, Master's in Networking and Communication Management, and an MBA from Keller Graduate School.
He currently serves as a captain in the United States Army and has worked in the information technology field for over 24 years.
His research interests include information security, cyber security, forensics, risk analysis, cyber resiliency, and information assurance ethics.
He'll be talking to us today about the application of natural language processing in advanced persistent threat domain.
So, Captain Holzer, it's all yours.
Thank you very much, yes.
So, it's a long title, I got it, but being in the military, you're used to acronyms, and the one thing we're always told is spell out the acronyms the first time you use them.
So what this is, this is my dissertation work, and I'm looking at natural language processing of open source documents out on the internet.
So there's nothing in here, obviously, that's classified or sensitive, but what it also means is it's also nothing that's proprietary.
So what I've done is, I started actually, and I'll actually plug, although none of her students here, but the Dr.
Dark's information security challenges class, and this actually started out in her course.
I did a project on cyber resiliency and looking at the advanced persistent threat, and that kind of springboarded me into doing some of this work about the advanced persistent threat domain, which is a relatively new domain.
And just to define advanced persistent threat, if you haven't heard of it before, advanced obviously being adversaries that are employing tactics that cover a full spectrum.
This isn't a drive-by attack on a website, this isn't just installing some malware and doing something that's pretty innocuous.
This is actually a very strategically designed attack from beginning to end, which you'll see later on, I'm going to actually go through the cyber kill chain, but it's all the way from planning, execution, staying within the system, which is where your persistence comes from,
and then ultimately waiting in a low-key fashion until you find the data or the system that you're looking for, and then exfiltrating and executing your final attack.
So, as we continue with this, there's actually...
Excuse me, I apologize, I have a bit of a dry mouth.
The term APT was actually coined by the Air Force about 10 years ago, when they wanted to describe this kind of low and slow attack that they were seeing across the networks and across government networks.
But they wanted to be able to discuss this with organizations, with academia, that didn't necessarily have the classification or the authority to see classified information.
So they came up with the term Advanced Persistent Threat, because, as I showed in the last slide and as I'm going to show this in the next slide, this really defines not only the actor that's executing this kind of attack, but it also describes the attack itself.
Now, as an actor, these are adversaries that will employ a wide range of tactics, from the use of phishing, to the use of malware, to how they're going to actually sell the data and make a profit on it, or turn data into money, into goods, and then back into money again,
so that they basically do like the old organized crime and clean their money.
They're well-funded and well-organized, and they're patient.
They can stay in a system from six months to 18 months.
There's actually been some reports on the extreme side that some of these APT attacks have extended over two years.
So it's not something that they're just trying to deface a website and get out of there, as I said before.
They can infiltrate a network and remain hidden, as I said.
And their goal is stealthy execution.
They don't want you to know they're there, because the longer they can stay there, the more information and the more valuable information they can obtain.
And the challenge here is that, as the actor, he only has to find one flaw in the system, whereas the defender has to defend every flaw.
We've heard this before in other talks previously.
And their priority is this attack, whereas the defender also has to worry about business processes and other things that they have to do, what risks they're going to accept.
So it's a bit more challenging for the defender.
As an attack, it involves multiple methods, tools, and techniques used in a sophisticated and complex manner.
A single APT can include social engineering, like I said before, phishing campaign, as well as malware, and maybe even, what's really interesting is, once they get past the initial phase of the APT attack, they can start using traditional packages that you and I might use,
SSH, SFTP, other things like that, that would actually go through a firewall or security measures and would probably be considered just normal traffic and probably ignored for that reason.
So here is, and I apologize that this looks like a bit of an eye chart, but this is the cyber kill chain that you're seeing here.
And it includes reconnaissance, weaponization, which are the first two phases, excuse me.
Then you have delivery, exploitation, installation, command and control, which is maintaining communications with those systems, and then actions on the objective.
I know it sounds very military and probably that's my fault, but actually it's the term that Lockheed Martin uses in defining this kill chain because this is when they find the data they want, how do they get it out of there.
So going back to reconnaissance, this is, if you've ever heard of the term footprinting, that's what this is.
This is finding out information, finding your flaws, finding the flaws in the system, finding the internal person that you might be able to take advantage of, whether it's a spear phishing, a whaling campaign, or just the person who might have some interest that you could send them that email and just get them through a phishing attack.
You have weaponization, and here the actor puts together codes used to compromise the target system.
It involves the use of existing and proven code that they've adapted, that they will adapt to their need.
So, again, this goes back to the complexity thing.
They can go out there and actually, I apologize, they can go out there and they can actually hire someone to write a piece of code or take another piece of code that might be out there on the internet and then weaponize it for their attack.
There's delivery.
And the most common, of course, are going to be email attachments, malicious websites, removable media, all the same things that we hear about in general cybersecurity.
But this doesn't have to be that sophisticated.
Each component doesn't need to be that sophisticated.
It is the overall complexity of the attack that makes it a sophisticated attack.
Exploitation is, it doesn't work unless the person clicks on it.
So how do you exploit the user and get him to actually trigger the next step, which is installation.
Installation will often include a remote administration tool or a RAT, or some kind of backdoor that the APT can use to gain control of the target system.
Once triggered, the malicious code reaches back to its command and control servers, which I'll get back, and they provide the APT actor with useful and usable information that they can use in later stages of the attack, whether it's usernames, passwords,
IP addresses, IP address ranges, and the like.
Once installed, the RAT can also lay dormant and can be turned off.
Now normally what will happen in a much more simple attack is a remote access tool will be used, get the information they want, delete it off the system, and basically wipe away their tracks.
But here what they do is they let the RAT lie dormant in the system.
And again, this goes back to that persistence aspect.
So they get bits of information at a time.
They may only get a username the first time they connect to this computer.
Then maybe they'll try to figure out how elevated permissions work, and that might be three months later.
Again, that low and slow approach, because the more activity they show between the RAT and the command and control servers, the more likely it is that it will trip some sort of sensor, whether it's an IDS or an IPS system that runs on the network.
So by doing it in small doses, it's a lot harder for the defending systems to detect malicious activity.
Command and control phase starts when that RAT starts beaconing back to the command and control servers.
Or it could be a botnet.
They'll use a variety of different things so that no two APTs are exactly alike.
They'll also install additional software.
They might roll their own encryption.
They might, again, like I said earlier, use what would be generally accepted programs like FileZilla or SFTP or SSH to actually get themselves entrenched in the system and then allow them to extract the data.
Because, of course, over an encrypted pipe, it's a lot harder to detect what actual information is going out.
And then the actions on objective, we're talking about actively going after the data they're looking for, actively going after and possibly damaging systems that they want to do.
Because APTs aren't just about getting data out.
Stuxnet is actually classified as an APT.
And the goal of Stuxnet was actually to identify SCADA systems and cause issues in nuclear reactors.
So we can see how the APT can not only be about data theft, but it can also have a malicious intent.
Cost of APT attacks.
Just quickly, I mean, a study by the Ponemon Institute, the researchers estimated that it cost up to $161 for each record lost in an event.
Think about it.
RSA here, they estimated it was $66 million U.S.
to undo the damage caused by the APT attack.
Now, this isn't just in the data they lost, but this was also providing services to the compromised individuals and compromised organizations, and having to basically fix the damage.
So that's the area that I'm doing my research about.
But what I'm trying to do now is actually use open source documentation, using natural language processing to learn key terms within this domain, in order to build an ontology that can be used by anyone who's working with APTs.
And it's basically, as Dr.
Rasman was talking about last week, it's partial part taxonomy and also understanding more details about and standardizing the way terminology is used.
So open source technology has been used for years, used by law enforcement, intelligence, as a way to gain information that's actually publicly available.
Facebook would be considered open source.
Obviously not applicable for my work here, but definitely is considered open source.
I'm going to skip that.
Why is this significant?
Well, yes, part of this comes from my proposal.
Companies are facing this challenge all the time.
APTs, as I said, they take this approach that keeps them obscure and hidden.
Imagine a company like Target.
If an APT infected their system and was slowly siphoning off credit cards over years and years of time, that damage and the impact to that company could be significant.
Okay, I'm going to skip this.
So here's the methodology and the approach that I've been using in my research.
I started out and I found two open source knowledge bases about the advanced persistent threat.
I took them, I collated them, de-duplicated because there was a lot of repeat documents in there, and I ended up, actually that number is a little high, I actually ended up with a few more documents I had to remove, but 425 documents, which I felt was a significant enough base to start with developing an ontology.
I selected 22 of those documents for manual processing, and I'll get into that a little bit.
All that was, was actually me going through these documents and reading them, identifying key terms and key definitions, because I had to be able to use what I found in those documents to actually build not only my draft ontology, but also a coreference library that the natural language processing software will use to ideally filter out and understand ambiguous references within a document.
I then took, I'm then taking that draft ontology and the actual same pool of 425 documents, using natural language processing to identify key terms within those documents, and then going to, and developing a protege, an ontology, excuse me, using protege,
which is a recognized piece of software, and then ultimately the final product is an OWL2 compliant, which is an industry standard developed by the World Wide Web Consortium for an ontology.
So what is natural language processing?
Natural language processing is just that.
It takes naturally found text in documents like these slides, like an article, like a blog, and will actually read through it and analyze it and break it down into the tokens and the words and the terms, and identify and categorize those terms.
It looks at things syntactically and semantically.
Natural language processing can actually deal with multiple languages.
I had to exclude foreign languages from my research only because of my own inability to work with those languages.
So I had to actually, that's why actually part of my document size dropped a bit, was because not only were there duplicates, but there were also ones that were in Chinese and Cyrillic and Arabic that I just couldn't process.
So I had to accept, at least for this work, that I couldn't use that.
So, some basic terms that you'll hear when you hear about natural language processing.
You have a domain, which is just the area we're discussing, APT in this case.
You have the corpus or the corpora, which is all the documents you're using.
So my corpus currently consists of 425 documents.
You have a lemma, which, if you remember back to basic English classes in high school and grammar school, it's your root word.
So, as the example was here, I have the word capture.
And you have variations, captured, captures, capturing.
And then all documents are broken down into tokens, and I'll talk a little bit more about them later.
But primarily, anything between white spaces are considered tokens.
So they can be words, they can also be punctuation, all that, which actually became a bit of a challenge for me as I was trying to sort through this.
NLPs will identify parts of speech tagging.
They'll identify things as verbs, verb phrases, nouns, adjectives, and the like.
They'll also use Named Entity Recognition, or NER.
And what that is, is it sees the word Microsoft and recognizes it as an organization.
It sees the term states, or a country like India, and they'll recognize that as a location, or Baghdad as a city, which is also a location.
And there's actually about 13 common NERs that are used.
And part of my work was to actually try to improve that a little bit, or refine it.
Because, for example, United could be an organization as in the United Nations, or it could be a country, a location as in the United States.
The role of an ontology I've already spoken about, so I'm actually going to skip.
Here, and I hope you can see this a little bit clearer than I can on my small screen.
Ontologies associate terms vertically and horizontally.
Vertically, as in if you think, as I have in the example here, we have communications.
Underneath communications, we have encrypted and unencrypted.
And then below encrypted, for example, you have things like unencrypted, you'll have file transfer protocol, HTTP.
You'll have Telnet.
Under encrypted, you'd have SSH, SFTP, secure HTTP.
And as you can see there, as I've kind of done it in the tree diagram here, the horizontal would be the items at the same level.
So encrypted and unencrypted would be at the same level because of the broadness of the two topics.
And the more specific examples or the identities are actually at that lowest level.
Now, when I took these documents, actually one of the interesting things, as I said, I started with about 22 documents and tried to actually do it myself.
I think from that experience, if I had tried to go through all 425, it would have taken me years.
And to see this natural language processing software only take maybe four days on my personal computer to process through 400 documents actually is very helpful.
The only challenge is it doesn't just spit out a list of useful terms.
It gives you everything back that's in the document.
So the 425 documents broke down into almost 4.5 million tokens to be used.
Now a lot of those, and I don't want to trivialize it, some of them were meaningless.
Some of them were actually parentheses.
Other symbols that were just useless wouldn't provide anything useful for the ontology development.
But then you also have common words.
And this is actually where, what I actually did, this wasn't in my original plan, but I adapted on the fly.
I actually went out and used the contemporary American English 5000 most common words, and I actually just filtered those out of the documents.
Now I only did that in terms of identifying important terms for the ontology.
I couldn't just delete them because obviously that would potentially destroy context of what I was trying to work on in finding definitions.
But almost half of the words that we found, 1.25 million, was actually those common words.
So you can see how quickly we were getting down.
Another half a million were punctuation, some kinds of punctuation, so I could get rid of those, at least as far as identifying terms for terminology.
And then another 85,000 were just numbers, whether it's 1024, 64, whatever it is, which weren't going to be meaningful terms for the ontology.
And I ultimately came down to about 600,000 tokens that I could actually say some ontological terms could come from these.
Now, one thing I discovered, and this is actually getting back to... I'll get to this slide.
The chart on the left shows the distribution of how many tokens per document.
And you can see that actually most of them fell in the 0 to 30,000 range, with that one little outlier, or two little outliers, that were between 80 and 90,000.
And those were actually full-blown books that I had, or several chapter books.
I think they were more like dissertations.
But most of these documents, as you can see, really fall in that 0 to 10,000 range, because a lot of these are articles that are on websites.
Short papers that are done by things like Kaspersky, papers done by Kaspersky or McAfee or things like that.
So they're actually designed to be a lot shorter.
Now, one of the things I quickly, again, you only realize as you're going through your research, is that I was finding many vari...
Like I showed you the example of captured before, or the word captured before.
A lot of my... I was seeing variations of the same word, and I'm like, well, how many times...
I had to ask... it made me ask myself, how many times should this word actually appear in the ontology?
Well, theoretically, only once it should appear in the ontology.
So I started looking at the unique lemmas, which is why I show that here.
And the average document had only between 0 and 500 unique lemma.
That makes... that now starts to make my identification of ontological terms a lot easier than it might have been when I was looking at 2.5 million tokens.
And this is just... excuse me... this is just a quick breakdown of the distribution.
Now the one thing this doesn't show is the total tokens, but the first two are just punctuations and digits, which usually fell out.
Now you can... the third one is actually the interesting one.
That's where the unique lemma falls, and you can see how tight of a distribution that is in that box plot.
And that's basically what I knew I needed to focus on... excuse me... that's what I needed to focus on in order to find my ontological terms.
And the second from the right is actually those common words of 1.2 million tokens that I was talking about... or 2.5 million tokens that I was talking about earlier.
So this is... this is actually some statistical analysis that I did of some terms.
Now you can see the third one there is the apostrophe S, possessive.
That's not an ontological term, but you can see how many times... it appears over 6,000 times in 338 of the documents.
It in and of itself as a token is not useful.
Now the possessive... the word that's before it is of course important.
But you can see where that's... where again this shows where even though the natural language processing does a lot to... excuse me... help me break down these documents, there's still a lot of manual... manual lifting that has to be done on my part.
And where I first had thought... I said, well, the words that I'm going to want to use are going to be those terms that they appear a lot throughout the whole corpus.
But if you look at CID, which yes has 20,000 appearances, but it actually only appears in 28 documents.
And actually of the whole, it's only 3.6% so that we... I quickly realized, well, where's my... where's the sweet spot?
Where do I need to... if I'm going to use statistics to try to identify some of my terminology.
And it came down to appearances per corpus.
Now that appearances per corpus is the number of appearances averaged out over the 425 documents.
The column to the left of that, the appearances per document, is actually documents it appears in.
So that's what explains the difference in those numbers.
Now, it's... these are only the top 15 terms.
There's actually about... the table ends up, for Lemma, ends up being about 70,000.
So I'm not going to... there's not going to be slides for everything obviously.
But it's actually ironic here that if you look, the term APT actually isn't... by this... by the designation that I've used actually comes up as number 36.
So there's a lot of terms in here that you wouldn't necessarily think of or... that I wouldn't have thought of right away as being corpus terms.
But here are things like server, attackers, C2 and CNC are actually both shorthand for command and control.
So that's the one thing that I will say that would take probably a lot more time to work on is actually going through.
Because, again, this goes back to why we were looking for ontology in the first place.
C2 and CNC are... if I said C2 to somebody and they only knew it as CNC, they wouldn't necessarily know what I'm talking about.
And that's the value of the ontology.
Now, here's a simplified overview of basically the ontology as I'm developing it.
You can see I have the tables or the portions that include focus on the organization and then the parts that focus on the attack.
So as I go to the organization, we have... as an APT organization, it's going to have a name.
Most of them right now are just APT1, APT2, APT30 is a common one if you actually Google it.
Some, but not all, will have state sponsorships.
By motivation, I was looking at things like Mice Deep.
Some are looking to cause... most are criminal activities, stealing information.
Others are looking to do malicious things along the lines of Stuxnet.
Then we go to the organization and how it's structured.
Remember I said this is a complex thing.
You have things like code writers, virus writers, web designers, system administrators.
Now these are terms we probably all could come up with a definition pretty quickly.
But finding this kind of information and seeing how complex these organizations are shows how important it is to have this ontology development.
Now as I move down to the attack, obviously this is a lot more detailed.
You have things like the name of an attack and you'll hear things like Stuxnet, IceFog.
I'm trying to think... the RSA attack.
I'm drawing a blank on the name.
But, I mean, you have a lot of... and they'll get reused and they might get renamed, but there could be a relationship.
Attributed organizations.
Attribution is one of our toughest things to do.
But sometimes there are patterns and there is information embedded within the code that actually tells us the organization.
Not too bright, but it's what it is.
And again, some APT attacks may get reused by other organizations or by the same organization later on with a different set of targets.
Now again, like I said, APT name.
Here are some examples.
ShadyRat, NightDragon, IceFog, Arachnophobia.
And that's actually one thing.
I'm not sure if you can see it clearly.
But there's a link here between an APT attack and the malware that it uses.
And Bitterbug, which I've used here, is actually a sub-component of the Arachnophobia APT.
I guess that makes sense using... with the similar names.
And here we have the types of communications that are used, which again I've mentioned earlier.
And that actually pretty much covers this.
This... the ontology is still a work in progress and I've got to get it done pretty quick because I'm graduating here in December.
I'm about to... and I'm scheduled to defend here.
But the first 22 documents produced almost a thousand terms that I personally could identify.
Using some of the statistical analysis and the larger set of documents, I'm already seeing that list growing to well over 3,000.
If I would turn that ontology and now start to use it as a search tool to search the web because I now have terms that I can use, I would expect that it would continue to expand.
And that's actually one of the future aspects of research and using potentially link analysis as a means of understanding if two different APTs, for example, use the same kind of... same means of communication.
Are there other attributes that are similar?
Are there ways that we can apply detection methods from one to the other because of the similarities that they share?
And that's where the application comes in here for cybersecurity and for us as the defenders.
Not only having the common terminology but potentially starting to mine data, big data, and find improving our detection and defense methodology.
So, I'll open up the floor to questions.
I was interested in what software you used for natural language processing.
I'm using... oh, that's good.
Sorry, I didn't mention that.
There's actually a few out there.
There's a couple of open source ones.
I actually looked at the natural language toolkit, which is Python.
But I ended up on Stanford's Core NLP.
And there's actually an extension that somebody developed.
And I'm drawing a blank in his name, so I apologize.
I want to give him credit.
He created an app called Book NLP, which is just an extension of Stanford's product, which was meant to deal with large documents, entire books versus just sentences or paragraphs or short blogs.
It's called Book NLP.
Okay, Book NLP.
It's the extension.
Now, the one thing I will say, if anybody is looking into natural language processing, at least this far, Core NLP product, actually, there have been a lot of extensions that have been developed by people doing their own research.
I didn't personally have the time, because of my timeline here, to develop my own.
The only challenge that I've seen, and also with Protege as an ontology tool, is people use it, they develop it for as much as they need, and then it doesn't quite either continue to be developed because they've run out of funding or just they finished their project,
or it doesn't quite hit the mark that you might need.
But there are a lot of, there's a lot of tools and a lot of resources out there, as even as Dr.
Raskin was talking about two weeks ago.
Sure.
The software that you used, what format was your corpus in?
And did you get your corpus off the internet?
Oh, I'm sorry, I thought I, okay.
Yes, I actually found two knowledge bases online.
They were actually both on GitHub, and they were in PDF format.
What I ended up having to do, though, was using a Python tool to actually extract the text from those documents so that they could be processed by the NLP software.
Now, some PDFs can be processed natively through the NLP, but I was getting too many errors, and I, because it was kind of stoling my progress, I made the sacrifice of losing images and maybe graphically designed PDFs, accepting that I would lose those.
And I just extracted whatever text I could get from them and processed that.
Any other questions?
I have a question.
Sure.
You mentioned text categorization.
Do you use any machine learning algorithm to categorize text?
Core NLP, well, as a natural language processing tool, some of them have some algorithms.
I didn't.
I used what came out of the box.
I used it as it came.
Again, this is that ability to extend these packages.
That's where you could add algorithms like that.
I did, and it was part of the reason for using the 22 documents was to try to train their algorithm.
It didn't prove, and I'll honestly say it could be on me because I was actually trying to learn this as I was going.
My training of the tool didn't seem to change my results very much, but you can actually train these NLP software to recognize regular expressions that might be common.
You can use regular expressions to train it and algorithms to train it.
So, yes, they are possible.
Any other questions?
Well, thank you very much.
That does it.
It's weird being on this side of the screen, I'll tell you.
Thank you.