Thank you.
Thank you very much.
When I started writing this presentation, I definitely thought to myself, geez, why haven't I been doing it this way the entire time?
And I thought maybe life really is just a series of these moments where you wonder, wow, why haven't I been doing it this way for the entire time?
So envision the last pen test or vulnerability assessment report you read, be it as a client or as a vendor, and kind of think about what's missing.
And that's really the impetus for this talk, a seed to an idea that there's a lot more that me as a pen tester can be given to my client to help them mature their entire security program rather than just feed their vulnerability management application or program or whatever particular breed of management they're using for vulnerabilities.
So there are a couple of different types of ways to test for security.
Some of them include a vulnerability assessment, penetration test, red team engagement.
These terms are used and abused quite frequently, especially by marketing people and people who have been informed by marketing people about how information security should be conducted.
Vulnerability assessment, in my opinion, is just that.
You look for vulnerabilities and you report on them.
A penetration test is a vulnerability assessment plus exploitation and plus infiltration.
It really amounts to a damage assessment.
What can happen if something were to come as a result of an exploitation of a vulnerability, for example.
The red team assessment is like the sniper rifle of penetration tests.
It's got to include some level of real-world adversary emulation in order to be a proper red team engagement.
And in terms of real-life threats, that's what the red team's role is, to actually demonstrate what's going to happen when the bad guys move in.
Some companies, of course, will try to sell you a vulnerability assessment branded as a red team engagement or as a penetration test.
There's no real good way to avoid falling into this trap without getting consultant resumes or talking about methodology in detail, what is your process.
Unfortunately, these assessments are usually very opaque.
They'll tell you, yeah, we'll do a red team assessment or yeah, we'll do a pen test.
And what they actually do when work begins is a mystery.
And unfortunately, a lot of clients aren't knowledgeable enough to know the difference.
And so what we end up with is a 300-page vulnerability report.
And the only thing you've done is made more work for your IT guys and for your application developers.
Because the primary audience of a vulnerability assessment or a penetration test report isn't the security operations team, it's not the blue team.
All they do is triage.
They take those vulnerabilities and they stick them in different databases, open tickets, and then they get somebody else to do the hard work.
Still getting used to the notion of presenter's notes in PowerPoints.
I apologize.
So I was at a client.
My very first gig with IoActive, it was wonderful.
They gave me carte blanche.
They said, here's an engagement.
Go forth and make the client happy.
And I did that.
And we came away with all something, 150 million of their credit cards and domain admin, all passwords, yadda yadda, whole kit and caboodle.
We gave them a very large report on our findings.
And we had an executive briefing in their conference room, big projection screen, very expensive table.
And everyone was nodding their heads and shaking their heads in disbelief.
They were very appreciative of the work we did and said, okay, we're going to take all of these findings and we're going to improve our security program.
I come back the next year, I do the same engagement, and I give them the same report.
No improvement.
Still got domain admin, still got all the credit cards.
Fast forward to the fourth year that I'm doing the same engagement with this client, and they're still having these problems.
And it begins to dawn on me that maybe it's not just their fault that this is happening.
Maybe there's more that I can do as a penetration tester, as a consultant, to give them more information and more capabilities to build their security program.
So we've really reached a point in the maturity of the security industry that we are in a post-vulnerability age.
We used to call this defense in depth like 10 years ago, but somehow that was just too difficult and we just never ended up doing it.
So we say, okay, now vulnerabilities don't matter.
Stop thinking about vulnerabilities.
Focus on detection.
And when doing a red team engagement, one of our primary objectives is to measure the detection capability of our organization.
Because if we can't protect against all vulnerabilities, we have to learn how to detect all of the behaviors, or some of the behaviors, or as many as we can gather intelligence on that we associate with compromise.
When we see a penetration test in progress, this is typically what it looks like.
Some pen testers are noisier or sloppier than others.
Some of them do a really good job.
Some of them document everything, have formal methodologies, really good communicators, but that's certainly the minority of penetration testers.
And again, based on the individual, that methodology can be very opaque to the client or to whoever is engaged, that particular pen tester.
And I've been to a lot of defensive security conferences lately, namely besides Gusta, Security Onion Con, SANS, DFIR Summit.
And I've talked to a lot of blue team members, and the overwhelming sentiment towards penetration testers is that they're all a bunch of jackasses.
The backlash has been primarily because they're focused on breaking, and they don't have much of the components required for helping to build in security.
And there's a disconnect, because all the red team wants to do is break, all the blue team does is want to build.
So if they're not communicating, you end up hearing things like, oh, they're just a bunch of jackasses.
We don't want to work with them, because they don't see it from our perspective.
So vulnerability assessments, or what you might call penetration tests, generally reach their peak utility once the organization begins monitoring adequately.
So you've got a vulnerability management program.
You have someone triaging and taking care of these vulnerabilities.
You have scanners in place doing automated scanning weekly, or however you choose to schedule it.
Then you begin to collect logs, put them into your SIEM solution.
You have analysts sitting in front of glass watching things happen on the network.
That is when penetration tests or vulnerability assessments are just routine.
And that's about the time you need to start thinking about doing red team exercises.
So I was lucky enough to go into a client who had had a recent infusion of security money.
And they had a SOC which had probably 50, 60, I don't even remember the count now, but 50 or 60 actual people working in the SOC, which is huge for a SOC.
And they were already processing billions of events per day.
And this was my first insight into a truly mature and well-oiled machine of a SOC.
I sat down.
I had been hired to do a penetration test in their organization.
And I started an NMAP scan, and all of a sudden my port gets turned off.
And I'm confused at first.
Nothing is working.
And I turn around and the guy behind me said, hey, my port's not working.
And he goes, oh, yeah, we probably turned that off because we detected you doing a port scan.
And it dawned on me that that was really cool.
That was like some next-level stuff, right?
I had done probably 50 or 60 assessments, weeks-long assessments previous to this, and that had never happened to me before.
And we can contrast this from a maturity standpoint with what we would call the half-security guy.
Usually an IT person who runs IT for a small organization.
And he gets, like, an email every day with all of the alerts that come out of the, like, Palo Alto or whatever device they have doing firewall and proxy.
And whether or not he reads them every day is, you know, he'll tell the compliance people one thing, but the reality will be completely different.
So the type of engagements that benefit a particular organization are going to vary pretty widely.
If I do a full-on, like, super elite, all intelligence-gathered and cutting-edge red-team assessment against that guy, that one guy who only reads the email once a day, they're not going to get much benefit because they're not going to see anything anyway.
So I like to tailor my red-team engagements so that, although I have the capability of doing super advanced things, I want to be able to do things that I know that company can detect or might have the capability of detecting, so that we can get a baseline and actually measure and quantify their detection capability.
And we can do this iteratively over time so that once they're detecting Metasploit or Meterpreter payloads running on the network or crossing network boundaries, we can move up and say, okay, we're going to do all custom payloads now so that you can't use AV or network IDS to detect those types of payloads.
And once we get up to those organizations that have huge socks and well-oiled machines, then we're doing malware analysis, then we're using cutting-edge attacker and adversary techniques in order to test their capability.
Prevention.
Vulnerability management is 100% about prevention and only that.
If you stop vulnerability management, you'll never be able to address real-life threats because, like I alluded to previously, there will always be a vulnerability, there will always be a way in, there's always going to be someone who's going to click on a link in an email and do something stupid.
So we have to have a mature detection capability.
So I was in a payment processor and a couple of years previous, we'd been doing compliance checks with these guys, just kind of check-boxing, saying, are you doing this, are you doing that?
They finally brought us in for a real pen test.
And this was quite some time after I'd visited that large stock and I'd been thinking about how to better service my clients.
And what I did was, as I went through this pen test, I opened just an Excel spreadsheet and I wrote down a timestamp, I wrote what I did, and then I wrote a source address and a destination address, if applicable.
I did this for probably 25 data points over the two-week long assessment.
And after the end of that assessment, I gave that spreadsheet to my point of contact and I asked him to take this list of data points and match them to everything in their logging capability.
I want you to go find all of these events on your network.
24 hours later, they came back to me and they said, well, we didn't detect this, and this was the Nessus scan on their internal network, probably 1,000 hosts were participating in that scan.
And it ended up being that the reason they didn't detect that scan is that somebody had unplugged the tap and nobody knows when their tap was unplugged, they don't know who did it, they don't know why it was unplugged, but at some arbitrary point in the past,
that had occurred.
And had I not gone in there and not only done a penetration test, but also told them what I had done so they could go find the activities, behaviors, and indicators that I had put out there, they would have never known this was happening until they were potentially breached and had to go do an investigation.
I do the blue team, my boss is the red team leader, and when I back some of that stuff up, or maybe late after
, and use the GUID as the user ID and the commands they're executing, and that way I can just look everywhere for that GUID, it's a smoke test.
That way I'm not looking for just specific socket information which is still helpful, but then I can just search all of our logs for that GUID and go, okay, we didn't get the system logs, or it's coming in with the wrong host name, or all that makes it easier to find.
And build that into the payload, so that once the payload's finally detected, they can go back and say, oh, okay, here's the indicator, here's the hash we need to go back and look for.
And what I also ended up doing is I started going to defensive conferences in order to learn about how the blue team is actually operating, how they defend, what they look for, what their techniques are, what their capabilities are, so that I can go in and I can subvert those capabilities.
So if I know that they're now looking for exfiltration via Dropbox, and they're looking at large Dropbox payloads, or looking for specific indicators around Dropbox, I can say, okay, I need to change it up and start uploading all of my files to Imgur and code it into JPEGs or something like that.
And that was a real good education as a penetration tester and as a red team member, is participating in and talking to the people who actually run blue teams, who are responsible for security operations, and figure out not only what they need, but also their capabilities.
And of course, without detection, there can be no response.
And that's the most critical thing that you need to take away from this talk.
If you don't have the proper detection capability, if you don't iterate, if you don't constantly improve that detection capability, and that with the assistance of the intelligence and the capabilities that the red team brings, then there's no way you're going to be able to respond to an incident.
There's no way you're ever going to know.
Security engineering.
So one of the other painful lessons that I had to learn was that it takes a lot of effort to do security engineering and security operations is a very difficult job to do.
There's so much information out there, so much traffic, so much data to sort through, and so much new intelligence coming in every single day that you barely have time to sit down and have a beer at the end of the day.
And if me as a penetration tester or red team member just comes in and dumps in a bunch of vulns or techniques or whatever and I walk away, then I'm doing them a disservice by just giving them more work instead of giving them solutions to problems that they have.
I like to casually mention that I think penetration testers have some sort of sociopathic tendencies because they don't have really good empathy for the defenders that they're actually trying to support.
And that's their job, right?
You're in there to help them support their security efforts.
It's great to solve challenges and break things every day.
It's a lot of fun.
But if you're not coming up with solutions and doing it in a way that works for the people who hired you, then you're really not doing your job as a penetration tester.
Prioritization is a huge issue in security.
And, of course, that's limited by money, time, and resources.
If I can't properly assess the real risk that's associated with a lot of the things that I do as a penetration tester, and I can't communicate that in the same language and with the same understanding as my client or whatever organization I'm working with,
then I'm also doing them a disservice because I'm just saying, Oh, you got some cross-site scripting out here, man.
You could get totally pwned.
And without any kind of thought to, does this domain actually matter?
Is there any sensitive information here?
What is the real-world impact of having a cross-site vulnerability out on our WordPress site for our press releases?
Well, if I come crying wolf every time I find something like that, they're just going to start ignoring me.
If I can't demonstrate that I understand their business and how their risk assessment works for the particular vulnerabilities that I'm reporting, then I'm just making it more work for them and making their lives difficult.
Also, automation is a huge part of security operations now, and one of the things I can do is I can tell teams how to detect things that I do automatically.
I can give them advice on, say, look for these particular changes in the file system if you want to detect this pattern of behavior that these adversaries are looking for.
This is something that blue teams should be doing on their own as part of their own intelligence-gathering operations, but because I spend all my time doing research on adversaries as well, but I do it from the perspective of offense, I can usually offer them really good solutions to those problems that they can automate away instead of having to put something in front of an analyst and have them make a decision based on my advice.
And much like understanding the real risks associated with vulnerabilities, you really have to understand how that particular business works and how the money flows, and you have to understand that everyone there that you're interacting with is there because there's a line item in a spreadsheet that has a dollar amount on it,
and that's the budget that they work under.
If you don't understand that the things you're reporting and the objectives that you're all trying to reach are part of a very constrained and very focused effort to do the most important things first, like reporting cross-site scripting is critical in any particular situation,
you're, again, giving more work for your audience than is absolutely necessary.
So, adversary emulation or adversary simulation or attacker simulation, however you choose to phrase it, this is one of the critical components of Red Team, and we need to do evil as evil does in the real world, as penetration testers participating in Red Team.
The normal path of education for penetration testers starts with, I don't know if you're still writing AOL proggies at this point, I definitely was at the beginning of my career, but the evolution of the educational process for penetration testers is almost all self-taught.
You're learning from other penetration testers if you do some on-the-job training, which almost all the penetration testers that I've met have done on-the-job training, and they've learned from others.
They look at YouTube videos, they read books.
All these things are great sources of information, but unfortunately, every single one of them is outdated.
None of them are current enough to go out and say, okay, there's some malware out here exhibiting a particular type of behavior that nobody's ever seen before, and if the only thing you do is learn from your own experience on the job or what somebody else taught you or what you read in a book,
you're not going to be at the cutting edge of your capabilities and security.
And adversary emulation absolutely depends on research that is current, malware analysis, people who have their own intelligence-gathering infrastructures.
Some of the best clients that I have have internal intelligence-gathering capabilities where they're actually looking for atomic IOCs on their network, which means file hashes, IP addresses, and when they see those things contextually, they can illuminate other behaviors that are associated with the evil guys.
So, okay, I've found this CNC signature, now I see that it's connected to a particular host in a file that I've never seen before, all of a sudden I have a new IOC, or a new payload, or I find a new variant of a piece of malware.
And the people who are doing this research are people that I need to listen to, that I need to talk to, that I need to be in a slack with in order to absorb all of that great intelligence on things that are actually happening in the real world right now and not just something that I watched on YouTube.
You know, having grown up in the hacker mindset and not wanting to get arrested, everything I had to do, I had to do that I was given permission for.
And you have to go out and you have to study the people who break the law, who do evil.
If you don't do that, your value as a penetration tester is lost at this point.
As with any business, you have to be goal-oriented or else you're just spending money and you don't know why.
And the same with attackers, as such should be the red team.
Red team absolutely needs to be goal-oriented.
You need to focus on specific objectives.
You need to look for business-critical assets.
And if you aren't already cataloging where important data is, where your critical assets are, then also don't do a red team assessment.
Do that first.
Things like credit cards, where do you store your hardware security modules, where your private certificates are, where are your production web services hosted?
Those things are particular objectives for the red team, particular flags that you want to be able to offer and say, do whatever you have to do to get here.
And giving a red team that kind of permission gives you the same kind of agency that a real-world attacker has and allows you to create and collect indicators at all points from the outside to that particular end goal, to that flag.
It also gives you the opportunity to exercise specific policy and procedural objectives.
So, like, if we say, okay, we deployed WAFs, you know, organization-wide, how has this deployment impacted our resistance to infiltration?
And that can be an objective of a red team as well.
Say, okay, guys, we put in the WAFs, we want you to get around the WAFs and see how we did there.
This is more of an edge case.
You know, this shouldn't be how red team engagements are done all the time.
But it is one of the objective, goal-oriented things that you can do as part of a red team.
The worst thing that I've seen clients do is bring me in on a network with 150,000 hosts and say, okay, we're kind of concerned about our credit card infrastructure.
I mean, they're there.
But I'll go in and then I'll have no scope.
So, the first time I go in, I'll focus on the credit cards.
But I'll come back and I'll focus on infrastructure that's connected to the credit cards.
And then I'll focus on the infrastructure that's connected to the thing that's connected to the thing that's connected to the credit cards.
And the scope will just get wider and move further out.
Not giving... in these cases, your client is expecting you to come back with a vulnerability report.
They want to know all the vulnerabilities that you detected during this process.
And the problem with that, of course, is that the scope keeps expanding and your report keeps expanding as you've scanned 100,000 hosts and every single one of them has an SSL problem.
That's a lot of time wasted and that's a lot of effort for the IT team that's unnecessarily spent putting new certificates on printers.
Tools and tactics.
What changed most dramatically about the tools that I use when participating in Red Team engagements is not the tools themselves, but how I use them.
For example, with Cobalt Strike, the default C2, the malleable C2 plugins, which is essentially how you define how communication happens between beacons.
The default installation of Cobalt Strike is instantly detectable by Windows Defender.
So if you've taken no effort to go in and modify the tools that you're working with to remove particular signatures that these network IDS or host-based or endpoint protection is looking for, then you're being silly.
Because nobody in China is using Meterpreter in order to conduct their exploits.
So you have to tailor your tools specifically for stealth.
You can't use nmap too much effect anymore, because what I'll have to end up doing is scan one port at a time across multiple hosts, but only send out one packet every 30 seconds.
And in order to scan a class B, that would take a couple of years.
You can't just say, okay, I want to dash T2 for this engagement and then hope nmap is doing the right thing to avoid the signature on the network intrusion detection module or the IDS in order to keep you safe from the blue team.
You have to understand very specifically what those signatures may be, and then use the timing parameters on nmap in order to make sure that no more than one packet every 30 seconds goes out, regardless of the other settings that you've applied to nmap.
You can use Nessus, and I've used Nessus on a red team engagement, but the sole purpose of using Nessus was to create an ungodly amount of noise on some other part of the network.
So they were busy over there taking care of that problem while I was somewhere else being nice and quiet.
You can't use things like vulnerability scanners on a red team engagement.
And if you are responsible for buying these types of services and your red team engagement included scans from a vulnerability scanner, ask for your money back.
Okay, so one of the other things that I generally never did as a penetration tester is I never cleaned up after myself.
I never went into the logs and removed entry.
I never disabled logging capability.
Never tried to tamper with anything that might leave a signature.
If I didn't have a whole lot of time on engagement, I would just leave things on the hosts and tell the client, this is all the stuff I left.
Go back and clean it up.
That sounds a bit, you know, not very courteous, but that's just kind of the behaviors that have been built into the penetration testing process and that we've all been used to.
So the most successful breaches in recent history have been the result of social engineering attacks.
I can give you a lot of examples, but I'm sure you're already familiar with them.
So as a penetration tester, I've encountered lots of people who have dabbled in social engineering or have this, oh man, I'd love to learn social engineering.
But if you're working as part of a red team, either you or someone on that red team must have a lot of social engineering experience.
Because the first thing you're going to do when you come in and start a red team engagement is assess that organization for the people, what they're responsible for, and what the best avenue of infiltration is going to be.
And that's almost always going to be a social engineering email at this point.
Companies have been very good at, you know, buying QALYS services to scan their external surface area.
And you're going to run into a lot of problems trying to exploit organizations from the outside using vulnerable network services at this point.
So the best thing you can do is, well, probably read a book on social engineering, because that's how I learned.
Social engineering is very nuanced.
You have to take a lot of time to think about how people interact with things like their email every day.
Do they just click on every link that comes from it at example.com, example being their organization?
You have to learn from real world attackers again.
You can see these types of emails that come in.
If you just look in your spam folder, you'll see all the different ways people try to get you to open links or open attachments.
It's a fiddly little process, but learning how to socially engineer people is really fun.
And if you can remove the fear of being caught or feeling like an imposter by saying, okay, I'm supposed to be doing this.
This is my job.
I'm supposed to be this person.
You're not pretending to be that person.
You are becoming that person.
This is a long process.
It took me a couple years to learn how to do it right.
But social engineering, super important on a red team engagement.
Speaking of teams, red team must absolutely be a team.
If you start a red team engagement and there's only one guy who shows up, that's not a red team engagement, because the perspective of that one guy is going to be extremely limited.
Me, I like to think of myself as a jack of all trades.
But of course, the problem with that is that I can't write exploits very well, like binary exploitation.
I can't really do it.
That's a rabbit hole that I never chose to go down.
I bought the shell coders handbook.
I got to chapter four and I'm like, man, do I really want to go through all this BS?
So what I ended up doing is saying, okay, whenever I come into a situation where I think that might be necessary, when I need something weaponized or I find something that may be exploitable, I'll hand that off to somebody else on my team.
I'll let him or her worry about it.
And then they'll send me an email back with an executable.
And I'm like, good, I'm good to go, man.
Thanks a lot.
So that's what it means to be working as a team.
You have somebody who knows about social engineering, knows about exploit writing, knows about covert C2 channels.
Somebody who knows how to spin up instances in AWS to use as proxies to make sure that you don't come from a single source IP address.
Someone to sit just with Cobalt Strike open, waiting for beacons to come back and begin the process.
Somebody who knows about physical security, somebody who can take a device that they've built and plug it into the network when they've piggybacked behind somebody else in the organization.
And once that thing comes back to Cobalt Strike, you say, okay, now next guy takes over on the team.
So it's very important that you have a diverse set of perspectives on security testing on your red team.
And make sure that it's not just one source of intelligence that you're looking at, not just one perspective.
I'm repeating myself, I apologize.
So collaboration and participation.
So a lot of my clients ask me, you know, who should participate in the meetings that we have during our red team engagements?
And I have regular checkpoints in our red team engagements.
We'll sit down and we'll discuss what we did and what they detected, ways we can improve detection based on what they saw.
And if we're confident that they won't detect something that we know we did and they haven't detected yet, we'll give them hints and try to push them in the right direction.
We don't tell them what we're going to do in the future.
Of course, that's counterproductive.
You don't want to keep too many secrets in a red team engagement, though.
So you want to be able to communicate clearly with the blue team so that they can understand and they can improve.
But you don't want to give them the capability of predicting where you'll be in the future so that they cheat.
You don't want them to be able to cheat, right?
There is no red team without blue team, unfortunately, because by definition, red team, the goal is to educate the defender.
And if there's no one at the organization who can sit down with the red team and have this discussion, no one there to ingest the information and the data that the red team generates, then you're wasting your money.
You might as well just go back to doing regular pen tests or vulnerability assessments until you reach that maturity point.
Until you have enough people, enough bodies on the security operations side.
And speaking of security operations, you should have people from...
You should have at least one person who sits in front of glass.
You should have someone who does configurations.
You should have someone who does malware reverse engineering.
Diverse perspectives on the blue side that participate in these meetings so that every time the red team comes down and spills their guts, there are people there to listen and learn from that process.
And vice versa, the red team should also sit down and listen to what the blue team's feedback is for that particular cycle of the engagement.
The most mature organizations that I've worked with actually swap members of the red team and blue team so that they can better develop that empathy.
So you take a member from the blue team, they shadow or OJT on the red team for like three or four weeks.
And then you go through another cycle and you change them out so that they understand, right?
They see it from the other perspective.
Documentation.
As a penetration tester for the vast majority of my career, and much like all other penetration testers, I resisted the request to have my process documented, to write down how I did my job.
That always feels like a threat to your creative freedom in the offensive role.
I hear people tell me all the time that they don't want to log what they're doing because they don't want to take the time to open a spreadsheet or open a text file and write down what they did moment by moment in a gig.
And, you know, I can understand that from the perspective of, well, maybe they're trying to track like how much work I'm actually doing during engagement.
You know, I don't want them to find out that I watched Netflix for three hours today instead of working.
But...
It was on the back.
You don't have to tell me, man.
The importance of being able to document your methodology and your techniques during this process is critical for allowing the organization to learn from your activities.
And it's really not that hard.
It takes me like ten minutes, ultimately, of effort during a two-week engagement to put in those little entries in my log saying what I did where.
And, yeah, it looks a little sparse.
It looks like I didn't do a whole lot.
But that's the caveat you deliver when you give them this document.
You say here are some major indicators that I've generated that I think you should look for.
It's not every single port.
It's not every single service.
It's not every single technique.
But it's things that I feel are important that will help you develop your detection capability in any particular part of the network.
And it's also really important to be capable of making a plan for testing.
Penetration testers, I know, have a difficulty with this as well.
Expressing that it would restrict their creative freedom, offensively, for them to write out a test plan for a penetration test.
And while I can see both sides of this argument, it's important that a penetration tester be able to say, okay, first I'm going to use MS0867.
And then I'm going to migrate to another process.
Then I'm going to attempt to do a local privilege escalation.
And then I'm going to yada, yada, yada.
If you can't make test cases or test plans in that regard, if you don't even have that desired capability, then you're not going to function well on a red team.
Because at some point, the process will become mature enough that you have to start with a plan.
And if you don't start with a plan, then everyone drifts off focus.
People lose sight of the goal.
And you're wasting a lot of time and money doing testing.
So I'm a little biased.
I work for a consulting company.
And I recommend to every client or every person that I talk to who does penetration testing that they bring in a third party to do testing for penetration testing, red team, compliance, vulnerability assessment, client advisory.
Like, is your process mature?
Let's look at your process and your policies and see if they're adequate for what you do as a business.
The fresh perspective of having a third party come in and look at how you do things is so valuable.
The blindness and the assumptions that build up the longer you're at a company and the longer you're in the crap, the less effective you're going to be at assessing your own vulnerability.
And even if it's just once a year or once every two years, bring in a third party red team to do an assessment.
Have them sit down with your own red team if you have one.
Have them learn from that process.
Have them participate so that they can bring in fresh ideas and new techniques.
Encouraging people to learn and be constantly vigilant and researching is difficult.
It's even more so internally in an organization where that red team is so familiar with that environment, they already have a lot of preconceived notions about what's possible.
And a lot of that is driven by their own... I struggle to use the word ignorance, but obviously if you don't know everything, you don't know everything.
And if you continue in that process, eventually you'll get stale.
And that happens quite frequently, and I know because I've talked to leaders of blue teams that say, okay, you know, our red team is awesome, and our blue team is awesome, but our blue team is really only good at catching our red team.
And that's manifested in a discussion that I had, a different discussion, where they had a standard rat build.
Like, their payload was just the payload that the red team used.
And it evolved over time, of course, but once the blue team has the hash, the IOC of that payload, you know, they're busted.
And forevermore, because now you can detect the types of functions and keywords in that particular rat build.
And in that case, they whitelisted the rat.
The SOC would no longer alert on that particular IOC.
So you put on your blinders a little bit and say, okay, we'll use a standard rat, but we just won't pay attention to it.
A lot of people disagree with me that that's a good idea.
I'm kind of on the fence about it.
It's difficult to empower your red team to spend a lot of time developing new payloads every cycle.
Something new every time.
Something small, something agile, something purpose-built, and have to do that on a continuous basis.
That's just very resource-intensive.
So I'll leave it up to you whether or not you want to whitelist or not.
Use that methodology.
Intel gathering.
So I talked a little bit about the sources that I use to gather information about real-world attackers.
I'll be honest, the primary source is Twitter now.
Because I follow a lot of really smart people on Twitter.
Malware researchers, people on blue teams, people who investigate APTs.
I read a lot of white papers that places like Mandiant put out.
And look at the behaviors that are kind of cutting-edge on the offensive front.
Even the government agencies, like looking at NSA leaks.
Wow, that's a really cool idea.
I'm going to try that out.
All of these different sources, and it varies wildly.
It's difficult to find one really particular awesome place to find all this stuff.
So it's a constant process for me, personally, spending a lot of my time going out and trying to find what's the next best thing.
Where am I going to find the new techniques?
Who's doing the bad stuff, and how can I learn from them?
You can forget about IOCs, atomic IOCs, IP addresses, and file hashes as a red team.
The only thing that those are good for is making sure that your antivirus is working properly.
So if you're going to try to test the detection capability of AV or IDS, you might as well just use ICAR, and just drop it someplace randomly and see if the blue team has detected it.
Like, oh, I've been here for two days, they haven't detected me, just drop ICAR and then see what happens.
Oh, ICAR is whitelisted on your glass, huh?
Oh boy.
Those IOCs are ephemeral anyway, they'll change day to day and they just won't make much of a difference in the long term.
Science.
So I set myself up a lab at home, it's ESXi on a small mini-ITX.
It's got a bunch of RAM and an SSD in it.
It's got 12 VMs running currently, full Windows domain, some Linux, some OSX.
And I do all of my... I call them, as I've learned from somebody else and I don't recall their name, unfortunately, shell pinatas, where I'll just test all of my techniques.
And I have the vSwitch for ESXi in hub mode, which means all the traffic goes everywhere, and I have Security Onion running on that network, which means I have a monitoring infrastructure on my lab.
And that allows me to look at my traffic, my patterns of behavior, the file changes, from the perspective of a blue team.
I'm seeing exactly what somebody else would be seeing if they were watching me do these attacks.
And it allows me to tailor my attacks to reduce my footprint.
So in all of this, I've learned that blue teams have ultimately a whole lot of scary access to the assets that they have under their purview.
And if you talk to people on blue teams, they're like, it's a corporate device, it's a corporate network, why are you on Facebook and not expect me to look at the stuff you're posting?
I mean, geez, this is a business, man, come on.
You know, people say that, but the reality is that these are human beings that are using these networks and using these computers.
So you're almost exactly like the NSA of your corporation, if you're in SecOps, if you're on the blue team.
The amount of information that you have access to and the percentage of that information that's personal to somebody, it's not insignificant.
And if you come into a situation where you have dramatic access to an infrastructure, for example, at the DFIR summit, there was somebody who was doing a presentation on a tool.
It was an agent that ran on everything in the infrastructure.
And what this agent allowed you to do was to query both disk and memory in real time.
So if I found the string teddy bear, and this was part of an attacker pattern that I understood, so I just typed teddy bear into this tool and it searched every endpoint that was running Windows or Linux or OSX in the network and returned everywhere it found that signature.
Now imagine the kind of power that that gives an individual if you're trying to do malice within a network.
If you're curious about who's going on adultfriendfinder.com, right?
So you have a responsibility for those members here of the blue team and penetration testers who get access to these things through their efforts to be very cautious about how they use the information they see.
If you find something, should you report it if it's worthwhile?
You have to ask yourself and be very confident in your convictions about how you use these tools and capabilities.
Also, additionally, so one of the questions I asked the person who presented on this tool was, how do you control access to the tool and how do you audit functions within this tool?
So, I mean, we've seen other presentations where people are developing these types of capabilities and tools and you have to ask them, how do you restrict access to this tool?
How do you prevent people from abusing this access?
And it could be trivial.
It could be something simple like somebody looking for, you know, people cheating on their spouses.
But it could be something much more nefarious.
How can you know you can really trust everybody?
And that means you shouldn't allow that to happen.
You should have some sort of peer review mechanism in these tools in order to prevent people from going and doing arbitrary things in the network.
And one of the things that they had implemented, and I was very happy about this, is peer review.
But unfortunately, the peer review just required the person in charge of the tool to approve the queries that other people had made.
So you have to kind of think about, okay, so the cat is watching, or the fox is watching the hen house, or whatever.
Anyway, be very cautious about, be conscientious about these things.
We're all working towards a single goal, and that's to keep evil off of our networks.
And we should always act in that interest.
We should, you know, not just act in our own interest.
Oh, I want to do cool things.
I want to break stuff.
You have to remember that if you're in the security industry, and if you're being paid to do security, the ultimate goal is to keep evil off of our infrastructures.
And always act within that interest.
Be kind.
Share information.
And if you write a tool, try to convince your employer to open source it.
That's all I have for you guys.
Thank you very much for listening.
Does anybody have any questions, or want to argue with me about something I said?
Sure.
So how often do I have to educate organizations on what maturity is?
It's almost the first part of the conversation, even before...
And it's normally part of the sales process, where a salesperson will be having a discussion with a high-level member of a client.
And they'll say, OK, we want a red team engagement.
I heard about this on CNN.
We need a red team engagement.
And so part of the pre-sales process is me on the call, asking questions about the client's maturity level.
Say , do you guys do logging and monitoring?
How many people on your security operations team?
Do you do automated vulnerability scanning?
What's your vulnerability management program look like?
Do you have the time to give me three people from your team to sit down for two hours a week and talk about what we're doing?
And sometimes that's even a deal-breaker.
And the time scales that I like to work on...
Red team engagement should last a couple months.
It shouldn't be on the order of weeks.
These are campaigns that we're trying to emulate.
I'm talking to a client, and they can't explain to me why they should be running red team within my own definition.
Then we talk about something else.
We say, OK, let's scale this down a bit, and let's try to evolve your capability.
If you can't detect the things that we're doing yet, then we need to get you to the point where you're capable of building a detection capability.
And that happens pretty much with every conversation.
There's very few companies that we work with that have fully mature capabilities, have their own red teams, have their own blue teams, and are doing it all correctly.
So that conversation is consistent.
Any other questions?
I have a quick question from the couch.
Oh, from the couch.
As somebody who spent most of his time on the blue side, you made a statement that watching the YouTube videos on the how-tos and the exploits and stuff like that, they're all really pretty much dated by the time they get online.
I would argue that from either the red or blue side, the value in being able to have those online and watch them and spend time analyzing, learning them, and understanding them isn't necessarily so much about the specific technical exploit, but really about getting into that hacker mindset that you explained.
Because as an incident responder, I had a much better success in my career understanding how the red team thought, and how they organized themselves, how they really approached that attack.
Right, right.
And that's part of the thinking behind swapping members of the red team and blue team.
It's nice to say that as a SOC analyst, I should spend a portion of my time doing research on offense, but that's not always a luxury that they have, and it's not always a desire that they have.
So doing professional development on the defensive side has to have some component of, okay, we need to slice out some people's time in order to be able to train them properly and get them thinking in the right mindset.
Right.
That's really almost kind of a nice way of saying they're not always that smart.
But no, but it's your duty to make them that smart, right?
By allowing some of that time to go spend shadowing other folks.
Yeah, and I mean, swapping is the easiest thing to do because it's a smaller risk on resources.
It's a smaller time crunch.
Just with my experience, it's very difficult to give time for professional development for any part of an organization.
And especially so for security, because everyone's spending their time on what's in front of them.
So you have to have maturity in the management chain in your security organization in order to say, okay, we need to do training and make sure that these people understand exactly what they're supposed to be doing.
They're not just looking at alerts and saying, that looks bad, that might be bad, that looks good.
They have to understand why these things bubble up to the surface and what behaviors could be associated and stacked against the things that they're seeing.
So absolutely, yes, do whatever you can to get yourself trained as a defender on the offensive side.
But like this talk is, as an offensive team member, get yourself trained on the defensive side.
That cross-pollination and that communication and that empathy is everything in making those two programs successful.
Thank you.
Thank you very much, everyone.