[Music]
so I'm gonna be talking about open
source pen testing now a lot of you
already use Metasploit right so that's
your you're done that's all I'm gonna
play talking about um so this is me just
for an introduction I actually started
off with just the dad thing and then I
got addicted to Google Images as we all
do and just threw everything up there if
you didn't notice the my CH is expired
I can't I'm not allowed to hack anymore
because it's expired so why why made
these documents it's really because I'm
tired of recreating them at every job
that I go to tired of different pen
testing companies having so many
different formats and scenarios like if
you take ten of even the best pen to
interesting companies that are probably
in this room and put them together and
say what kind of pen test can you do I
guarantee there won't be a similarity in
a single one of them other than they do
pen testing or red teaming or whatever
they want to call adversarial simulation
and there's really a lot of times no
expectations on either side other than
you're gonna break in we're gonna get a
gigantic report and we're gonna have to
do some extra work right too many
different levels on communication how
many how many pen testers do I have in
the room okay how many have you had a
customer client partner person that is
your point of contact that wanted
communications every day yeah that's fun
isn't it it's awesome how about every
hour yeah yeah we were right I on that
one there was a specific customer we
both had
good starting point for anyone starting
fresh so the reason I am open sourcing
these documents is because I want it so
that anyone near the industry can start
off and be able to hit the ground
running with documents they don't want
to make like have you any of you ever
had to make an NDA from scratch not fun
or steal it from the internet
so what are these open source documents
they are a NDA a pitch an Esso w
statement of work are OE rules of
engagement a few others that you might
not have heard of or had any idea that
you needed first one is the deliverables
request very few times have I ever seen
a pen test company or even teams that
have been on or me asked for a certain
number of deliverables on a specific
date right give me the IP scope or IP
ranges that you have in scope on this
date it's kind of normal
what about what's there as you want or
or give me some users to target it's
usually like halfway through the
engagement that you get that kind of
stuff right so this is a concrete
document that says I need these things
at this specific time or else we have
and that that's the important piece the
if statement right if then or else we
have the right to reschedule right this
way you're setting expectations and no
one can get mad I mean they still get
mad but whatever and engages report this
is just a report but it's in a little
different format than normal and we'll
get over it
so who's this for anyone who does pen
testing also blue or people who want to
get a pen test in because nine times out
of ten I mean how many let's just do it
on the audience how many of you have
ever had a customer that just said I
want a pen test and had no idea what
that meant right or I want a red team
that's the new hotness right or I want
an adversarial simulation hey it's
impossible for you for IT guys or anyone
to keep up with our industry moves too
fast so I don't fault them for it right
but this thought these set of documents
are here so that they can go and see
what's available kind of like a menu
right instead of having to do this so
all of these documents are Creative
Commons as open sources that can be you
can steal them you can use them as your
own I don't care they're just they're
mostly selfish reasons so I don't have
to make them again how can you
contribute back now I don't expect all
of you to to rewrite a bunch of
documents for me because I've already
done it but I if you have comments if
you have a customer who doesn't like one
of the things in here or doesn't like
how its worded or anything if there's
any point in time that you actually use
these documents and say that's stupid
I'm okay with that please make a comment
or email me and say hey that's dumb and
I'm totally fine with that you will not
insult me so let's see the documents
let's actually go into one of them and
the one we're gonna go into is the pitch
because the pitch is the most important
part it has all of the links to the
documents that I'm going to be open
sourcing it also has all of the
descriptions so for here on out if you
don't see Halloween stuff pretend I'm
talking to a client or customer or
whatever so this is intended for them
not you guys you on you understand most
of this okay protein pen testing so a
quick you can remove this slide when you
get it but this is basically the authors
that helped me get here
terminology yes I understand we all
redefine everything on a daily basis
however try finding a definition for pen
testing or writing any definition that
can be clear and concise and you can say
it within an hour time frame
so I made my own as everyone does pen
testing to me is more about prevention
red teaming is more about detection in
response and that's how I define it
there was a really great tweet the other
day when I asked about this that pen
testing is is a punching a punching bag
and red teaming is having a sparring
partner and I thought that was amazing
so for the rest of this talk we're gonna
I'm gonna switch terminology and I'm
gonna screw it up because I'm on kind of
on stage but I'm gonna talk about
engagements so when an engagement can be
a pen test it can be a red team it can
be an adversarial simulation it's it's
an engagement right the other thing is
that I know I'm gonna I'm an info sock
tree hugger InfoSec tree hugger so I
want everyone to be nice and happy and
and get along so I'm gonna refer to
everyone as all the clients and
customers as partners now this actually
started as a suggestion that was not
mine inside of GE when I worked there
and it was awesome I fought against it
initially but then I saw the the the
turnaround when our communications with
our partners turned non-adversarial
psychologically this is weird it just
clicks in some people's heads it's
awesome it's hacking people so mindset
you guys kind of already know this this
is why you're doing the stuff so this
the client already kind of know or this
is mostly for the client setting goals
you guys already know this this is
setting useful goals for the client or
already messed it up partner for the
partner saying hey DA's fine he is
awesome but if there's nothing attached
to it you're not really getting the
value out of it core company software
that kind of stuff one of the best
engagements I was ever on
we targeted not domain I mean not any
kind of administrative rights we
targeted the software developer that if
if he left the company the company died
right and that was really horrible
management on their part but it that's
how it was so we targeted him got in got
his source code and then gauged it over
I don't think we ever gotten domain
admin or even administrative rights on
anything but his box communication
timeline this is an example of a
communication timeline this goes in the
report it says these are the things that
happened have you ever had an
unscheduled outage right a box goes down
you think it's you and it's just them
taking down a server because someone
messed up and then they blame it on you
usually right non-engagement related
intrusion detection now I'll give you a
hint if you ever interview at GE for the
red team
there is prior or there's this oh
there's this pen test that you have to
do as part of the interview process
don't tell anyone this is not recorded
right during that during that interview
process there is a prior intrusion
inside the pen test if you don't stop
the test and notify the point of contact
you pretty much fail the interview right
so have you ever been on a test that you
saw prior intrusion on right did you
stop the test
everyone's gonna answer yes right
because I just made that a ding all
right so engagement related intrusion
detected this is mostly just showing how
things work there along the on long be
engagement this is just more information
about it and these are the things that
are in the documents so during the pitch
this is what you're gonna say hey I'm
gonna give you all of these things and
here's what's in them this is where your
expectations one of the biggest things I
learned throughout my career is that if
you don't set expectations people get
mad when something doesn't come through
that they assume you're supposed to do
so agreements on both parties
Costin and scenarios included in the
gauge mint oh well good we'll be going
into the costs and scenarios I'm sorry
they no one wants to share costs the
scenarios the stopping points are or
limits to the engagement and ROA is
basically saying I won't go past this
line there are some there are some
companies that say if you get to box X
pen test over right they and that's
that's totally fine if that's what they
want like if it's a SCADA system or if
it's a airline control system or
something like that they don't want you
to touch it they say if you can if you
can like see it on the network stop
that's totally fine but that's the what
the documents therefore that's how you
set expectations that's why you know
what's there and a lot of these
documents have examples in them so that
you can kind of level set with the
customer or though with partner as well
the Liberals request these are all the
deliveries I kind of already went over
that the DECA this is where you level
set communications when I'm going to
talk to you and what happens like a
decision tree or communication tree when
something goes wrong has anyone in here
ever had a partner a customer client
that went on vacation during engagement
when something went wrong obviously
right something always goes wrong when
the customer when the partner goes on
vacation who do you call after that no
that's what this documents for it's like
these old phone trees anyone old enough
to know what a phone tree is I
I had one too all right
this is the rubric the next thing that's
the reason why these things are here
right
I almost on every test that I've ever
been on they wanted numbers or colors or
an easy way to identify how they're
doing how am i compared to the rest of
this small businesses am I good am i
okay am I not doing so good and how do
you qualify that now it took me a long
time to figure out what to do here
like do I say you got a five is that a
five out of five or ten out of five
like what here's a number that's really
what it boils down to doesn't it like
you got a green so we there are other
examples of this they're kind of hard to
fill out but they're good they're where
the industry is going so what this
rubric is is essentially there to make
it easy for a customer or partner to
fill out as well as you so this is what
I came up with a good friend of mine
Adam SLO and it works really well
because I can give it to a partner at
the beginning of engagement they can
fill that out and during the engagement
I fill it out and at the end we compare
scores or I just give them my side and
they compare scores themselves this way
next year when they get their next test
they can fill out again see how they're
doing and again and it's really been
amazing to see companies take this
rubric and go I'm actually getting
better and I have to ask me it's great
they do stuff on their own yay not
belittling partners at all so this is
what it boils down to did I notice it
easily bypassed bypass with effort not
bypassed this is version one I've had
some awesome recommendations already to
make this vert this better relate it
more to miter and
or categories I am totally open for this
changing na means they don't have it at
all not deployed yet is I've bought it
but I haven't done anything with it yet
which happens a lot
I got a blinky box it's plugged into my
network it's supposed to do good things
right silence
deployed in watching fully deployed
actively defending it's not active
defense right so these are the scores
this is how it boils down and this is an
actual customer that we had we gave them
a 50 they thought they had a 63 and it
was great because we could identify the
differences what we actually saw
compared to what they thought they had
and then they could improve so for
example their event logging stuff we
thought it was amazing they had all
these all the event logging turned on
and going to Splunk and we thought that
they were doing amazing we couldn't get
around it because all of their logs were
like it was in a millisecond or whatever
being shipped off to another server but
they weren't looking at it at all right
so bypassed or not and stuff like that
so this is how that works and it works
great over time so attack narrative
diagram that's within the end of
engagement report the ER now this is a
little different than most reports this
is a PowerPoint presentation versus a 40
page report this is really cool because
I think that it it changes things a
little bit because what you're doing is
at the end of the report instead of just
shipping them a report and saying let's
go over some of the findings this is a
PowerPoint presentation of some of the
findings these are the important
findings that we saw and this is you're
basically in in this report doing the
attack narrative and showing them how
you got through things instead of here's
40,000 findings in necess that I gave
you right we all used to use necessary
or
I mean that's how we do pentas okay just
making sure
um I wanted level set so it also
includes prevention and detection
opportunities how many of us have ever
said you could have caught me here on
report
not many okay how many of us said here's
some stuff to clean up that our
engagement ended and we couldn't clean
up this is being honest with a partner
honest with the client right I know
someone who left a binary all over a
network at some point in their career
that got detected about a week or month
later and the customer called him back
and said sorry for stealing your story
so it was been night the next day I'm
sorry so if you don't like my pen test
report example there's a vast majority
of them wow that doesn't turn out very
well
QR code go ahead scan the QR code
randomly on the internet but this is a
great example of all of some different
companies doing pen test reports and how
they work there's a lot of different
differences in all of them but find what
works for you or find what works for
your partners and use those if you find
something that works really well submit
it as a comment back to the docs make
sure that we help get everyone better
write some methodology PETA's is a great
set of methodologies it is vast anyone
ever try and read PETA's keyword try
doesn't apply to everything common fears
outages everyone always asks the
question especially in in the initial
pitch what are you gonna break stuff no
thank you
how do you answer that with a straight
face it's usually on a call so it's good
but like we're gonna take every possible
precaution actually mess this up I think
yeah this is still the binaries and
stuff so there's a physical damages part
that is in the actual document that
didn't get synced over here so here's
where I talk about examples I'm gonna
doing on time whoa really fast that's
awesome
so the opportunistic attacker so this
set of documents has a number of
different scenarios ending and I call
them scenarios because you have to get
into the mindset of that individual or
that person right instead of instead of
what Rob can do it's what the
opportunistic attacker can do right so
in this one it's extremely loud this is
a level set engagement this is a place
where this is the scenario that you add
to the DSO W that says we just want a
level set if you don't know what a pen
test is and you had to we had to read
through the definition and kind of talk
about it we're going to level set your
organization we're gonna say I'm gonna
run every possible tool from the
internet and see if I pop you if I pop
you with necess or an expose or any one
of the other tools out there we got
bigger problems and we're gonna work on
that right this is a level set
engagement and we separate attack
techniques by day for ease of
identification later so we can say hey
we're an nmap on this date did you see
it and stuff like that for for the
detection piece but it's mostly about
prevention so this is more on the pen
test side of things right this slide
looks like crud but it actually looks
better in the slide in the thing so this
is where we're talking about the
information exchange saying I'm gonna
give you these things you're gonna give
me these things and this is where the
deliverables request comes in right
before the engagement on these dates
after the engagement on those days
I tried to put it in there it didn't fit
so we're gonna talk more about scenarios
and so opportunistic attacker we already
talked about and these are just these
scenarios that I have that I created by
via the all of the other authors that
have helped me with create this right so
external with the credentials so
external with credentials is essentially
we've all heard that there's been one or
two breaches in the last couple years
right right where people have gotten
passwords I mean it's kind of not really
in the news like you got to kind of look
for it but it's out there right so what
happens if someone gets a set of
credentials that works for your company
well the partners company right so in
this scenario the partner gives you a
set of credentials that have been stolen
or whatever and you just see what you
can get access to can you get can you
access OWA what can you do from there
can you ask this office 365 without
two-factor authentication see what you
can do for there can you get on the VPN
can you do this can you do that can you
do this anything with credentials I've
had partners that didn't know they had
SharePoint on the Internet not a good
idea right or ntlm authentication at all
anywhere right so this is just a
fact-finding engagement mostly to see
what you can get access to a lot of
companies just didn't know they had X
stolen laptop stolen mobile this is a
three part scenario first part they give
you a mobile device or computer or a
laptop that has been provisioned for
that company it is locked out I have
just stolen it from a car not really
that's kind of hurts because I had a
laptop stolen and they gave it to you
locked out and and they say
what can you do with this thing can you
reboot it in empty password it and
change the password and login and then
once you're logged in can you get on the
VPN can you get on the Wi-Fi automatic
or can you do this can you do that the
next part is they give you a set of
credentials to log into the box or the
mobile device and then you try that out
you see what you can do from there see
if you can access the shares or or
whatever their office 365 automatically
and then final scenario final part of
the scenario they initiate their stolen
pass or a stolen laptop or stolen mobile
device process if they have one they say
on MDM rais'd erase all of the data I've
done this engagement a number of times
you know how many times the data
actually got erased from MDM zero this
is pretty sad right but they didn't know
they thought their their tool worked
it's supposed to work so the other thing
that we go into the on this specific
scenario is the the process that you had
to say that it was it was stolen did you
then go back in logs at all and see if
any authentication with that user
happened right this all makes perfect
sense to us once we think about it right
do you go and say did after this after
we deleted and said that laptop is off
the network forever and ever and ever
right
did it ever access the network again did
we put an alert in that said hey that
laptop was stolen if it's seen again
alert us instantly cuz that's a known
bad right there's never gonna be a false
positive there right unless they named
the
the exact same thing automatically for
some reason breach simulation we kind of
already mostly know about as an
organized a community it's I don't want
to do fishing that's that's the scenario
right I don't want to do fishing I know
you can get in
that's not how partners sound right so
this starts with a binary getting added
to a machine
they run the binary then you continue
from there that's really what it is but
the thing about this scenario the the
slight or you modification of this is
the tap on the shoulder piece so the tap
on the shoulder piece is hey I'm the CSO
I need you to run this this binary on
this box to a random person not not one
that's been provisioned oh my gosh I had
one engagement one time that some what
that their idea of a breach simulation
was a provisioned machine inside their
server network that was segmented off
from everything right here go have fun
try anything like okay there's no real
user there's no real data there's no
real anything and they just provisioned
it as a VM they had access to nothing
yeah you win so so the breach simulation
is a tap on the shoulder binary setup
right so you have just been fished let
us know if you see anything weird right
insider now this is another piece where
it's no longer Rob what Rob can do this
is a non-technical insider so they gave
us RDP access or straight-up like giving
us a laptop and have someone on site
doing this where you start downloading
in map you do Google searches for like
how to hack 101 right this is showing
inside real insider threat right you
have to get into that mindset you can't
use all of sub T stuff just because you
know it and these
Insider might not right this is also
insider this is really going at your oh
the downloading and downloading stuff
from shares and stuff like that
so if I downloaded 500 gigs of data from
your corporate share will you see it
right is that on your DLP solution is
that on your insider threat awesome
blinky box and then black box this is a
zero knowledge we'd know about this
right this is what Rob can do or what
what I can do as an engagement and then
we have the add-ons
now these add-ons can be their own
scenario but normally they're attached
to something else so see Excel breach
training is one of my favorite anytime I
get to do this this is amazing because
how this works is there's no there's no
computers involved we go and we say all
right c-level executive there are all
these breaches that go on all the time
and every single c-level executive that
gets in front of a news reporter sucks
they have no idea what technical things
they need to talk about right so what we
do is we give them here is a report
that's really technical about the breach
we want you to read this you have about
five minutes ten minutes and then I
which is so much fun get to sit down in
front of them and pretend to be a
reporter now we use stock questions but
they're mean questions they're really
mean questions and we ask them all these
things and they get flustered they they
they don't like it initially it's not
great as you would in front of a real
reporter but they quickly see the the
most of them quickly see the the value
in it when they go back and start asking
real questions to their their instant
responder that we have so two two people
on the engagement the answer responder
and the reporter
sometimes they can do one so they start
asking better questions then you sit
them down again and we have other
questions that are even meaner which is
so much fun but at the end of this at
the end of this engagement you have you
now have a partner in crime
you have someone who understands for a
piece of it what you do and why you do
it and they are they have a lot of fun
right and then they say hey you should
get the CTO to do this I want to watch
and the CTO sits down and you see him
like crying it's great but this that's
definitely one of my favorites so
password auditing this one's important
to even say hey we can do this if you
are not in Kerry
Kerry's talk I think it's tomorrow about
her tool that does pass for robbing
stuff you should be but password
auditing you can say hey first day of
the engagement give us access to your
domain controller how many of you think
that goes over well so you say you know
we're adults we can actually separate
that what we're going to do is say hey
best use of your time is we want to do
an audit of your passwords and say how
good are you at doing passwords that way
if we have this as a finding that we
kind of saw during the engagement right
we say oh well your your SPN passwords
stink or or two of your admins password
stink and we don't get to domain admin
or that's not even not even the target
they lose out because how many partners
actually know how to do a password audit
of a domain controller or even their
sequel server or their whatever right
very few I actually was on an engagement
one time where the customer had a share
that we found
someone in the audience found that had
wafak the pirated version in the share
with the password dump file next to it
so they did all the work for us we just
downloaded the pirated I mean we didn't
do it we we paid for off track then
imported the file right so so this the
password auditing is great because you
can say hey we're gonna do this for you
and you you will get value out of it
you'll see how bad your passwords are or
will save you the time on getting
through this at the very beginning of
the engagement Wireless social
engineering physical these are pretty
well known topics I won't go over those
types aggress testing egress testing is
pretty straightforward we give the
client a binary and say run this on all
your systems and they're like no or the
partner right but this this gives them
the report this is it just goes to my
website let me out of your dotnet and it
requests it and then prints out a little
HTML document that says these are the
port's that were allowed out and that
way they can run it and keep it and run
it wherever they want and it gives them
a way to do this test by themselves
right and see when their firewall
changes to allow any any to 1521 anyone
know what 1521 is Oracle seen it on an
engagement inbound yep all right
detecting collaboration this is a lot of
fun - I wish so I actually wanted to
participate in a conference that was
going to have this right this was this
was going to be great this is where I
blue teamers and red teamers sat down
next to each other and said I'm gonna
try something can you see it yeah you
saw it cool I'm gonna try something else
can you see it alright
and then all of those things got
published publicly but when when we
talked through this idea was with the
person I said well I can do this with a
partner too right I can do this with the
client too so we started doing it right
and it was awesome so I'll give you a
little example we sat down and with with
our blue team counterparts and we did a
we ran Mamie cats just dot exe compiled
straight from there and we're and we're
like did you see it yeah all right we're
gonna run it in PowerShell invoke
mimikatz see if you see it do you see it
yeah all right I'm gonna run it in a new
process when I load the power sell stuff
and then do invoke mimikatz did you see
it wait what
no how did you do that
I showed him we looked further into it
got a better detection right and then we
kept doing that I had a teammate awhile
ago that was a forensics crazy God right
he was he would do amazing things with
forensics and we would fight back and
forth about time stomp how hey I'm gonna
do this on time stomp can you see it no
and that was great this is so much fun
for pen testers but I think we're mostly
afraid to do it because we don't want to
give away all of our secrets but that's
how we get better right that's how we we
get pushed to do more things see - yeah
see - detection exercise so I was
already engaged in it once that so this
scenario is about just running it's like
a lot like the detection collaboration
but it's focused around command and
control so high random interpretive
session reversed CCP they saw it I ran a
reverse HTV and HTS they saw it hmm they
ran or I ran a Twitter see - I'm like
you ain't catching this this is amazing
this is all HTTP to a known site you're
not gonna
catch this they saw it ouch
that sucks and then I was looking for
other SI toos because I was sweating it
because they saw everything I had right
and I found my RC bot and I ran it they
didn't see it i RC are you guys old
enough to know what I RC is right they
didn't see i RC it was amazing so I
think Egypt's gonna be working on a
meterpreter version that goes over IR C
which is good awesome
rogue device rogue devices essentially
just a device that is put it in a
network no attacks are performing
against it it's just there to cause and
network traffic to see if they see it
with a raise of hands how many of you
think that your organization right now
could find a Raspberry Pi without any
changes to it put in your conference
room not a single hand raised one hand
raised all right
fire
tabletop now this is not fun but it's
useful so tabletop exercises is where
you look through a bunch of documents
and say that's stupid that's stupid
that's stupid this is okay this is
stupid this is stupid now I don't have
fun with them but I it took me a while
to find out how useful they are because
one I learned that technology inside and
out when I read about it
right it's awesome I didn't realize that
people would do this very often but it's
great because they give you like here's
all the firewall rules that apply to
this thing here's all the the default
usernames and passwords that we
supposedly change and they give you all
this stuff and you and you go back into
them and talk to them about it and
[Music]
tabletop exercises are really useful to
the customer as well because they can
stop something from being insecure at
least for the most part before it even
gets deployed so most of these tabletop
exercises happen when they're in the
planning phase we all want to stop
things earlier on right yeah thought so
vendor proving ground now this used to
be illegal because it was in their
contract saying you cannot pit our
product against each other now there was
a recent court case where this was
proved to be incorrect and I find this
really useful the customers they say we
just got blinky box X and we got blinky
box Y & Z can you come in and see if
they actually work all right it's really
useful and a really short engagement you
can just try unicorn and say not at all
you're welcome
so this is where I get societal time I'm
really ahead of time I think so let's
see what's on the slide oh and see who
gets the stuff the packet squirrel all
right
have Docs validated let's just go down
here have Doc's been validated vetted by
any legal counsel at all at this point
sort of so my wife is she reads law
books for fun which is insane to me like
she'll be in bed with a 4,000 page
legal document or a legal book that is
like a dictionary like this is not a
story of all of these stuff and she'll
read it back to back to front in like a
day and a half two days it's insane
so but she's not technically a lawyer in
the United States she's Japanese
so no I'll have to say no on that but
kind of is there a pre-engagement
questionnaire for the CXO training no
but that's a good idea
so I'm sorry
release the mean questions oh yes oh
yeah all the questions are will be in
the document for is this your music no
all right are these documents something
the blue tumour can present to a pen
test vendor instead of vendor Duncans
yes absolutely I would love to see blue
teamers say I want you to do this that's
really what this is for is to get a
standardization of of of things and give
a blue Timur or or CIO CSO thanks Dave
all right so I think that by just sheer
number of eleven ups I think Brian T
gets the packets crow
[Applause]
all right now open this up to any
questions that are not on here anyone
have any questions about the documents
or or stuff
there
so those are open right now some of the
documents I didn't get a chance to
change the permission on yet so they are
there I just got to open them up if you
request access to them I'm not going to
say yes don't do it but they will be
open later right so this one is open
right now you can go to it right now
that's the black and red part of the
talk that I just gave but all the
scenarios are in there
feel free to go through the slides as
what you want make comments anyone with
the link to make comments now that could
get out of hand and I might change that
if it does but please make comments and
say what you think should be changed
other than that thank you for your time
real quick story though so this morning
this morning there was a fly in my room
and it was really weird I'd fly in my
hotel room and during me looking at this
flaw and I'm getting ready to give this
talk I had to take a shower got in the
shower and I swear this happened and
I've never seen this happen before the
fly committed suicide it ran through the
the stream of the water and went
straight into the the drain and all I
could think of was Trevor his cousin or
something all right
see you guys later