hi I'm Dan thank you for having me back
again I'm looking forward to
embarrassing myself on stage for a
second year
um so a little bit of a primer let's see
um who here has uh heard of the Verizon
dbir report
wow like three people great okay if
you've never heard of the Verizon DVR
report check it out it's really awesome
Verizon has this like Department of
folks that basically they just Hoover up
all the data they can get on all the
breaches that they have seen or have
been submitted to them over the last
year and they put together this really
colorful Report with all sorts of rad
graphs and charts and stuff that say
here's like the top 10 ways that bad
guys are getting in here's the top 10
techniques they're using and so if
you're a defender or if you're an
attacker trying to pretend you're
another attacker like if you're being
you're like doing my thing or you're on
the red team
um you want to emulate what the real
uh real world threats are the Verizon D
buyer report is a really good resource
for that and from that uh these are the
figures from 2021 so they're a little
old but they haven't really changed
um 90 of all the breaches that they have
recorded uh involves some kind of
perimeter exposure or perimeter breach
which means if you're in the uh in the
world of just put everything in the
cloud and make everything an ec2
instance and they're all online well
that's a lot of hosts online that I can
see which means the attackers can see
which means they become targets right
um and then there's some figures uh with
regards to costs and things like that
um you know a third of all incidents or
you know million and a half dollars to
remediate and 75 of all incidents have a
follow-on legal cost of dealing with all
the courts in the Fallout and all that
fun stuff
back in the Triassic area the
prehistoric area or area at a time when
I used to ride my Triceratops to work in
my loincloth
um there were servers in a Data Center
and at the top of those data center
racks there was a firewall and then a
switch and then below that equipment
like physical stuff you could go into a
Data Center and touch and I know that
this makes me 7 billion years old and
this is like unheard of crazy nonsense
back pre-youtube uh era before time
existed and Back Then There Were
firewalls that people used
for Stuff
um and uh at some point uh the internet
started getting really popular
and uh some people decided you know what
we're going to take VMware and we're
going to sell VMS attached to the
internet and that sort of like developed
into the cloud and then so as the cloud
happened this happened they were like
okay
um well now you have these machines that
are directly plugged into the internet
you have like Windows machines directly
online because that's safe
um and we ended up with this and they
said okay well so that's bad because uh
if you recall again back in the Triassic
area and Windows XP being put on online
uh directly had a lifespan of something
on the order of 20 to 30 seconds before
it was infected and they said that's bad
let's do this let's put a cloud
dashboard in front of all of that stuff
and we're going to have a whole bunch of
people that have never touched
networking and don't understand how
packet work packets work design a UI for
people to do firewalling in the cloud
and that went really really well if you
were around for that I'm sure it
probably involved lots of drugs and
maybe banging your head against the car
or the ground or jumping off a building
or what you know normal people do when
they encounter a cloud dashboard and so
that was complex and confusing and
people thought this is stupid we don't
like it
so they turned it off because of course
you know what are you going to do I'm
not going to read the manual we I don't
read my logs why would I read a manual
right and so now we're back to square
two I guess where now we have a bunch of
machines online that have no firewall
and have no protection are directly
connected to the internet because the
cloud dashboard was too hard
and so
at this point we're getting into like a
whole CI CD pipeline stuff right because
um back in the day of 2005 or so we were
using CF engine at the time the company
I worked for and CF engine was like an
SSA SSH based solution that did push
right so if you wanted to deploy CF
engine and its configs to a bunch of
machines out on the Internet or on your
land or whatever they expected that you
had SSH open and you would use ideally
keys and you would make that SSH
connection and then it would like
basically run commands just shell
scripting essentially there's lots of
manual stuff it worked it was great
um but in this circumstance now you have
like Jenkins bamboo cicd stuff you know
file stores test servers all of your Dev
machines that for some reason never get
logged all that stuff is just online
with no protection of any kind it's just
on the internet
um
and so uh
it this stuff normally would have lived
in like an office network but because
Cloud it doesn't anymore it lives in the
cloud and it's completely public so the
stuff that you would expect is in an
infrastructure that's being protected in
some way by like a Nat Gateway or some
firewall rules or something is just
completely public
um so it gets colorful and interesting
because anyone can see it and so and
practically every scenario and this is
where it starts getting colorful and
interesting uh is that in an
organization in most cases the same
people that manage the infrastructure
inside the network also manage the
infrastructure outside the network or
that Bridges the Gap if you have like a
data center uh or a a DMZ or an
environment where you have stuff that's
supposed to be exposed to the public
because you're offering Services web
stuff mail stuff and API it doesn't
matter whatever there's stuff that's
exposed to the public
um and so uh it may not necessarily be
the same human beings it may not
necessarily be the same department but
generally speaking if you're talking
about a corporation or organization the
decision of how to handle Computing
infrastructure as it relates to touching
the internet all rolls up to like a
decision maker or two and
um if you find that the outside of the
environment is full of dumpster fires
and like
burning things and that sort of stuff
you can expect that the inside of the
network will be very similar often worse
because if it's on the public internet
or exposed to the public internet the
general consensus is oh well it could
get attacked so we might need to lift at
least the tiniest finger to try to
secure it in some way
uh
so in short essentially
um evaluating a company's perimeter is a
really good way of identifying like how
they handle themselves with regards to
security and their posture uh as far as
what they think is safe and what they
think Isn't So a really good example is
if you look at the outside perimeter of
an organization and you see like
Tomcat seven point something's admin
interface or like IIs 8.5 or you know an
F5 appliance that hasn't been patched in
10 years
chances are incredibly good that the
same people that left that stuff to
gather dust connected to the perimeter
uh have left a ton more on the inside of
the network and so you get a really good
idea of like how difficult it could be
to go after an organization
um based on what the perimeter footprint
is
and so I can't really put my finger on
why
um but for some reason at some point
people decided that Network diagrams
were dumb and stopped making them which
is
silly I guess
um because it's way easier to understand
like your environment if you could draw
a picture of it like if you can draw a
picture of your environment it's really
easy to see like oh this is where the
packets go and this is where like stuff
talks
um so uh another thing that I don't
really get with this sort of concept is
that
um Cloud vendors have made it expressly
difficult to make Network diagrams of
your infrastructure like if you've ever
raise your hand if you've ever killed a
bottle of whiskey because you were
trying to map an ec2 environment and
couldn't right like I'm not alone in
this and so if you can't draw a picture
um I mean I guess in some ways it's like
this is the Silver Lining is if you
can't draw a picture of your own network
then it's very challenging for attackers
to draw a picture of your network but if
everything's in one giant VPC on one
broadcast domain any who cares it's all
just in a pile on the floor
um but yeah like uh being able to draw a
picture generally means what you're
drawing is the freeway system that is
your environment you're drawing the
networking connections between services
and and servers and equipment and things
like that and you get a better idea of
how packets flow and where they go if
you can draw a picture so it's really
really handy
um
yeah not sure exactly why that is but
here we are and so you've probably heard
a lot of discussion from security people
about Network segmentation
uh and these discussions usually involve
some kind of breach where an attacker
gained access to a system and then they
were able to move around laterally
inside of the environment uh because
there was nothing in their way
um we have that now except it's just
public because these machines are just
on the internet with no protection no
firewall no anything they're just some
some box with a publicly routable host
online
um and so these are two things that were
in the news fairly recently
um if anybody remembers the vercata
breach from a couple of years ago
um the same researcher that that did the
vercata breach uh did the commuter air
one and the commuter Air One the same
thing they're both identical uh publicly
exposed instance of Jenkins no
credentials to speak of you aim a
browser at an IP and you get their
Jenkins install and if you're not
familiar with Jenkins basically the
easiest way to think about it is it's a
is a clumsily built trebuchet that
launches you into a ball pit full of
shells
um and so yeah you just swim around in
there and it's just root on everything
and then the ricotta hack it was
basically cameras that ended up in like
cloudflare's Lobby and SpaceX warehouses
and Amazon warehouses and like all sorts
of places that you wouldn't expect like
accidentally stumbling into but it was
like Jenkins install on the perimeter to
Layer Two adjacency to a bunch of
cameras that had an SSH maintenance back
door and the credentials were in Jenkins
and so it was just like boop boop hey I
can see like them building Rockets now
okay
um and so the this one with the um
commuter arrow is very similar except
instead of like boop boop Rockets it was
like boop boop the no-fly list and that
made the news and it was a very big deal
um and so as we we zoom out and we look
at why on Earth does this keep happening
it's because what we have been able to
assert is that like everybody wants the
newest sexiest coolest most expensive
possible thing we want post post Quantum
crypto breaking uh uh quantum
entanglement Hardware that does
polymorphic you know malware and AV
evasion and EDR and all this fun stuff
and they're wheelbarrowing money and
gold bars in to buy all this stuff but
over here on the left they're like
test.php where if you just add like
question mark CMD equals LS you get a
root prompt and it's like you guys maybe
you're not doing it right
and so like more and more to to mayor's
point if you got a really good idea at
like now Nation some nation states are
handling their attacker infrastructure
it's written incredibly haphazardly uh
it's just Ultra donkey mode garbage fire
the whole way and and like while
everybody is distracted with like Let's
Jam everything into chat GPT and now
that we're done with jamming everything
into blockchain like that's how yeah
it's a thing and this is why we drink
um but right so like uh somebody if
somebody drops like semicolon reboot
into the field of some fancy Appliance
you just bought and the thing reboots
then that's usually a bad sign
um and so like the top one of the one
it's always in the last few years one of
the top three ways that attackers are
getting in is because of like perimeter
breaches
um and those numbers are just going up
it's getting really silly
um and most breaches can be prevented
these days because in large part again
to Tamara's point it's not necessarily
exploits that are being exploited it's
just bad config it's like people that
don't know what they're doing putting
stuff on the internet and like crossing
their fingers and squeezing their butt
cheeks together and just hoping really
hard that they don't get hacked and it
doesn't it doesn't really work
um and so uh if I think really hard
maybe the last time that we use an X an
actual exploit on an engagement was like
2018
like maybe so it's been like five years
since I've had to think about actual
exploits because it's just like test
that PHP LOL I mean whatever but it's
like that everywhere it's kind of wild
and so like if you're in the cloud it
doesn't necessarily mean that like the
cloud is going to do your security for
you like a Linux box is a Linux box it
doesn't matter where it lives like if
you don't know how to work the Linux box
and you don't understand networking then
like you're going to have just as
difficult of a Time whether it's a
physical piece of equipment or whether
it's a VM in the cloud
um and so it gets again it gets colorful
fast because because
the trends are changing and how people
deploy stuff to the internet uh it's
making it a lot easier for these
mistakes to happen
and so again a box as a box doesn't
matter if it's cloud or not the
architecture should reflect that
um but so the intention here is
essentially if we sort of zoom out as
from The Blue Team side of things you
want to expose as little as humanly
possible
um because if we all agree knowledge is
power then you're giving the attackers
knowledge by telling them everything
about your infrastructure by putting all
of your infrastructure online and
letting anybody just see it
um and so like for example one really
hilarious piece of reconnaissance you
can do is to just look at SSL
certificates if you look at SSL
certificates depending on who deployed
them and where they go and what they're
doing some of them will leak subject
alternative names and if you look at
those subject alternative names you'll
get like internal naming conventions and
internal domains that aren't supposed to
be public but because the company
deployed a CA
they put one cert everywhere and that
one cert is supposed to be for the
internal test QA whatever Dev version of
the same thing as the external version
they're like meh let's just use the same
cert so they put all the domain names in
One cert and they put the cert
everywhere and guess what now you're
insert.sh and now you're in census and
now you're on Showdown and now you're
everywhere I look when I find this stuff
um and so the more I can see as an
attacker the more the real bad guys can
see and and the more they learn about
the organization before they break in
and it makes it really really easy to
conduct like fishing uh campaigns and to
stand up
um like evil twin look-alike
infrastructure and assets to ensnare
employees or to to fish Executives
um because if you know what to if you
know what the employees are seeing then
you can reproduce it and essentially
it's it's like a shell game
um so uh the big sort of takeaway uh is
what we call Asset value asymmetry which
this was kind of an enlightening thing
um for me when I learned about it as
described by Alex gampen gantman on on I
think this was uh Mastodon actually
because nobody should be on Twitter
anymore
um a paper written by Bruce schneier and
a bunch of other people in 1999 uh
covered this example of asset value
asymmetry which I was like this is
genius I love it uh essentially it's
think of like a grocery store chain
Right grocery stores tend to have
shopping carts and the shopping carts
are valued at somewhere on the order of
a hundred dollars per shopping cart well
uh shopping carts will get stolen by
people by homeless people by kids doing
whatever like they just disappear right
and every time a shopping cart
disappears it costs the the grocery
store chain call it a hundred dollars
well if they install those machines that
you have to put like a coin in like a
quarter or a pound coin or what have you
or a token you buy or get from the you
know a machine inside the store
uh you can't just walk away with the
carts anymore
um they stop disappearing they stop
walking away and so what that basically
teaches us is that the attacker doesn't
think that a shopping cart is worth a
coin but the Defenders think that the
shopping cart is worth a hundred dollars
and so you have this asset value
asymmetry where like the Defenders
believe that there's value to this thing
and the attackers don't and there's a
way that you can turn that on its head
and exploit that
um
which gets very hilarious very fast uh
to impose cost on attackers is actually
a lot easier and a lot cheaper than you
think
um and if tomer's uh slides were a
really good example of how the
attacker's infrastructure tends to work
uh it becomes a almost Benny hill-esque
in terms of its comedic value for how to
mess with attackers if they're coming
after your equipment this is a lot of uh
a fun discussion with regards to people
that end up jumping back and forth
between blue team to red team because
every time you make that jump you take
little nuggets of Intel and knowledge
back and forth you figure out how the
blue team does stuff that makes your
head team better and you figure out how
the red team does stuff it makes you
blue team better and you just do that
forever until like it's all the same
thing and it's all just this purple goo
because every we're all working on the
same infrastructure and Equipment we're
just looking at it from two different
angles but at the end of the day it's
like the same the same thing right
um and so like uh making life difficult
for attackers
um
is is probably something that I imagine
will get a lot more attention in the
near future as it's considerably easier
than I I thought and so here's a really
hilarious
um
well not really hilarious but this is a
a point of contention for a lot of
people who live in the devops world
where you have like a control tool in
this case you have something like
ansible or CF engine or uh what's
another one puppet Chef these sorts of
things that use like SSH as a basis to
like you have a whole Fleet of machines
out on the internet and you want to like
push config to them you use a tool like
this well in you know easy mode is you
just open SSH on all of these machines
and then you just do your push and the
the the server end pushes config via SSH
to the receiving end well the receiving
end could it could be passwords it could
be Keys
um but
architecturally speaking there's like
one sort of source of Truth and if you
control that one source of truth then
you control the rest of the fleet and a
lot of cases that's one fishing exercise
away like if you can get the the admin
who runs that component of the business
to fall for a phishing campaign not only
do you get their machine that is the
admin of their entire ec2 or their
Amazon account you get all of their
credentialing and you get key logging
whatever I mean you control the machine
it's the equivalent of pushing them out
of their chair and sitting down at their
workstation after they've logged logged
in you can kind of do whatever you want
because now you their machines are now
your machines and it gets even more
colorful if they're using something like
keys because then you just steal the
keys and who is actually putting
firewall rules on all these machines
that are connected directly to the
internet not not as many people as I
would like to see and in multiple
instances when I have done these sorts
of red teams I gain control of that key
and if I have control of that key I
control the entire fleet and so I can RM
the entire fleet which that would be bad
but I could like
install miners on that whole Fleet I can
turn it into you know a a Spam Cannon I
can use it to do impersonation and use
your imagination you just got a new 3000
node deep Linux Fleet of machines
because you have one single SSH key like
user imagination and so from an
architecture perspective if you use a
pool architecture instead of a push
architecture all these host machines are
reaching out to the server and pulling
down config so uh you can say
um
you could use like a for example one of
those what do they call it an overlay
Network like nebula or
um zero tier or or tail scale or
something to that effect where you have
like no actual open ports on the
internet and you're and you're running
that operation over that channel and if
anybody finds one of these hosts and
scans them there are no open ports and
so from an attacker's perspective
there's not a whole lot you can do and
the host isn't actually telling you
anything you can't tell like unless
there's a DNS name attached to it it's
just some IP like there's nothing to go
on and so you can't tell as an attacker
do I spend my time on this resource or
not
um because you don't you don't know
um but yeah it's uh I remember there was
one really hilarious story um in 2005 I
was working for anonymizer and we were
using CF engine at the time and it was
using a push architecture and we were
deploying code to web servers and one of
the web servers kept getting compromised
and
CF engine runs out of cron every minute
and one of the things you can make CF
engine do is check for file system
changes and you can say if any files
change that are not specifically
mandated in the CF engine config revert
them to their previous model or mode
um and so it was really hilarious
watching people break in somehow
drop files into like the web root like a
web shell or c99 I don't even think c99
might even be too new for that I can't
remember what it was but it was some
janky PHP webshell uh it would get
dropped and they would start doing their
Ops and then the whole directory would
get deleted and then they would do it
again and after six or seven tries they
just kind of gave up and it was just fun
watching like we didn't even fix the bug
that was letting them in because it was
funny watching them get frustrated walk
away and it's like hey that's hmm we can
we can mess with these people and it's
not expensive like you don't need to
drop 500 000 or a million dollars on
Fancy Pants new crazy equipment like
literally this is like a couple of
command lines in bash and it just turns
into this hilarious like these Antics
right and another piece of ancient
Forerunner technology if anybody's ever
heard of a tool called fail to band
um this thing is probably close to 20
years old maybe 25.
um it is really really simple you
install it out of something like apt uh
and out of the box it looks at your SSH
logs and it does uh basically it does
thresholding where it looks at like
auth.log and says oh some IP address
failed log in five times in a minute or
five times in two minutes or what have
you
um it it adds them using iptables to
basically a null route that says that
host can't talk to the IP anymore and so
they just it just goes away they can no
longer contact it
um and however it's fully extensible
since it's literally reading text files
and then doing whatever generic action
you want and so you can make it do all
sorts of interesting things so for
example
um
you can use it as sort of a universal
adapter like you can have a look at DNS
logs if you're actually running your own
DNS servers you can have a look at look
at web logs like a really really
hilarious thing you can do is to say
um if if you're getting brute forced
right if you're actually looking at your
like say Apache or nginx logs and you're
getting like maybe the red team is after
you and they're doing a directory
enumeration on one of your servers and
you see them doing that you know what's
the directory enumeration look like from
The Blue Team side pages and pages and
pages of 404s so you say Okay fail to
band hey look at this log and if you see
more than I don't know 10 404s from the
same IP in like five seconds add that
Source IP address to this text file over
here and then you have another config
file that says anybody in this config
file gets a different version of the
website that's full of like trap doors
and zip bombs and icar strings and Benny
Hill nonsense and like Rick rolls like
whatever and or like um a Sim link to
itself if you've ever done that in Linux
and actually lets you do it you can
basically have a directory that's never
ending deep and if you've ever run a
tool like a web
um a web app pen testing tool and you
aim it at a web server and the customer
has like a Sim link to itself you end up
in this like
like black hole that's a million
directories deep but they're all logical
but the tool it breaks the tool because
none of the tools know what to do when
you're like I'm 7 000 directories deep
and I haven't found any files yet but
every time I go into a new director I
find another directory and that's the
only thing so I go another directory and
it's like it Tar Pits the tool and so
now like if you're familiar with like
Mark Rober and his Antics and other
folks who are like our whole bag is
screwing with scammers that try and scam
you out of money on the phone if you
hang out with them on the phone and take
their time then they can't attack
anybody else while you have them and so
like this is that but digitally like
this is that but for attackers out on
the internet and so like oh and then
things are getting colorful and
interesting now and so uh you can go
wider right you can say okay well I want
all of my equipment out on the internet
to have a bunch of these different tools
installed that do these things that
collate together logs for example like
Ancient Ancient Forerunner technology
like if you don't want to give Splunk
ten thousand dollars and you have more
than a gig a day of logs you can use
syslog like we used to back when we used
to ride our dinosaurs to work
um and you can just have all of your
servers collect this log and dump them
into one place and have
uh fail to ban look at that one place
and just do these these thresholding
rules and so you can say
uh if you have like 10 000 servers out
on the Internet or 5000 or what have you
um and they're all getting pummeled
because maybe they'll have a web web
server installed or they're all
listening on SSH you can collate
together all those logs and then you
de-duplicate your attacker IPS and now
you have your own CTI
resource you have your own thread Intel
right like cool so I have these 10 IP
addresses that are beating up on us
maybe I tell my colleagues and you know
at another company hey there's these are
the known bad guys and then like now
you're kind of sort of almost one of the
you know antivirus vendors that sells
that information so now you don't
necessarily have to buy it because you
have your own sensors and again free
right syslog it's been used for decades
um so Apache has a really neat trick
um that they started doing quite a long
time ago called rewrite rules rewrite
rules are like regular Expressions but
they're for web web requests you can
make decisions based on web requests or
you can make decisions on web requests
based on a whole variety of different
things from like headers to file names
to Source IP address like whatever you
want and you can basically configure
um this is like the The Mechanics for
the thing I described before where you
have like a text file and everybody in
this text file gets like a different
version of the website you do that with
rewrite rules yeah the the rewrite rule
will say if the source IP address is X
or whatever in this list then they get
this directory and not the original
directory
um
it's really interesting
um and so it works in Reverse right so
like if you're on the red team
um and you don't want the blue team to
find your equipment you find out what
their egress IP address is which is
usually generally pretty easy to find
out and uh you add that into your
infrastructure using rewrite rules to
say hey if if this IP address makes
requests they get this website over here
that is all about
um methodically reinstalling eggs back
into chickens or mustache wax or
whatever the popular thing people are
screaming about now is uh and not the
actual
C2 infrastructure of all your equipment
that you're using for your red team
engagement
and so like it works both ways right
because again it's all kind of the same
purple mush and we all live in the same
purple mush and so like you can use
these techniques whatever team you're on
right if you're on the blue team you can
mess with the red team and vice versa
and if you're if you're on the blue team
you can mess with the attackers
um so like if you think rewrite rules
are cool this is actually pretty neat so
AJ proxy is another web technology you
can stick in front of things
um but one of the cool things that ha
proxy does that I I don't think you can
do this with Apache or nginx natively
but you can basically say look at the
first few bytes of the packet that comes
in and make a decision on where to Route
the traffic based on like the packet
bytes and so for example you can say all
inbound traffic on Port 443 if it has
the first bytes that look like SSH use
this other back end which is an actual
SSH server and so you can at the front
designate
um
Technologies behind the scenes so like
if you SSH into Port 443 you get an SSH
connection but if you hit it with a
browser you get Apache under the hood
and then Apache under the hood you can
do you can use
um
rewrite rules to say if the requesting
URI is like slash msf it gets rerouted
to this other Port where Metasploit is
running and then you can you do that ad
nauseum to a variety of other sorts of
tools right and so when we do
engagements we're running like four or
five c2s all at the same time and so you
can say any traffic come that comes in
to the front end is wrapped in SSL so
people can't see what the actual URI is
but when ha proxy sees oh this is a web
request that goes to Apache Apache then
unrolls the SSL and goes oh here's the
URI oh this is like Empire goes to
Empire or msf goes to msf and so you can
hide
you know whatever size infrastructure
you want behind a single IP in a single
port
um and so it becomes very difficult to
enumerate your infrastructure in that
way and this works the same thing you
can do red team blue team the
applications are are uh pretty broad so
like what's that phrase if if all you
have is a hammer everything looks like a
nail well that gets turned on its head
right if if you are the nail then
everything looks like a hammer
so
um you can automate yourself out of an
architecture job which I encourage you
to do because it is quite a lot of fun
um
uh you can deploy entire systems this
way
um you can make alerts and changes
happen based on file system changes like
with the CF engine story I told uh you
can reroute application layer traffic
and all that literally this is like just
the sort of foundation we're talking
about like Apache nginx and AJ proxy
um and and it gets Wilder from there
like I really like the term bear
trapping servers which I'll get into in
a bit but that's really fun um and this
whole infrastructure I call it a fire
law right because it's not really a
firewall it's more like a fire they
tried it again
um right and so
um you can set honey tokens
um and this is I cannot believe this
works I tried it just before I put it in
the slide just to make sure I wasn't
like completely talking trash but you
can go into like Etsy shells and you can
select or you can you can make curl dash
s parrot.lall a shell so if somebody SSH
is into a machine they don't get a
prompt they get the ASCII version of the
dancing parrot forever until they
control C which is really fun if if a
bad guy lands on that server
um because again tomorrow's slides
showed us that these people don't really
take a lot of sort of robust coding
practices into account and so if they're
making a connection to your SSH server
and recording everything they get
they're going to fill the disc with a
dancing parrot like like a lot and it's
going to cause a problem for whoever's
hacked server that is because what
happens on a Linux box when the disk
Runs Out of Space
you're taking attackers infrastructure
out from under them which is incredibly
costly for them because then they have
to go hack another box
um and it cost you nothing and all you
got to do is sit and watch it's great uh
and so
yeah uh
oh yeah I think article oh here this is
a good one right so we go back to that
example of like you have a whole bunch
of workers and you're Gathering logs
um you could do this with any logs
because it's all text files at the end
of the day right so uh if you're pushing
everything into syslog and all your
syslogs are getting funneled into one
collection host you can then run
interesting stuff on that collection
host and if you don't want to Shell out
a whole bunch of money for expensive
tools or if you're doing an incident
response and you need something like
right now and you can't wait for the
sales guy to give you a license
um here's a one-liner that you can use
like you can do this on your Mac if you
have a Mac or you can do this on Windows
too there's a command you can like on
Mac you can just say say something and
it will like speak to you and I think
you can do this on windows boxes too but
it's not quite the same command you can
basically say here's a tail Dash F
command that will read the log and if I
find this pattern that I'm looking for
it will make a noise or say a thing and
so you can like do that off to the side
and keep doing something else and then
you'll have like an audible alert go off
which is like an advanced feature you'd
have to probably pay people for but
here's a one-liner you can do it in bash
go nuts right and so like there's a lot
of this stuff that happens
that we've been doing for 20 years that
is basically it lives under the hood of
all the expensive appliances you can buy
so if you were like rip the cover off of
any expensive like multi hundred
thousand dollar appliance that you'll
find in a rack somewhere it's a probably
an unpatched ancient Linux install
probably running kernel 2.4 before
system D which actually makes it better
um uh running grep said awk Apache
failed to ban you name it all the stuff
is what's living under the hood and if
you know how this works then you don't
need to Shell out a quarter million
dollars for a piece of Kit to do it for
you you just do the one liner yourself
it's pretty great
um so yeah like if you're yeah if you're
looking to improve your swordsmanship
then yeah uh you can practice on that
gigantic log file that's created if
you're collecting all uh all your
syslogs in one place by doing all your
grep setauk Pearl python stuff against
that live giant log because it's a big
meaty thing you can go after
so
um
yeah this stuff this kind of works in
Reverse too because if you know how this
stuff works you get some idea of like
the shenanigans you can pull if you're
on the red team you can hide from The
Blue Team by like mounting a zip file or
an ISO file over a directory in Linux so
that when the AV or EDR runs it looks
inside the immutable read-only file
system and now underneath it at all
malware you put you know right and so
you can like unmount it run your malware
get that into memory and then remount it
and then when they're you know stuff
runs out of cron every 15 minutes or
what have you it looks at the mounted
ISO or mounted zip file and not at the
directory that you've mounted on top of
and so it's like throwing a blanket over
your mess but it totally works right
another really hilarious one is to you
can if you use that dash dash operator
with maker you can make a directory
that's got a space in it and that drives
people crazy namely me uh when I found
it so I thought oh this would be a fun
thing to use during engagements um but
yeah you can make a directory that looks
like a DOT and like spaces don't render
right and so if you do an LS or whatever
they just find it's a DOT and
yeah it's uh little things like that uh
or you know you can fidget with the with
the file dates to make sure the file
looks like it was changed a year ago and
not now
um and so there's a lot of these like if
you understand the like Linux
fundamentals you get a little bit like
elbow deep into it you can go very very
far
but yeah
um
yes another really hilarious one is
making the shell on an inbound SSH
connection something like tmux or biobu
and then recording all of that with like
ASCII Cinema or ask ASCII Cinema is that
what it's how it's pronounced
um there's a recording tool that you can
install that basically says record this
console uh live uh and then the
recording can be played in a web browser
and so you can automate the idea of I
have a Honeypot out on the Internet it's
listening on SSH when the attackers ssh
in we're using um biobu and team or
bilbu which is like a tmux like it's
tmux on steroids
um to record like we're using Bilbo as
the shell and so when they get the shell
everything gets recorded and then we can
play it back with ASCII Cinema and it
becomes like a television show like a
clip show you can watch attackers get
frustrated and leave when they realize
that like it's a Solaris box and not a
Linux box or or like you change the
Apache um uh header to say that it's IIs
6 and really it's nginx or Apache like
oh a lot of security
um it it hinges on the honor System
right you look at the the banner of a
web server and it says it's Apache and
you just believe it's Apache and in most
cases it it's Apache but when it's not
things get really fun
um because different browsers react
differently to different user agents and
different server strings
um and if attackers are looking for a
specific thing and they don't find that
specific thing because you're lying to
them uh they go away or they know that
like this these are people are just
going to eat up our time and if it's one
of those sort of spray and prey
operations then they won't they won't
waste their time on you uh it gets
pretty colorful fast uh oh yeah CH
rootkit and RK Hunter are really great
if if you're just getting into this sort
of stuff and you want to see just at a
high level like I want to see file
system changes on my machines for the
last 24 hours like I want or you can run
out on a cron every minute and have an
email you for example and say like you
you will know like what was God in 2022
the average dwell time of an attacker on
an in an environment before they were
discovered was something like six to
nine months like bad guys are are like
reading your documentation and like
reading all the you know office gossip
out of your email for six to nine months
before they get caught and usually when
they get caught it's because it's they
cause some sort of production outage or
they trip to sensor or they did
something that like oh no they just
pushed the whole water cooler over and I
hope nobody would notice
um but uh if you're using stuff like
this you'll notice that they're there
within a minute or two
um and that's going to be very
disruptive
um and in most cases they will probably
get frustrated and go away unless for
some reason they really really don't
like you and they target you which
there's a whole different set of rules
for targeted versus opportunistic stuff
so
um and closing all the stuff can stack
like Devo hats
um like for example that that Port 443
is the only thing that's open and you
have like five different c2s and SSH
running behind it is a really cool thing
that you can employ
um like for example if you have a failed
to ban rule fire you can say I want a
full enumeration of the source IP
address that's causing the shenanigans
like somebody Port scans you great you
do who is on them you do a port scan on
them you look them up and show Dan and
census and and binary Edge and you get
all the Intel you can about their IP
um and then maybe you automate notifying
their ISP oh hey there's some IP address
that's causing trouble or if it's from a
certain country or from a certain region
and you know that there's like
problematic uh traffic coming from that
region or that Network segment uh you
can then divert it using you know custom
Apache rewrite rules
right
um yeah combining fail to band logs from
5000 machines to make your own personal
CTI thread like people pay tens of
thousands or hundreds of thousands of
dollars a year for these like threat
Intel feeds that are literally just like
lists of IP addresses
um and you can kind of do that yourself
for free with just syslog and like fail
to ban and it's it's kind of wild that
like people are paying through the nose
for the sort of stuff
um and yeah you can like Auto scale with
dialysis are really another uh hilarious
one that I found was like you can Auto
scale Docker based on
um inbound connections like you can say
uh if certain environmental rules are
present and you have um traffic coming
in through Apache landing on like
back-end hosts that are uh Docker you
can Auto scale those based on what
failed to ban an Apache see which is
pretty great
and then so
um essentially you can change your like
some advice I have basically if you're
on the blue team if you're on you know
in SRE land or devops land change your
server strings to fake like
because why not like who cares if the
browser
um if the if you look at the headers for
a web server and the server line says
haha I can't believe you looked like the
web server isn't going to Care the web
server or not I'm sorry your browser
isn't going to care at all uh what your
server string is um the only people that
seem to care are security vendors and
attackers right if your web server says
that you're running you know sun glass
fish and not Apache or nginx or IIs like
it will change attackers Behavior
because again all this seems to work on
the honor System if you tell them you're
running Solaris they're just going to
believe you so you should exploit that
um if you put something online it's
going to get scanned like period no
matter what if if you put something
online it will get hit with stuff
um I encourage you to mess with those
people
um only expose the absolute bare minimum
of anything that you possibly can um
like take some of the stuff to Heart put
stuff behind firewalls put stuff behind
vpns uh put stuff behind ha proxy you
name it just like the the as little as
you can expose do that
um yeah put everything behind a VPN
um if you think that there's a reason
that it shouldn't be behind a VPN I
don't care put it behind a VPN anyway
um let's see uh the more essentially the
more there is that I can see the more I
can learn about you and the more that I
can learn about you the easier it is for
me to put together phishing campaigns
enumerate your environment figure out
what technologies you use figure out
what you don't patch what to go after
what good pretexts are all that fun
stuff
uh and if you if you tell me everything
uh via your perimeter then you're going
to end up being like the next Colonial
pipeline ricotta commuter air OCTA
T-Mobile Etc et cetera Etc
and
that's me
um
thank you for letting me come up here
and embarrass myself uh that's me on
Mastodon if you're on Mastodon I'm also
the same handle on wire and on Blue Sky
um if you're on Twitter stop being on
Twitter it's getting weird
um I'll be in the crowd I know where we
the schedule got pushed a little bit so
um do I have time for Q a or should I
jump off are we done we can do a quick q
a do we have any questions
anyone or you want to find uh just grab
Dan he will be here anyone no you want
fake yeah this is the Swedish crowd
after all ah so there is actually fika
uh in the sponsor area so we will um
yeah a Roundup rewards for a very uh
thank you very much I appreciate it
thank you very much
[Applause]