hello and welcome to my talk like beast
to honeypot
a journey through honeypot deployments
so
i have prepared a little bit of
content for you basically um we'll start
with
some technicalities and then do some fun
stuff
in the end all rounded up by by a
conclusion so
maybe we should start by giving you some
context so about me
what i do for work is i work for vmray
as a labs infrastructure tooling and
automation developer
so basically i'm providing most of the
tools and automation for the labs team
so that they can work efficiently
what i do for fun is what you are
about to see today as well as ctfs and
photography so
basically this is what i do for fun so
please don't lay yell at my employer
yell at me instead
so maybe let's preface this a little bit
more
because it's a tale of things i've tried
and am trying so it's an ongoing process
that never really is is done
so i am no expert in any kind of way i'm
just
it's just a tale of things i've tried
and am trying
so if you see something that you
think i can do better please go ahead
and tell me
i would be very interested to learn from
you as well
so as this is a public talk
and honeypots are kind of a
one-shot customized thing i'm
trying to to show you what you can do
but
not how i did it in detail just to
preserve
my tweaks and customizations and try to
at least
have my honeypots undetectable or hard
to detect as long as possible
so this is also one of the main reasons
you'd
find comparatively less talks about this
topic because
people want to preserve their knowledge
and not
burn their customizations by telling the
world how they can detect the honeypots
again if you're interested in details
find me afterwards
find me on twitter or just message me or
mail me if you want to
so maybe we should start
with the intro of what a honeypot is so
a honeypot basically is a system
intended to mimic
likely targets of a cyber attack
what does that mean um so
honeypots basically
look like vulnerable systems and they
can
either emulate a server so something
like
ssh mongodb an iot device or even a mail
server
or they can emulate a client like
browser for finding drive-by downloads
or malicious javascripts
the most the majority of honeypots i
know of
are server honeypots so this is what you
what you tend to find outside
you can further group honeypots into
three distinctive groups which are low
medium and high interaction honeypots
low interaction honeypots are quite easy
to define
so these are basic scripts that present
a protocol conform login
and try to grab the credentials so if we
take ssh as an example
this could even be a simple script that
does nothing else
than presenting an ssh login and then
logging the credentials so the username
and the password that were entered
medium interaction honey pots are a bit
squishy to define
these are still scripts but they emulate
a full system without being one as an
example there is kaori
which is a medium interaction ssh
honeypot so
kaori emulates common commands and
even part of a file
system without running a full linux
kernel
somewhere this is
what high interaction honeypots do so
for example
docpod is running a full linux image
as uh as a docker image that is then
open to the world as a man in the middle
middle proxied connection that you can
connect to so every
command in every interaction with a real
linux system
is logged and the system is discarded
afterwards as soon as you close the
connection
so if we think of the three
main groups as kind of an inverse
pyramid
you can you can organize them like this
and what's interesting is the detail in
the inside
increase if you if you go from low to
high interaction honeypots
so you start with just grabbing
passwords and usernames and you end up
with
seeing exactly every command and
interaction with the system
but detail and insight aren't the only
things that grow
the error potential and the attack
surface are
also things that increase and
with that i mean the more complex the
system is the more chances for mistakes
there are especially in regards of
deploying and configuring the honeypot
right
and last but not least uh the hardware
demand and the complexity increase as
well as
um high interaction honeypots tend to be
quite computationally intensive
tasks as well as being quite the complex
systems
so up until this point you might ask
why should i do that then so
basically there are three main scenarios
that might be of interest so
you can use uh honeypots to find
attackers in your network so
basically use them as kind of a trap
as in you deploy them into your internal
network you
don't tell anyone that you did
and then you monitor for incoming
connections or pod scans
basically you can tell that as no one
knows
this is a system that exists no one
should be
connecting to it anyway so you can think
of
every or you can you can think of every
connection as being
potentially malicious and something you
should investigate as soon as possible
the second big use case of my of
honeypots is
getting general grasp of attacks as in
what is happening outside so looking at
the new
malware campaigns or the newest
credential stuffing attacks or even
the new exploits that are out there
especially the last over the last weeks
all major vendors got hit with
more or less bad
bad software flaws that they had to
patch and you can build honeypots for
this and
have a look at what people try to do
with them
and last but not least you can check for
open
directories and more payloads because
usually when malware is distributed it
has to be distributed from somewhere and
this can be interesting as in
finding new payloads or finding
variations of existing ones as well as
even
finding campaigns ideally so what this
talk focuses
on is options two and three so getting
general graphs
of attacks and finding open deers
deploying honeypots in your own network
has some drawbacks and considerations
that should be seen
as yeah different than at least getting
open deers and getting a general grasp
of attacks
so maybe it's time to tell you
how i got into this because
many years ago i set up an ftp server
with uh
quite weak credentials by accident and
it ended up hosting things that i
definitely did not put there so
i then started noticing that basically
every
public-facing service was getting
brute-forced more or less
and years later i i revisited this topic
by uh using a raspberry pi
or starting with a raspberry pi
installing kaori
the medium interaction ssh unipod on it
and just
having it running in my home network and
it ended up being a multi-install
multi-install system that is spanning
the globe on different vpss
and i then started seeing that crawling
logs of every
single honeypot by hand is a tedious
task so i wanted to do centralized
visualization
and when i had this i ended up thinking
well i do have visualization but i still
need to copy and paste
every single payload around and then as
still ssh to the honeypots to find
something
so i started automated collection and
storage
in the platform which is misp and
finally i thought about i can that i can
even enrich
uh based on miss and with miss
so unfortunately because of the time
frame
uh we will just focus
on the visualization part and the
deployment and collection of data
but again if you're interested in more
just
hit me up i will gladly tell you more
about this
so how do you collect data
basically you can start with yeah spare
hardware that you've got lying around
and even use vpss so the pull
the the possibilities i endless
so i started with a single used
raspberry pi 1b
so there is basically no excuse in not
doing this
what i would recommend you to do is uh
scenario one if you do this for an
internal honeypot
deployment as in maybe a preemptive
system
that is that shouldn't be known by
anyone
then please do not expose this honeypot
to the internet because you need to put
it
near your production servers and
having something vulnerable in the same
network as your production servers
isn't something that you should do at
least
not with public access so if you want to
have public access on your honeypot
please do this
by putting the honeypot into a dmz or
at least having it separated in a
different network than your
production environment so you should
treat your honeypot as a
throwaway system so please do not use
production data
so no no reuse of usernames
passwords or ssh keys and please try to
avoid root
i will tell you a little story about
that later on
so what's important is
you're exposing code to the world that
looks vulnerable
but it most likely is vulnerable but in
other ways
than you'd think so the attacker has the
advantage here
so they have infinite time and ways to
try
things you on the other hand need to
mess up once
for them being able to possibly own the
system and everything in the same
network
which brings me to the next point of
detection
and evasion so
what's happening outside is that people
constantly try to
detect and debate honeypots because
they don't want their payloads analyzed
by researchers as you would think
so one of the easiest things that you
can do
is evaluating
host names service versions uh
and trying default usernames and
passwords so
that's something if you get a honeypot
and you
set it up please try to change as many
default settings as you can because
for example what you can see here is
that
kaori default used to have the host name
svr04 which
led to automated attacks
trying to or passing the hostname and
then disconnecting if they found that
the system
had this hostname another common thing
that um that authors try
to evade honeypots is
using a shell script or a multi-stage
payload
um to try and um
and and make use of one of the main
principles of
low and medium interaction honeypots
especially
these two tend to uh disown
um these own payloads as soon as or
and files as soon as they are written to
disk so that they cannot be executed
which means if you upload a bash script
to a honeypot and try to execute this
afterwards it won't be there it won't
work
and therefore you successfully invaded
the honeypot what you can do against
this
is basically only uh the use of high
interaction honeypots
that emulate a full system
but the whole thing is a cat and mouse
game
and if you are interested in a little
bit
more of detail about this
i can happily recommend you the torque
breaking honeypots for fun and profit
which which is a very interesting talk
that goes a little bit more into detail
on
which honeypots can be detected in which
way so they are
they they they present some common and
interesting ways how they
uh were able to find honey pots
and as i mentioned before
the most important thing is
try to customize your honey pot so
most attacks out there are automated
which means there is no human that
looks at every single thing or every
single connection
their scripts do so if you want to
evade this
if you want to try and keep them
keep them on your pot try to change as
much defaults as you can and
also make the make your servers look
real so
that's another thing if you think about
a mongodb cluster
it is unlikely that a big mongodb
cluster deployment of
five or six five or six machines
that every single machine of them has
only 256
mbs of ram so try to make it look real
try to have reasonable sizing
you don't have to have it in hardware
but make the honeypot show
reasonable sizing which is something
that it can do
usually and also think about cluster
state
that's something that especially bigger
deployments
are quite good at in emulating
so you've got all your honey pots
up and running now but
they look like this so all you can see
is text scrolling down your screen
which is something that is not nice to
read for humans
so what can you do what can i recommend
you to do is
use something like splunk or elastic
to find to sift through this haystack of
semi-structured log data and visualize
it
how can this look in reality just as a
quick glance
as you can see here this is my live
splunk deployment um
and as you can see this is kind of an
overview of the main
yeah this is the main dashboard that
then branches out into
details on how each honeypot performs
so if we take an iot honeypot that
we've developed ourselves you can see
that
we've got quite a bit of insight in how
this honeypot is used how it get got hit
and
what people do on it
so
basically um this
visualization is something that heavily
depends on your use case
so it is always best recommended best
practice
to define your use case and especially
define what you want to see or
visualize beforehand and then building
dashboards and visualizations for this
and there is something that you should
keep in mind when you build
dashboards which is filtering noise
what you see here is something
this is these are payloads that got
uploaded
and if you have a look at the shot
hashes you can see that every
payload has a different hash but
all of them share a common file name or
upload path
and this is where you can see that these
so these files don't hold
any valuable information they are just
generated noise
and this is something you should always
be on the lookout for
especially as it pollutes your uh your
visualization
and your dashboards but
enough from the technicalities let's
have some fun and look at the fun stuff
which is also known as everybody face
from time to time
so what this is is common pitfalls that
i
encountered in running honeypots and
common pitfalls that
users of my honeypots encountered so
what happens to me more often than i
would like to to
admit is uh i connect to my honeypots or
want to connect to my honeypots for some
quick maintenance
and i'm wondering why my sshk isn't
working
i mean i'm sure it's the right password
so
think about it what could happen
and yes so this
usually uh ends up me being
me just could really connecting to the
honeypot itself and not on the host
system
so i reset all my passwords keys scrub
all of my data out of my splunk
dashboards
and basically do a cleanup on l5
unfortunately so
all infrastructure was up and running at
one point
so i had multiple vps providers running
my honeypots
and i had all my infrastructure spread
around the world
the problem was there was quite a big
debt pooling scam going on and
mind you this is my private money as it
this is my private project
so i chose the least expensive providers
that i could find which all
basically closed down without further
notice so
all my honeypot infrastructure was gone
in the process of having my
infrastructure i decided to move splunk
to another host
so splunk holds at this point
holds all of my data and all of my
payloads and
executed commands
and i forgot to check backups which
means
yeah i tried to move it i just i decided
on
just deleting it on on the one provider
and then using my backup
to restore it on another one and yeah
basically i'm not the only one that this
happened to
so please check your backups
check that backups are done and verify
your backups
it's a heartfelt thing that you should
think of
so what i also did and what i wouldn't
recommend
is deploying honeypots to system python
so
i would recommend at least if you're
running python
python honeypots use a virtualmf because
the next apt update will definitely
update some python dependency and this
will
break all of your honeypots at least
this is what happened to me
what i also did was i updated the list
of allowed passwords
on one of my honeypots and i started
seeing lots of denied password attempts
and
yeah i ended up
pasting the wrong format for the file
which
allowed my honeypot no more logins at
all
and that's something yeah i haven't seen
because i was monitoring my honeypot so
maybe either have this done
by configuration management as an
ansible or
maybe monitor your your honeypots which
is also a thing that i would recommend
and finally there it is again
please do not run your honeypots as rude
i did this the system
got owned quicker than i would like to
admit
and yeah i ended up rebuilding the whole
system please don't make the same
mistake i did
so basically best practices please
monitor your honey pots
especially storage and services
as in are they running correctly are
they
are they storing is there enough storage
is there enough room
and regularly clean your payloads and
payloads and logs
because what happens is you can have the
best honeypot infrastructure
if the device is full it's full
so what have i found as well
um i have found something
some high or i can give some best
practices maybe for aspiring developers
which is
maybe do something after you connect so
this is something
from my smtp honeypot and as you can see
uh this is the connection count per
minute so
there is a barely noticeable spike with
with uh 250 connections from a single ip
in
in two minutes i mean maybe my honeypot
is broken
but it feels a little bit
like this so
another option is please provide actual
login data if you try to log into one of
my systems because
if you're using the port and the ip that
is
definitely something that shouldn't be
in username and password
or system commands shouldn't be username
password
or your any file title
for login shouldn't be there
and finally always check your strings
so in this case
the avid reader might have seen it as in
this is an
invalid address so there's no ip address
and that this there's something
also interesting for me because uh this
is something that happened on my adb
honeypot
more often than on any other honeypot
attackers just dumped all their
payloads in a hail mary attack and tried
to execute them which means
i get to have all your fancy honeypot uh
all your fancy payloads on my honeypot
and finally there's some just
weird things that i found over the time
so
for example these are user agents on my
honeypot
on my iot honeypot and keep in mind
these are easily forgeable
but there's something interesting as in
there are some messages from well-known
agents and well-known services
and then there's this i still don't know
how this got there
and there's always this one special
friend you have
that runs his own asn
which is also an interesting thing to
see
on your honeypot infrastructure
and finally having a look
at things that get uploaded is something
that is definitely
always worth a shot for example this is
an ssh key or public part of an ssh key
and if you find this in your
infrastructure
you should maybe start thinking about
if you're getting breached or not
because
this is an ongoing distribution of an
ssh key and this key
i started seeing it roughly in december
last year
and to this date i have i have catched
it from
more than 1 100 different source ips so
if you find this in your infrastructure
maybe think about nuking your most of
your infrastructure
so conclusion the next
things for me are high interaction
honeypots on nested virtualization
which is the thing that is quite complex
but
is getting more and more usable by day
and in general i would like to do
more tooling and integration tooling
integration and more
automation as in i'm not good at reverse
engineering so
i'm leaning on my strong suits
in automation and building workflows
that do stuff automatically without
me interfering with it
i want to give you on the other hand the
chance to
to use the things that we've built
and profit from the research we did
which is um i can recommend you
the um the honeypots that we've built
and the splunk dashboard that i have
shown you
in this presentation you can download
these and use them with
little tuning in your own infrastructure
and there is generally
a list of highly recommended resources
and projects that you should check out
should you be interested so
in conclusion i want you to deploy
honeypots
deploy them today it's low cost it
is virtually no maintenance and gives
you higher value
so what's holding you back
so you've made it thank you for your
attention
that's it from my side which means if
there are any questions from your site
please go ahead and contact me for
example
via twitter or write me a mail i'm happy
to
discuss and share my knowledge with you
thank you