- So as I mentioned earlier in the course,
there's a significant amount of information
in the GitHub repository that we have actually been sharing
throughout the course.
And this time what I would like to draw your attention to
is in the section related to open source intelligence,
which is this directory here called OSINT.
Open source intelligence is basically public information
that you can gather from things like DNS records
or using tools like WHOIS, or from public websites,
from social media websites and so on.
And these techniques are not only used by attackers
to find their victims, but also ethical hackers of course,
to find information and mimic what an attacker can do
but as well as law enforcement and private investigators,
will use these type of techniques to also
try to catch the bad guys out there.
In this section of the GitHub repository,
I have a list of tools.
This is definitely not the complete list
of all the different tools that are in the market
or in the community.
And as a matter of fact, what I'm highlighting is
open source tools.
There are also commercial tools that you can acquire
or some of these actually have a professional version,
if you will, that they either sell those tools
through a subscription or an upsell, right?
But everything that we're gonna be using
throughout this course is completely open source,
completely free and so on.
But what I want to highlight,
as I mentioned earlier in the course, is that
cybersecurity is like math.
The more that you practice, the better you become.
Tools will come and go focus on the methodology, right?
And in this case, it's actually looking
for different artifacts, looking for different information
that can allow you to then use additional tools
to then accomplish the task.
That task can be gathering information about
what web applications the company actually has
in a cloud environment, let's say in AWS, Azure and so on.
And then if that is within scope,
and scope is extremely important here, right?
So if you're permitted to do that type of assessment,
then of course, launch different attacks and so on.
Another example of using open source intelligence
is to find information about key individuals
within the organization to then send phishing campaigns
or then target them in a different way
using social engineering techniques,
of course, if that is actually allowed
and if it's within scope of the engagement
and you see that very often
in red team engagements nowadays.
So if you go down the list,
there are a few tools that I want to actually highlight
out of this list.
One is called AMass.
AMass is actually a tool that has been developed
by a colleague of mine.
His name is Jeff F. Foley.
This is a project
from the Open Worldwide Application Security Project.
They used to be called
the Web Application Security Project, right,
but if you know them by that previous name.
However, this is a very powerful tool
because it can also integrate
with many different sources of information.
So for example, you have here information about APIs
that they support, including things like SOCRadar
and Searchcode and Shodan, which is very popular
and we're gonna cover Shodan in the next few minutes
and many other ones out there.
As a matter of fact, some of these tools can integrate
with each other, the tools that I'm highlighting here.
So for example, this Sublist3rAPI is yet another tool
that we actually have in the listing here.
But the other things that I want to actually highlight is
that open source intelligence goes way beyond tools, right?
You have each sources of information
that you can just launch your web browser
and then search for different artifacts,
for example, certificate information.
And we're gonna go into different examples later
in the course on how you can gather information
about even internal hosts within the organization
using certificate transparency.
Another thing that I want to actually highlight to you is
other resources like Shodan.
Shodan is a organization, used to be an individual
a long time ago, but is now an organization
that basically is scanning the internet as we speak 24/7,
365 days a year with many nodes around the world
and then the results of those scans are put into a database.
And then you can query this database and say for example,
things like give me all the known devices
or known applications that are listening
to something like Telnet.
And you see that there's 337,000 devices according to Shodan
that are listening to a non-secure protocol, right?
And you see the distribution, right?
You see China on the top, you see Canada, Argentina,
United States and so on.
However, yes, some of these can be false positives,
some of these can be honeypots.
So in other words, a honeypot is a system
or a network, a honeynet
on where you configure devices or containers or VMs
to impersonate vulnerable application or vulnerable device
to attract some network traffic from attackers
and then analyze what the tactics
and techniques of that attacker is using.
So some of these will be like that,
some of them will be definitely two positives.
So in other words, not false positives
and what I want to highlight in here,
even though if you are seeing things that may be insecure
in the results of Shodan or the results of GreyNoise,
which I'm gonna introduce in a few seconds.
Those you do not have permission to attack.
And I'm gonna be actually sharing with you
things like the Google hacking database
on where you have different search strings
to actually find information from Google.
Even though you may see passwords and usernames and dumps
that have been part of potential breaches,
you don't have the permission unless you're actually stated
and you have permission from somebody
to start attacking or using some of that information.
But a lot of organizations actually use this
in a different way and this is a way
that is very popular nowadays called,
Attack Surface Management.
And basically what it is, is actually looking
from the outside in,
into what is the security posterior organization.
So not only this is actually being used by ethical hackers,
are used in the world of actually even threat hunting
in some cases, but also for attack surface management,
along with other vulnerability management techniques.
It is a platform and a technique
that can benefit many different use cases.
So to go over a different example, I do work for Cisco.
So I guess, I'm part of the PSR team,
so I have permission to to do this.
And this has been remediate for quite some time.
Let me give you a history about this protocol
called Smart Install.
This was actually a very unsecured protocol
that we have removed from pretty much all products
for several years now.
And this protocol allows you to manipulate networking device
and it should not be exposed to the internet.
Currently, as you see in the screen,
as of today, there are 10,000 devices
that apparently actually are listening to this.
It's getting a lot better because it was 150,000 devices
a few months ago.
And whenever we actually started to do
these type of investigations,
which is dating back many years ago, 2018,
there were over half a million devices.
And I'm not gonna go into all the details
of the actual protocol because it's kind of obsolete
but however, if you see in here,
if you click on any of these,
as a matter fact, this is a perfect example
and this is just a random example
of an organization in India, right?
And this is all public information. I'm not hacking anybody.
This is actually part of a Shodan scan,
which is all public information.
The activity that I'm doing is open source intelligence,
passive reconnaissance,
even though Shodan did active reconnaissance.
So in other words, Shodan did the scan,
it found these type of devices, put them into a database,
but the act I'm actually going
and reviewing that information in the database
is completely passive.
So in other words, the organization doesn't know
that I'm doing this.
You cannot detect it with any security devices.
And yet along, I don't have to do a scan myself.
I'm already getting information
that this entity is listening to Port 23, Port 69,
Port 123, and 4786.
Now this may be a honeypot, it may be a real device.
How do will I know?
I will have to actually then start to interact
with that device, which is then active reconnaissance.
And whenever it comes to that,
you must get permission from the organization.
So I want to highlight a few of those.
There's another organization called GreyNoise,
which is basically yet another competitor, if you will,
of Shodan.
And it's an organization that you can search also for free.
And similar to what I did, I can put Telnet in here,
it will tell you that there are 39,533 devices in China
that is listening to Telnet.
India is on the top of the list
and as a matter of fact,
you can start to compare the results
between one and the other.
One thing that I want to highlight
before I move to the next example is that
all these entities Shodan, GreyNoise
and a few others out there, they have APIs.
And how you typically do these is programmatically
with APIs in, let's say Python
or using other tools like Recon-ng, that can interact
with these devices.
And as a matter of fact,
the the tool that I mentioned earlier, AMass,
and we're gonna cover that later in the course,
it can interact with these platforms through APIs.
So you do have to register, get an API key
if you want to do that.
What I'm doing here is, of course not authenticated,
just public information
and I'm not even creating an account
in these systems or services.
Now, I am not definitely gonna go over every single tool,
every single item in this page in here
that I have in my GitHub repository.
But I invite you to actually become familiar
with several of these.
Later in other lessons,
we are actually gonna go over different tools
that I'm highlighting here
and the different methodologies, right?
So you have things from data breach query tools
like h8mail or h8mail, they call it h8mail,
is a tool that allows you to interact
with dark web repositories to see if different
email addresses have been part of a breach.
And in some cases you can even extract,
not only the email address, but a password as well, right?
So you can actually do those type of investigations as well.
Now, there are of course, many, many different avenues
that you can do IP address and DNS lookups,
so interaction with social media platforms and so on
who is BGP looking glasses, DNS, and so on.
So what I would like to draw your attention to is
to the OSINT framework.
And this is an amazing set of tools, sources,
and information about open source intelligence.
And let me make this a little bit bigger in the screen.
So as you see, you know,
you have a pretty cool visualization, made in D3 and so on.
But if you want to start going over
different public records, so for example,
voter records or patent records
or government records and so on, right,
you can start gathering a lot of information,
especially in the US.
If you live in the US, probably your county is posting
where you live because you probably own the house
and you have to pay taxes of to that house
and then of course,
all the information that is completely public.
Threat actors and scammers
actually use that information all the time.
You may actually ask, why am I getting all these calls?
Why is somebody's actually texting me
if I want to sell my property?
And so on. Well, all this is actually public, right?
Along with court records, criminal records,
birth records and so on.
So this is being used for not only scammers,
but also from law enforcement entities
to actually find information about people
that are probably wanted.
Now in this case, as an ethical hacker,
depending of course on the scope
and the engagement that you have,
you can take advantage of this information
to find key individuals and probably make
different phishing campaigns
that are very, very, very, very targeted to the entity.
And in some cases, these are scare tactics
that attackers actually use
that not only they're looking into somebody's LinkedIn post
that they were probably presenting at a conference,
or they were trying to pitch a new service
or a new product within their organization,
but also finding information to where they live
and then construct these emails
that are very, very attractive to, of course, the attacker,
but also to the victim to probably click on an attachment,
click on a link, and so on.
And now with AI and tools like ChatGPT and many others,
these attacks and attackers are taking it to the next level
into the fact that they are actually are constructing emails
that are extremely convincing
with public information like this to allow the victim
to click on an attachment or to click on a link
and then compromise their devices.