- Let's talk testing.
And by the way, this comes from the ISACA,
which is a good resource.
They have the CISM Certification,
the Certified Information Security Manager Certification.
I also want you to realize that some
of these early tests, some
of the low level tests will also apply to incident response.
So, for example when you're gonna test
your incident response team or your CSIRT,
you will very likely do a read-through testing
or checklist just like you would with a disaster recovery.
So call it a plan review or getting a legal pad
and going through a checklist.
This is the lowest level test, kind of a read-through,
maybe it's part of a brainstorming session.
And again, this could apply to incident response testing
but also your disaster recovery planning as well.
Looking for gaps, looking for missing processes,
discrepancies, talking through scenarios,
the most basic fundamental test, we
also call it a checklist.
Now, this can be beneficial
when you're training new team members, new members
of the CSIRT or the Disaster Recovery Team.
Maybe if you're adding a new business function
or you're rolling out a new campus or a new site
for a small business, this may be the only test
that they ever do, a read-through test.
It's the least expensive
and it uses up the smallest amount of resources.
Now, the next level of tests that will usually build off
of this test is tabletop testing.
Think about that old World War II movie
where maybe it's a World War bunch of admirals
in a big smoky room with cigars and they're pushing ships
around with sticks on a giant table, it's
a big tabletop drill.
That's where this comes from.
So, what makes this different
than the previous checklist is there's visualization.
Now, it could be actual models on a table
but more often it's software driven,
computer aided drafting type things.
It can utilize physical blueprints,
but usually software driven topological plans,
response plans, exit routes that you would have
in a building, computer modeling.
So, more visualization,
that's what sets the tabletop testing apart from the kind
of the previous check, the read-through or the checklist.
The goal of tabletop testing is to better determine
if everybody knows their responsibilities as
in a incident response or a disaster recovery training.
So you're gonna better document errors.
You're gonna find missing information.
You're gonna see inconsistencies in the continuity plans
because things are being more visualized.
Now let me give you an example of how you can have testing
for an incident or a negative occurrence
which could quickly escalate or elevate
to kicking in the disaster recovery plan.
Let's say you have somebody
in your organization who's a leaver
for whatever reason, there's a layoff, the person has
to be let go for business purposes, doesn't matter.
And let's say they're a privileged insider
and once they are let go, they then become
a disgruntled, privileged insider.
Okay, so let's say it's a Friday afternoon
and you let this person go.
Monday morning they show up back
at the building, they have a bullhorn, they have a sign,
and they're yelling this is a bad company, blah, blah, blah,
evil people type of thing.
It's an incident, it's a negative occurrence.
You may send out somebody from the security team,
maybe a previous supervisor or a C-suite will go out
and talk to the person and deescalate the scenario.
It's an incident.
But if that same person, that disgruntled insider comes back
on Monday with an automatic weapon,
they're now an active shooter, and that's gonna kick
in the active shooter disaster recovery scenario.
So, an example of a difference between incidents
and catastrophic events.
The next level would be the walkthrough test
or the walkthrough drill.
Now, this is the first one that
actually involves people moving around.
Now, a walkthrough is kind of the first level.
It's done on a more limited basis.
So it's like the entry level type of drill
where maybe it's a department by department
or just a particular area doing a dry run.
Here's the differentiator.
A walkthrough test should have no impact at all
on productivity or delivering your value proposition.
So often you'll do this maybe
during the lunch hour or an extended lunch hour
or you'll have people come in early or stay
for an extra 30 minutes or come in on a weekend, whatever.
So remember, this first level should have
no productivity impact.
It's done on a limited basis.
It's an opportunity for you to kind of build
on the previous types of tests that you do
and involve active participants.
Maybe it's scheduled, maybe it's limited.
The next test starts to have some impact.
We're gonna start using our recovery site.
So, a simulation test is going to
at some degree use the hot site, the warm site,
the cold site, the mobile site, or the Cloud DRP solution.
This may be the most elaborate test
that an organization conducts, the simulation test.
This is desirable, but anything
beyond this may be cost prohibitive
for most commercial organizations
'cause you are gonna use your business continuity resources.
There is gonna be a loss of productivity.
However, in the simulation test, you're still operating,
you're still delivering your value proposition
to an acceptable level.
It may be 75 or 80%,
but you've still got people doing their jobs.
This test does not involve a full interruption
or everybody in the organization.
Now, if that is an issue for your organization
with a simulation test
which is gonna use the recovery site, it's gonna use
the backup and recovery process, it's gonna use
your recovery vendors.
It's gonna be using
your recovery communication, transportation systems.
If you're doing this and you're requiring teams
and you're pulling them away from their duties,
this may introduce what we called crisis actors
or third party simulation teams.
So to do a simulation if you really cannot afford taking
a large percentage of people away from their duties,
some companies will hire crisis actors
or actual companies to come in,
and they're gonna provide people
to replace your own employees.
And then of course, you'll document it in video
and then at a later time your employees can go back
and watch this simulation or this drill
without having to actually be there with the fake blood
and the bandages and laying around in a parking lot all day.
The parallel test involves full
on launching your recovery site.
So you are using to the full extent your cold site,
warm site, hot site, but you have
at least some skeleton crew performing operations.
It may be at a very limited level
but things are happening in parallel,
but you are relocating staff.
You're moving media, you're moving equipment.
You have a large percentage of your organization going
to the warm site solution or deploying your Cloud-based,
your staged AWS Cloud Endure environment.
But the thing about parallel testing is it's elaborate.
It can be very cost-prohibitive
but you are still operating
to some low acceptable level, like a skeleton crew.
Now, the parallel test may be the most comprehensive test
that an organization, a large organization ever does.
This may be a government agency,
may be a government contractor, it
may be some military environment, and they may also
if it's a commercial environment, they may use crisis actors
or other third party solutions.
Now, the one that's rare that if you have lots
of funding or you have lots of taxpayer dollars,
the full interruptions test.
And this is going to the warm site or the hot site
or the Cloud Endure solution
and all operations are shut down.
You are going through the process
of a catastrophic event has occurred
and we are no longer delivering our value proposition.
This is the most exhaustive simulation drill or exercise.
It's the most expensive.
And like I said, it's probably cost prohibitive
unless you're just a very successful commercial entity
or you have funding, government funding taxpayer dollars.
It has the potential, if you undertake this type
of exercise in a commercial organization,
this could have a major disruption.
You could lose money, you could affect the value proposition
where if you're not careful and you do this type
of test, you may lose customers, you may lose goodwill,
you may lose competitive advantage.
So commercial organizations, it's really unlikely
they're gonna do these types
of full interruption tests for those reasons.
Now, regardless if there's an actual event,
an incident or a catastrophic event, or it's one
of these types of drills you're doing, parallel, simulation,
full interruption, there's gonna be lessons learned,
the reporting after the event, or the exercise or the drill.
We often call this an AAR.
So the after-action report is gonna have
a bunch of information.
It's also gonna have ongoing countermeasures
and things you can implement to improve the process.
But it'll have a lessons learned section.
And that's the knowledge that you gain
from conducting the exercise or the after-event information.
So lessons learned is gonna populate
at the very least, your risk register, your risk ledger,
or it'll be part of lessons learned
of a larger organization that's gonna be used
in your iterative process
for business continuity planning.
It may affect your BIA metrics.
So the lessons learned, you'll have
to revisit things like the RTO, the RPO,
the mean time to repair, issues like that.
It's the formal kind of close-out results.
So what are you doing with lessons learned?
You are sharing and using the knowledge
and what is that sharing and using knowledge going
to lead to?
Wisdom, right?
Because data becomes information,
information becomes knowledge, knowledge becomes wisdom.
So this leads to wisdom.
This ultimately leads to a more mature organization
in the capability maturity model.
Possibly a tier four or tier five in the CMM.
You also want to sanction the recurrence of positive results
and avoid the recurrence of negative results.
And we don't wanna do blamestorming, okay?
That's not the goal of lessons learned.
Will, is it possible that there may be some blame attached?
Maybe there's somebody where the buck stops.
A CIO, a chief privacy officer, some C-suite member
or CT member, is it possible
that they did not do their due diligence and do care
and it results in a demotion or losing their job?
Perhaps, but that's part of corporate governance,
that's not part of the lessons learned
and the disaster recovery process.