I have the great opportunity to
introduce our first Speaker today
um Robert M Lee is the CEO of dragos
corporation uh as many of you know
Sirius has got has been the lead
Technical University for simani which is
a department of energy Initiative for
cyber security for advanced
manufacturing before that we'd already
ventured quite deeply uh into the area
of cyber physical security and ICS
but along the way we've had this great
opportunity to call upon expertise
dragos and have built a relationship
with their president
so it is our great honor to actually
have him for the very first time here on
campus to give a talk so
um Mr Lee
I'm gonna like thank you welcome
come down the aisle like a wedding so
this is good yeah yeah awesome
all right you just want me to jump right
into it are you done awesome perfect
well thanks everybody appreciate your uh
time and attention this morning as
mentioned I'm Rob Lee I run a company
called dragos past life I started out in
the Air Force side of the house I spent
most of my time over the National
Security Agency built out the US
government's Mission looking at State
actors breaking into industrial sites
around the world and then spent time on
the offense doing similar to other
people
um so fun stuff uh and my talk today is
not going to have slides and PowerPoint
so you're welcome in advance my head of
Intel would always tell me that
PowerPoint and indicators are compromise
were very similar in the fact that
they're absolutely pointless but if you
don't use them you look stupid but uh
will uh accept the stupidity and just
talk anyways um but my talk today is
really around the industrial cyber
Trends so I'll touch on the threat
landscape but I also want to touch on
some of those boring things like policy
and some of the things that are
happening with governments around the
world on this topic
um so just a little bit of a catch up
here for those of you that may not be
industrial folks so we talk about
industrial control systems at ICS stuff
we usually use the term operations
technology or OT instead a big reason
for that is a lot of our industrial
control systems that we talk about tends
to be Power Water gas manufacturing
Pharmaceuticals Etc but then a lot of
the other Industries with physical
systems like Aviation and hospitals were
like hey what about us HVAC systems for
a data center as an example hey what
about us
um so we all just kind of agreed fine
we'll just call it OT but still in the
industrial sectors they'll mutter under
their breath like yeah we're doing ICS
instead just to make one of those OT
folks but either way uh broader coverage
of OT also tends to make sense when you
talk to board of directors and
Executives hey you've got your
Enterprise I.T stuff over here is your
OT stuff so it just it works
um but the idea the thing that's
important here is that it's really all
the stuff you have in it plus physics
that's what OT is okay that's the
easiest way to think about it a lot of
folks will say things like oh it and OT
convergence stop that okay so that's
y'all just knock it off okay conversions
happened about 20 years ago we have
Windows operating systems in a power
plant we've had it there for a long time
we've had CP cards on the side of an ABB
rtu and all those different acronyms
just mean special as controllers
um so we've had convergence what we're
seeing now is kind of hyper digitization
hyper connectivity some people call it
like industry 4.0 or digital
transformation depending on which big
four auditing firm you talk to and
ultimately what we're seeing is all this
stuff that we've usually had as analog
disconnected kind of Serial stuff being
highly connected and accessible not only
by adversaries but by our own folks to
do new cool things like data modeling
forecasting better manufacturing lots of
fun things that we can do with those
advances
but of course
um the point is still physics so for
those of you that also look at these
systems and think oh well they're
changing maybe one day we'll just have
I.T security stuff in that operations
environment I would caution you against
that we look at the it security controls
they were built with what they were
built with I.T security threats and
risks in mind
if I have a Windows operating system I
really care about stopping somebody from
getting on that system escalating
privileges stealing the data off that
system you know that's the stuff we care
about system and data security in it
and OT we care about systems of systems
and physics that's what we're worried
about the fact that we could lose data
sucks like we don't want to lose data
there's a lot of interesting
intellectual property in those
environments the intellectual property
for a Manufacturing Company usually
isn't the recipe or the trade secret
around the recipe usually it's the
actual Industrial Automation environment
itself how did we tune up to take low
quality inputs and make high quality
outputs with that set of Industrial
Automation that's the intellectual
property that's not stored in your email
server that's in your operations
environment so yeah intellectual
property theft happens there but what we
care about a lot is that physical side
of life and safety right just making
sure that people are taken care of
um so if you take it security controls
that are designed on one set of systems
against one set of risks with one set of
impact with one set of threats that
operate in one set of ways and we look
at an operations environment with
different risks impact threats
vulnerabilities uh and and adversaries
that operate in different ways the idea
that we would just copy and paste one
set of controls into another environment
seems pretty silly and yet what do we do
when we walk into an automation
environment oh my gosh this Legacy
equipment and it's unpatched cool let's
let's set a stage here how many ICS or
OT vulnerabilities
known at the time have ever been used in
any industrial attack
zero
zero and yeah the first thing we yell at
operations people about is unpatched
systems it's an I.T mindset okay so you
walk into these environments you kind of
gotta take the bias in the baggage and
put it off to the side for a second
actually one of the most impressive
things you can do in one of these plant
environments walk in and find the oldest
crustiest looking engineer and just say
hey what are you worried about oh boy
that turban over there that thing around
that direction like okay thank you I
appreciate you identifying your crown
jewels and now we know where to start on
the things that actually have a risk in
that environment right that's probably
going to get you a lot further than
maybe a safety standard or a cyber
security standard all right so that's a
itvers OT very very high level but
everything that you interact with comes
out of OT the chairs the computer the
mobile phone everything
industrial is the world which was very
funny to me in my own Journey when I
went to start a company and all the
Venture capitalists out there were like
why aren't you doing stuff in cloud or
like apps I'm like it's everything
what's the size of the market I don't
know the world like it's everything and
there's a lot of people who
misunderstand just how uh pervasive all
this automation is all right so that's
that piece so what's been going on and
why are we all worried about it well
years ago we dealt with I would say kind
of ICS curious threats okay I put them
into three buckets ICS curious ICS
capable and okay we're gonna
come back to that last one all right so
ICS curious was those threats that
started out going huh I don't really
know this industrial stuff
but
that's an industrial company like a
power company and I bet if I break in
there's something there for me let me
see what I can do let me see what I can
learn so as a general curiosity and
they're breaking into places like
pipelines and so forth but maybe not
really being in the industrial side of
the house yet they were trying to get
into those operations environments all
right it says ICS curious threats and
that really lasted up until about 2009
2008 2009 time frame a lot of those
threads kind of captured that bucket of
not really in automation environments
but really curious about it and trying
then we kind of got quickly into this
phase where we had a bunch of different
threats that were ICS capable and they
got into the automation environments
they started stealing the right data
sets taking data historians and
Engineering work drawings and
screenshots of human machine interfaces
not
stuff like credentials and passwords but
stuff that showed they understood that
they needed to understand the physics of
the environment if they wanted to cause
disruption or an attack okay and so they
were capable they were in the
environments and they're starting to
develop knowledge and capabilities and
this is a really important sort of
tangent
to do the types of attacks we're
concerned with most in Industrial
Automation environments does not require
exploits vulnerabilities or malware
which makes up a significant portion of
what we care about in I.T security and
that's not saying you can't use those
things it's just not required because of
the systems of systems nature a lot of
what we deal with in automation
environments is native functionality I
can open up a circuit breaker so can the
adversary if I can change the logic on
the safety system not to activate in an
unsafe condition so can the adversary
doesn't require vulnerabilities exploits
in an hour to do those things now you
can use malware to make it more
automated you can make it more scalable
and repeatable we don't need it okay
so it doesn't have to be a capability
that's developed it can just be
knowledge and that's what those ICS
capable adversaries were doing now we we
did have that s word that dropped in
Iran um that you can't get through an
ICS presentation without somebody
mentioning
um too bad we'll never know who did it
but it was a very interesting attack and
that's true but for that whole era I
would say we're mostly dealing with the
capable adversaries ones that hadn't
done destruction or disruption yet was
mostly Espionage for capability
development then everything changed in
2015 and that got us into that next era
of ICS if you will people that
are targeting civilian infrastructure so
I hate to cuss in front of a prestigious
audience that being said when we're
talking about people on the other side
of the world that are trying to kill
civilians hopefully we can all agree
there's only one word that actually
captures that correctly so when we look
at these adversaries and we look at 2015
and what changed I'm the first ever
Cyber attack to take down electric power
took place in 2015 and that happened in
Ukraine across three regions of Ukraine
and in December and it was December 24th
technically but I always remember
remember it because I was getting
married December 25th and I got the
phone call from a counterpart I just
left the government like three months
before I had been previously stationed
in Europe and had kind of all these
counterparts across Ukraine and
everywhere else I got a Ukrainian
counterpart calling me up going
power's down you know like that's cool
man I don't really care click uh
called back he's like no you don't
understand I bet it's a Cyber attack and
I'm like it might be squirrels I don't
know click uh it's like I you know when
you look like me and you serve in the
military and you get an opportunity to
get married and somebody looks like my
wife you take that opportunity right
like you just just make sure you jump on
and uh get on the ship
um so anyways I keep ignoring the phone
call uh and I'm really focused on trying
to get married the next day
and I get a phone call then the next
morning from a man that was my mentor in
peer Mike Asante and he called me and
said uh
it might be real it's like God bless
like let me wrap up this wedding
um and so I talked to my wife and she
was really nice about it and so we
finished the vows and then I left
um anyway so I got to investigate the
Ukrainian attack I'm losing a lot of
audience points I feel at this point in
the story
um anyways I got her a honeymoon in New
Zealand after it okay like calm down
with all the Judgment all right but uh
so I get to go do the investigation
Ukraine and what was interesting was it
was exactly what I just mentioned of
knowledge not capability and yet
all the world was talking about black
energy malware and the IT networks
that's what caused the power outage
that's on how physics works and you
can't just deploy something on an I.T
Network like that and cause a power
outage that's just it does not work that
way so it didn't make sense
um but what we knew and what what we had
from all the data and the insights was
we have us our use black energy malware
to get into the IT networks but then
just use the connections that existed
between it and OT got over the eot
networks learned the environment spent
six months in that environment learning
how to operate the systems and then
executed the attack and that attack
really was just operating like an
operator we're just opening and closing
circuit breakers and de-energizing the
Electric System it scared the hell out
of governments but every electric
engineer and operator out there was like
no kidding of course you can do that
what are you worried about
um now 2016 scared then but but this is
important part of the story of 2015. so
then I get the illustrious duty to go
briefs the National Security Council in
the White House and various individuals
that don't like to be named there and uh
talking to them it was very clear that
they were stunned
they were like well poor Ukraine well at
least that can't happen here I'm like
what sorry what what are you talking
about well well look at all the cyber
security work we've been doing across
critical infrastructure you know look at
all these efforts look at the nist
standards and look at Newark Sip and all
this other stuff
I was like yeah you do know though
that's on the I.T stuff right they're
like What's I.T and I was like that's
not the OT stuff they're like What's OT
and I'm like oh we're screwed
um
so I started educating them on like the
differences I was like look all these
presentations you're getting and all
these CEOs that come talk to you about
the Investments they're making they're
not lying but they're talking about the
Enterprise I.T you are talking about the
Enterprise
really different really simple terms big
difference and big impacts so the White
House started calling around to the
different companies and execs and going
uh can that actually happen here and the
power companies deserve a lot of credit
because all their CEOs were like yeah
it's what we've been warning you about
of course it can happen and we are not
actually able to make these Investments
because the regulated ways that were run
and the public utilities commissions and
all sorts of other stuff uh and so the
government started freaking out because
if you can think about it prior to 2015
the review point was this is handled and
sure there's always me more cyber
security stuff to do but at least we're
doing a lot to find out that they
weren't doing anything on the critical
part of critical infrastructure was
terrifying
so at that point we started seeing the
policy World actually get much more
aware of the terminology like OT one of
the reasons I do push back on the Cyber
physical system thing as a quick tangent
it is accurate but your policymakers
Executives board directors everything
else have learned the term OT let's just
stick with one instead of coming back
the next week and be like actually we
call ourselves differently today like
okay maybe not useful in policy settings
either anyways so we absolutely started
talking and digging into it and their
perception was yeah we're screwed my
perception was no you're not
our infrastructure providers have built
safe and reliable networks safe and
reliable systems they didn't do it with
security in mind but that safety and
reliability culture absolutely
contributes to security if you are
resilient and you're resilient in
general okay and that's very useful I
can bring back up the power from a
hurricane I can do it from a Cyber
attack now I can't do it with three or
four of them happening at a time because
we don't train for that so like one
little instance don't freak out if
somebody comes at us in a combined way
across a couple different sectors or
sites freak out right those are your
lovers
um so there's a lot of discussions a lot
of like okay how do we deal with this
and then by the time a lot of these
conversations are taking place the
second attack happened that was in
Ukraine in December of 2016. what the
adversary did which was very clever is
they just learned from all the things
they did in 2015 and put it into
automation software malware right and
call it crash override or in Destroyer
um and so that malware was then able to
work on the transmission side of the
house and one transmission uh site they
took down was three times the overall
power loss of everything in 2015 because
it was at a distribution level
so now your engineers and operators and
and electric Engineers were freaking out
because the idea that somebody can jump
on and use a distribution management
system to open up a circuit breaker who
cares the fact that somebody can build a
protocol aware software stack to
automatically disconnect transmission
substations across the country
terrifying
and this is where then I got invited to
testify in the Senate and in 2018 here's
the story I told them all right this is
this will kind of bridge us through the
next chapter of the talk here
story I told him was don't freak out
there are a lot of things we can do and
I hate when Defenders make these
statements and it's okay if you have
made it just because I hate it doesn't
mean it's wrong but I always hate when
Defenders make statements
um that I feel are very disconnected
from reality such as well the adversary
only has to get one thing right
Defenders have to protect everything man
none of you have been adversaries
apparently because that's not true if
you're on the adversary side of the
house time is on the defender side
because the moment I compromise your
network compromise your environment any
data and logging that you have for as
long as you're storing it is a loaded
gun waiting for me okay in other words I
have to anticipate maybe you have two
years of logs that I'm not clean out of
that operation for two years because
you've got two years worth of data to go
back and find that operation and if you
find the operation you find one thing in
pivot in the investigation you can also
start linking up to other companies and
other operations so actually from a
Defender standpoint as an adversary I
can take down one Defender but one
Defender can take down hundreds of
adversary operations I really hate that
one but anyways the point is a lot of
these things we throw out
that sound good and lose the traction
really quickly and so what I told the
Senate was we actually can do this
defense is doable a couple basic
security controls not next-gen Ai and
blockchain and chat DBT but next it's
just basic controls
can put us into an amazing place so
let's go invest on those I said but
here's the warning this gets us into the
phase of where we are now
said we have heterogeneous systems right
now
these heterogeneous systems mean that
the oil refinery in Texas has nothing to
do with the oil refinery in Saudi Arabia
the electric substation in DC has
nothing to do with the electric
substation in California
the physics profile the implementation
of the automation the vendors the oems
the integrators it's all different
very heterogeneous which means the only
attacks we see that are disruptive or
destructive are usually site-specific or
sub-industry specific crash override
didn't work on every piece of electric
environment let alone all the different
Industries it worked on transmission
substations
uh the trisis attack which was malware
deployed at a safety system in Saudi
Arabia that they actually tried to kill
people uh they screwed up the attacks
they only cost the company 300 million
dollars in downtime but they were trying
to kill 30 to 60 people that day in the
first ever Cyber attack to Target human
life
um that one only worked at that facility
the style of the attack the blueprint
the tactics techniques procedure sure
you could reuse those and teach people
how to do this and that's dangerous
with that attack that malware only in
that site one of the reasons I was
always laughing when people were like
chasing around the community passing
indicators to compromise out for crisis
I'm like that makes sense like it can't
work on any other safety system focus on
the behavior not the indicator okay
um anyways so we dealt with single site
maybe sub-industry I sold this in 2018 I
said look
my concern is because of this
digitization because the advancements
we're making the industry are all the
right reasons we don't want somebody to
get trained up at Cherry Point oil
refinery and then go to De mom and not
be able to transfer the skills doesn't
make sense from just Workforce
Development and sustainability we got to
move towards the homogeneous
infrastructure
but when you move towards homogeneous
you get scalability
if I'm a kid today that wants to go
learn how to compromise companies I can
set up a Windows domain controller
environment in my basement pretty quick
like I said some ec2 Cloud
infrastructure and then make a bank
setting up a power grid in your basement
is kind of hard you know that's like not
the simplest thing to do so it's it's
hard when it's all that heterogeneous
complexity move towards homogeneous
where we have common operating protocols
common stacks of software software
embedded and other controllers that we
don't even know about that's common it's
dangerous and I worn in 2018 I said
within the next four to five years I bet
we see an adversary come out and take
advantage of that
almost four years of the date of my
testimony in 2022 is when we saw that
it's almost been a second on what this
is make sure I'm on time so spap doesn't
murder me I'm gonna spend a second on
this one because I think this one's
important
so in 2022 in January I'm going to use
some lawyer language here for a second
dragos during the normal course of its
business operations was contacted by an
undisclosed third party partner of ours
and we were handed software that we were
to analyze and to help them out cool all
right got that down now what happened so
essentially what had happened is I had
told all of our partners that have
various sources of telemetry and data in
the world right we reach out it's not
security companies we reach out to like
telecom companies you reach out to
software providers Cloud providers Etc
and say have you ever seen this kind of
stuff call us right that's the kind of
relationships that exist in the industry
um and I've I said hey on this ICS stuff
I know it could be really hard to figure
out if it's even relevant but let's look
to the Past because I'm prior my wife to
an Intel I don't like predictions Intel
don't we don't do predictions we do
forecasting
predictions is here's what I think can
happen forecasting is here's what I've
seen happen and what it might mean about
the future
so I said look I don't want to be the
company that gets caught missing
something we've seen before
so hey Partners here's a couple things I
think you should look for have you ever
see anything that talks about like OPC
as a native protocol or modbus CCP we've
seen these two be abused in the 2015 and
2016 Ukraine tax
um so if you see anything with those
terms or whatever and and the metadata
like we want that one
um if you see anything that looks like
it can modify controllers like it can do
east west from like an HMI or a hue
machine interface down to controllers
like we've seen that before in that s
word if you see anything that looks like
it's actually living in firmware or
injected into memory on a running
controller we've seen that in crisis
um and if you see you know what just to
throw one out there kind of a prediction
but if you see anything targeting
codices because one of our researchers
read Whiteman's one of like the smartest
researchers on the planet he's always
freaking out about codices well all
right give him one if there's if there's
anything CODIS is related and like
software you find let us know
and I think it was like a couple months
later so it was January I get the phone
call hey remember those things I'm like
yeah did you find one with one of the
four modules they're like oh we found
something with all four modules I was
like oh cool
um I'm probably gonna have to brief the
president could you uh could you
authorize me to share this with uh parts
of the government and the reason why I
mentioned the brief the president
discussion is like the president doesn't
care about cyber security nine times out
of ten
but when you talk about somebody in
particular strategic adversary creating
a capability that is explicitly designed
to destroy critical infrastructure
around the country like yeah that's
that's like they care right that's
that's one of those things
um and so got permission started working
with NSA doe FBI and DHS because he had
to list all of them we really just
worked with the NSA but everybody wants
credit
um that's fine anyways um
so sizza was a great partner and they
were very helpful
thank you for your MFA anyway so the um
the discussion quickly uh evolved where
as the dragos analysts were tearing this
apart every piece that we looked at
started getting worse and worse and
worse and what pipe dream really is
that's the thing that we called it well
pipe dream really is is a collection of
software utilities a collection of
malware that allows it to operate in any
industrial environment anywhere on the
planet okay so anywhere it can be at
least a good Espionage tool in most
environments it can take down the
environment and in some environments
depending the physics profile it can
destroy the physical equipment and this
is the piece that gets everybody
uncomfortable that adversary had picked
out that same adversary had picked out
13 targets across the United States and
liquid natural gas and electric and our
assessment of what we were seeing was
that these were 13 targets that as the
Ukraine conflict sort of heated up that
these would be strategic sites to hit
so can't say who it is but you can't
guess and be wrong by the way anyways
but uh yeah but uh anyways so
it is a first time ever that we've seen
a cross industry
repeatable reusable scalable capability
to do destruction hell you can put it on
a remotely piloted aircraft to mess with
the servo Motors like anything physics
and networked this thing works that's
terrifying the worst part is there's no
fixing it there's no like
vulnerabilities that's taking advantage
of it's all the native functionality
across the software Stacks so this isn't
like like I want to patch or like
install a new software update and I'm
good to go it's just it's not how it
works you better be doing detection
response like in these environments
um so pipe dream kind of gets us to that
place of worry but what's the cool part
of the story it wasn't used that's why
like a lot of the community is not
talking about it is there was no big
explosions because the way we went about
our business was to get this before the
adversary employed it I've been coached
lawyers on that term to employed it
doesn't mean that it wasn't deployed
somewhere in the world for testing or
whatever but it had not been employed at
the actual targets yet and the
government the US government was
actually heavily involved with this with
us
um so to me it was one of the best
public-private partnership wins that
I've ever heard of usually they throw
that term out public-private partnership
means nothing almost as much as
information sharing okay whatever but
this one actually getting to work it in
real time and making sure that the
adversary didn't achieve their
objectives at that time now we bought
ourselves some time you still got to do
something with that time because they
can just reuse that thing again right so
I've got to heat it up but now we're in
this era
where we're not dealing with ICS
curiosity
we're not dealing with somebody can
cause an attack in a power sector in
Ukraine and we have a year or two to
learn we're now dealing with
capabilities that can be deployed
re-employed and used across Industries
time's up okay I don't I'm not a
hyperbolic person for most of my career
I've like destroyed all the journals
that come out and be like oh they sent
us phishing email to the Grid it's going
to go down like not how physics works so
I've always destroyed the hype but I'm
delivering a message to the community
this time and testify to the Senate two
weeks ago or something about it again
kind of came back as like I told you
what I told you and here's what I'm
going to tell you again kind of thing
um and uh the point is time's up and so
we got to think about things differently
and there's a lot of different areas of
research that's all kind of in the
discussion or start taking the
discussion down this route and then open
up our questions there's a lot of areas
for research
and I see some really cool stuff coming
out of Academia but here's some things I
would advise you on and right I'm just
some random dude with the beard but uh
advice number one don't recreate what's
already in the private sector and don't
think you're going to be a resourced to
do it better
so I talked to the National Labs about
this all the time I love the National
Labs they're great partners that being
said I cannot I testified in front of
the senate for this I cannot name one
cyber security technology that's come
out of a national lab that they've made
in the last 20 years that is still
viable or used anywhere in the industry
not one although we spent billions of
dollars doing it why it's not that they
don't have brilliantly smart people they
do
but if you look at the ecosystem behind
funding I can go raise 400 million
dollars in the valley to go put effort
on anything
at the same time that a researcher is
struggling for a five million Grant
and so you've got really smart people
and I'm sure that Innovation can come
out of places like this it's all the
others it's not you okay but but I would
tell you yeah just be careful of
recreating or doing a five percent
increase on something that already
exists
but there are things that are really
long-term and strategic Idaho National
Labs knocked it out of the park recently
when they looked at Cyber informed
engineering and what they want to do
there is think about how we're designing
the Electric System in the future and
see if we can design some of the risks
out at a physics layer we're worried
about the valve opening 70 degrees let's
Max it out at 30. what's the profile
that's actually usable what's the
profile that we actually care about
let's design in physical limitations now
you can't design all the Cyber risk out
especially that native functionality but
you can remove some of really important
stuff
uh now I will caution you uh especially
if you ever hang out with policy makers
they talk about that as like going back
to analog we don't use those words okay
that upsets the electric providers it's
not analog it's next Generation
non-digital assets okay that's the term
anyway so
some of the stuff just simply doesn't
need to be connected to the internet all
right some of it does this whole idea of
like I can't believe people with ICS
connected to the cloud okay
um we connected an oil refinery to the
cloud and saved I think 480 million
dollars a year yeah we're going to
connect that stuff up that's going to
happen but let's talk about the security
around it and how to do that okay so
think about those longer term
investments think about the things that
can be really impactful by thinking
about the problem differently if you're
coming up with yet another IDs or yet
another antivirus product or yet another
application white listing Source like
I'm just telling you it might be good
for your development might be good for
your education don't think it's going to
put a dent in the problem what's the
biggest part of the problem resourcing
like scissor just launched it I don't I
don't always beat up on says it but I'm
doing it twice so I have to apologize
now this is in advance and that's like
send Jenna tweet or something but
um
sizza went and testified the US
government that they needed resources so
they could go around and do cyber
security assessments at water companies
there are 55 000 water companies in the
United States about 54 000 of them don't
have resources more than 30 000 of them
share one it person between minimum of
two but usually six water utilities on a
contract
they don't need AI blockchain cyber
security assessments they need basic I.T
support
they need a firewall okay like that's
the level of wherever infrastructure is
so a lot of the Security Solutions get
designed for kind of some of the top end
companies but if you're talking about
actually making a dents in the problem
figure out how to have something
scalable and repeatable for all the
people that need the basics that ends up
being really good contribution so that
type of stuff and then when it relates
to Industrial Control Systems remember
the system is not the point so as you go
look at vulnerabilities and research it
and like fuzz it and all that and all
that stuff
please appreciate that it's a system of
systems so nobody actually cares if the
engineering workstation gets compromised
but did the engineer workstation get
compromised by somebody that has the
knowledge to modify the controller to
cause an impact that's physical and
system three system one to system two to
system three now we all care whether
it's an adversary or an Insider
so understand the place and the role
that it has before you prescribe
security answers I see a lot of folks
that want to tell me how to secure the
control system but in the environment of
systems and systems you're doing worse
as an example various folks keep asking
for TLS or encryption on the ICS
protocols that's a bad idea
sounds good we're going to encrypt the
protocols cool against what
well man in the middle like what pen
test or hurt you like what when have you
seen man in the middle attacks in an ICS
Network and also if they're in the ICS
network doing that they own the
endpoints and the encryption is
irrelevant but now you can't monitor it
so before you prescribe the security
answer understand the context of how
it's going to be used think about that
system of system then think about the
mission of the site when I try to tell
people getting into OT they're like I
want to get an OT I'm like awesome we
need you come on in the water's warm
okay but
take a box of donuts in a case of beer
and go talk to an engineer first okay go
talk to the people of the substation the
manufacturing facility et cetera hey
what are you trying to do here what's
important learn what they're doing
then think about the risks then think
about the security answers don't walk in
using this cyber security framework or
62443 like a bingo card and go in and
say look at all the things we can do for
you because I can't believe you're not
patching your systems it's an easy way
to get kicked out I have responded to
more cyber attacks excuse me I've
responded to more outages at plants
by well-intentioned I.T people then
China Russia and Iran combined
okay we got to be aware that sometimes
the operations people say get out it's
because people have been there before
and done it poorly so if you can
understand the mission and talk in their
language and and actually try to
understand them before you actually
prescribe a security answer it's going
to go a lot further okay I'm sure that's
applicable to I.T but it is really
applicable in OT all right so with that
I think I'm like winding it down but I
in in hopefully I'll have time for
questions but when you think about all
these policy makers around the world
freaking out this is kind of the story
arc of why
since 1998 Presidential Directive 63
we've been talking about critical
infrastructure cyber security
but everybody went and did it on the I.T
side of the house not in the OT side of
the house the critical part of critical
restructure now all your governments are
telling the CEOs figured out faster
we're going to regulate you and some of
the regulations are not all that helpful
as we saw with tssd2 in the pipeline
sector so with that hopefully we have
vibrant questions and insights I can
happy to answer anything that you'd like
until staff kicks me off the stage all
right what we got just don't be boring
yeah good
about them
is there like people online or something
or
recording they want to record you look
at all this next NSA people yeah
I'll walk back so it's not awkward
you'll like talk to me right when I'm
there yeah
all right so um
in 2018 I think Nesta published
the special publication 800 193. yeah
for more resilience and Recovery
and yet you know
six or five years since
there's only a few companies right now
that are making products that have
adopted that capability like microchip
Infinium lattice even Micron recently
wearing the standards or in the
requirements you've seen in government
procurements have those requirements
been included yeah no that's
um nowhere that I'm aware of and that is
explicitly that was I made two other
points in my Senate testimony this past
week um that hit on that exactly Point
number one was
when the government talks to the private
sector about the various standards and
regulations it is hypocritical when the
government does not follow them
themselves and I made the point because
when government agencies getting I'm
going to trust me I used to be in the
government and served my time I believe
red white and blue if it cut me deep
enough so anybody's like he's bashing on
America hold on just just the American
government agencies anyways
um so when we look at the government
agencies what happens for them to get
budget they go in front of those those
panels like the Senate energy committee
as an example if you're the department
of energy and all the problems that the
Senators are worried about which is the
critical infrastructure security they
take on and say we could fix it if we
only had more Authority in budget
even though they don't actually do
security in the private sector right
they're not responsible for any of it
and so there's a lot of senators who
genuinely believe that sizza as an
example is protecting U.S critical
infrastructure no they're not they're
contributing they're awesome people
they're good Americans they're not
protecting it the local power company is
protecting it so one of my statements
there was explicitly designed to help
the Senators understand that when I said
look the government agencies are less
protected than most energy sites in this
country and it kind of causes that
juxtaposition for them the second point
that I made is about the vendor
responsibility because they were talking
about more regulations on the electric
sector I was like why they're very well
regulated put regulations on the vendors
I was like I think I'd probably one of
the few vendors ever gone to front of
Congress and asked for regulation
um but I got up and said look here's
something that we do as an example I'm
not trying to brag but I'm trying to
talk to you about what you should not be
allowing me to do I said for our
software development for our technology
we know our technology gets deployed in
rail oil and gas glass factories nuclear
power plants things like that I said so
to me it's not that one country is
better than another it's just
controlling the supply chain it's hard
so we've made the very costly decision
to have 100 of our software developers
be U.S citizens in the U.S that sucks
from a balance sheet perspective okay
like that is very very costly but now I
have complete control over my supply
chain as it relates to what goes in my
software doesn't mean we can't do we
don't need to do all the rest of the
security but it adds an important piece
my point to the Senate was why am I
allowed to make that choice as a CEO of
a company that Services critical
National infrastructure why I'm allowed
to you want to say that the GrubHub app
doesn't have to do anything like that
that's fine
but if I'm allowed to sell into National
Security problem sets I should probably
have some standards that I have to
follow to be contributing to National
Security
and so we right now at all the
asset owners and operators and all the
vendors are there in op-eds oh yeah I
mean there was even this OT Coalition
group that stood up I can't talk about
this too publicly because I sound like
I'm just complaining about competitors
but they wanted us to join and we
refused to join why because they're
advocating for more regulation on
Industrial control systems in the
sectors my point of view is that vendors
shouldn't be having conversations with
Regulators about the regulated without
the regulated present and so I just I
have a fundamental conflict on that so I
think the vendors are screwing so as a
follow-up to to that
um The Trusted compute group The Trusted
Computing group you know came out with
TPM 2.0 the military made a mandate
we're going to buy your computers for
our systems desktops and servers because
they're going to have to have a TPM in
it yeah okay so then Along Came dice
that Microsoft outsourced but nobody's
implementing any mandates that talk
about recovery for firmware and
monitoring
that you mentioned earlier with pipe
dream yeah there's there's nothing
coming that I'm aware of on all that the
cnnmc stuff is probably the closest to
getting like the subcontractors involved
but all you actually have to do is
Define your risk profile by the standard
or by the regulation so there's no
actual follow-through so the good
companies will do good things the bad
companies won't and we'll still be
complaining about the bad companies
yeah
well good morning I'm Eugene from Purdue
manufacturing Extinction partnership and
I'll admit at the beginning I'm doing
all of those ineffective things that you
just spent the last 30 minutes talking
about you're doing something yeah
well okay so the part of the industrial
sector that specifically we're focused
on is manufacturing
what we're doing there is driven by
Contracting government and DOD
Contracting you want a DOD or a federal
contract you have to put these controls
in place they're focused on I.T yeah so
that's how that's what that's the need
right not necessarily focused on OT if I
were to go into a small manufacturer
these small manufacturers less than 100
people and say show me your CNC machine
what's the risk on that same machine CNC
machine if someone were to hack into the
OT side of the system
um and they'd say well how does that get
me to where I need to be in order to
meet the compliance requirements totally
different from Effectiveness yep but
that's the problem yep okay so
um with and so I did note that on
detection response detection and
response
um for a small manufacturer
uh is is pretty expensive if I'm
assuming and I'm not sure that if
looking at the event stack
using the I don't know let's just throw
one out there Alien Vault it will I can
see the things that you're talking about
I don't know if I can or I can't but um
it's pretty pricey for a small so so
then that's so I'm looking for I made
more of a statement than yeah
so so let me get to the question then
given my constraints given how I'm
funded to go help small manufacturers
continue to generate Revenue how is it
that
um that you think that I can do this and
still
um help them be more effective on the
way to compliance is the question one at
a at a price
secure and compliant yeah got it okay
all right let's solve this one for you
um I wrote a paper uh titled the five
critical controls for ICS security it's
over at the Sans Institute you should
follow that and you could do everything
you just said but I'll give you the
longer version of it which is one of the
problems that I see with most security
teams is they don't know the
requirements and so they look for
everything so like the fact that you
want to look at all the events in Alien
Vault by a default is wrong
um in my opinion right the whole my
opinion piece is important here as an
example what is that small manufacturer
actually care about they should be able
to decide the two or three scenarios
that they care about I bet ransomware
across their operations facilities is
one of them but pick out actual
scenarios that have actually happened
not theoretical I could use the
Bluetooth to spin the wind turbine I
don't care has it actually happened no
well then it's not my problem well
you're not being proactive I understand
that but that's also another thing that
people say just as a quick tangent that
always gets me going we want a
world-class security team no you don't I
don't want a world-class security team
my company exists to build and sell
software to help the companies we work
with protect the people in their areas
that's what I want to be world class at
you can be world class at one thing I
also want to do good security but I want
to have the bare minimum of what I would
consider reasonable security Now what I
consider reasonable security is a hell
of a lot more maybe some others but
that's what I'm doing I want a
reasonable Erp system a reasonable
netsuite a reason I don't want a
world-class Finance program I want a
world-class development team pick one
thing right so your manufacturer have
chosen be world-class what they're doing
that's great we should plot them for it
what are the scenarios that actually
matter there's probably like two for
them that actually impact them on on the
revenue and by doing those types of
things correctly in OT security you're
probably getting visibility just for
like root cause analysis of maintenance
issues and similar so this can be just
value on the back end of it I think a
lot of I.T security doesn't contribute
to I.T resilient like I.T in general
even though some of it does but a lot of
the OT security done correctly is just
getting through the complexity to
understand what the hell's going on
anyways and the ability to get root
cause analysis and downtime is super
core so it actually has a lot of value
anyways pick out the scenarios then it's
what can actually trigger those what
systems are relevant in that scenario
what data is relevant and you Pare down
the data set you Pare down the alerts
you pair down everything to just what
you actually should care about
everything else is go hunting later if
you have magical mythical free time
otherwise protect yourself against the
actual requirements that business not
what we think are the requirements which
is all the different cyber risks because
you just will never do it
uh it's the Sans uh ICS uh five critical
controls so it's five critical controls
for ICS security Tim Conway and I wrote
it right what's up what's that so
looking for another opinion
um two things one I spend my day doing
Enterprise ransomware rescue and so I'm
very interested in maybe some point
having a conversation with you about
what does that look like on the OT side
versus it
um the other thing is on the sensor
reliability of the data if you're
monitoring you have to be convinced that
the sensor data is giving you the
information you've been reading Joe
Weiss all right that's a really good
friend of mine and that's what great I'm
very interested in your opinion on let's
do it so that's why I brought it up so
I'm doing a lot of time and that's his
big thing is if you can't trust the data
so I'd love to hear what you think I
love Joe Joe's been a friend for a long
time I usually refer to Joe as my Winnie
the Pooh spirit animal call me at three
o'clock in the morning Rob it's an
aurora online okay Joe
um but super nice guy um and here's what
I would say there's a lot of like new
entrance to ICS security community that
like make fun of Joe I don't want to
like
stop it just for a quick second
um I feel like I can go toe-to-toe show
Even though 11 death he's been doing it
longer I feel like I've earned my
ability to do that if everybody else is
like taking pot shots let me be real
clear that the guy has earned his
stripes uh he's been there done that and
his opinions are valid I don't agree
with most of them but they're valid I
don't agree with this tact or they're
valid uh and further uh his mentor was
Oppenheimer so like everybody calmed
down a second the guy's like Royal
Legacy in the community okay anyways um
so sensor data I appreciate what Joe is
trying to highlight and from a process
Integrity perspective the man is
absolutely correct
where he's wrong in my opinion is two
things one he keeps saying that nobody's
doing this that's just not true like
Honeywell has been doing process
validation on their sensors and I O
units since like the 1980s you know this
is it's not true that nobody's doing
this
um but it is a very important thing and
a lot of Engineers don't do it correctly
to actually see what's going on the
process data have a loop that may go
directly to a story in a loop that then
goes to the HMI to be able to see and be
able to cross compare the data sets very
very important the other thing that I
think is doing wrong is he's equivalent
he's putting it with cyber and cyber
security and it has nothing to do with
cyber and has nothing to do with cyber
security well yeah but you know and then
you could see in the i o no by the time
you're looking at Analog data and like
an i o output the Factory's gone like it
already blew up okay if you're talking
about physical impact it blew up so
could it be useful for forensics
potentially but we already have all the
other forensics through all the other
sources we need his statement is there
are attacks that are undetectable
through the network no there are not
there are no cyber attacks possible
remote cyber attacks that don't go
through a network so the idea that you
can only do it in i o is ridiculous what
he means is once the attack goes to a
certain point there are some things like
heart over serial instead of heart IP
that you lose visibility into because of
the way they designed it that's
absolutely correct but all the detection
response recovery everything else is
already done so if I was
if I did everything else in security and
had resources left over for certain core
assets and like a nuclear power plant
I'd probably deploy some of the
solutions he's talking about but he also
ought to disclosed the solution that he
talks about the most and pitches the
most he has a significant Equity stake
in he doesn't tell people that yeah
yeah no problem
all right what else okay DD bootyharto
I'm a recovering seesaw from oil and gas
company
um so this is mostly called a call for
collaboration and get your thoughts
right as a practitioner in oil and gas
company the otn ID converge not converts
has always been a political discussion
but what what I am have been seeing is
that we need to have that three-headed
Monsters the OEM the original equipment
manufacturing the government and the
owner operators as the oil and gas
companies to work together this is the
case because
um the owner's operators are the ones
being penalized right now with the TSA
regulations whether or not to comply
with the regulations whereas there's no
incentive that I've seen or I've known
from the OEM to be in that compliance so
it's always us playing catch up and The
Regulators are off doing something else
and those three headed monsters are not
talking to each other and what's your
thought on that yeah I would add in your
integrator and I would also add in your
cyber security service or technology
providers especially because your oems
are now trying to pitch their own cyber
security offerings which really half of
them are things they should have just
built in their product but now they're
trying to sell to you
um so I would include those as like the
five discussions and then you got to put
teeth behind it because we have these
this is a sort of jcdc they try to have
meetings there's like a slack Channel
nothing happens in it uh there was the
DHS sponsored
um uh logic Forum which was exactly that
in the oil and gas industry yeah yeah so
we have all these forums that's why
every time some other new Cyber Security
Agency reform stands up you just kind of
roll your eyes you're like yet another
one
um and and so we have all them until you
actually have the stick nothing else is
going to matter you either need to
change the law around antitrust to allow
oil and gas companies as an example to
band together and say we all want these
features in a product and your combined
purchasing power can force it but right
now it's illegal to do that because
antitrust or you need regulations on the
actual vendors themselves but another
way to do it is for the government this
will go back to the point that the
gentleman made earlier the government
could also just assist in a heartbeat
like everyone thinks it has to be
regulatory if sizza came out tomorrow
and said here's what we view to be the
right security features on the next-gen
plcs every company would take those and
staple them to an RFP and not violate
antitrust and the vendors would provide
it overnight and then DOD take their
purchasing power and go we will only buy
control systems that meet to this level
you'd get everything on the market you
want but instead they want to talk about
you know Regulators want to regulate
policy makers want a policy but honestly
all they need to do is have a point of
view I've asked them why don't do this
and the explicit answer is yeah but what
if our point of view is wrong
okay but somebody's got to do something
and I do think that should be on
government agencies that are out there
like we protect critical infrastructure
okay do something just do something
right right um so yeah unfortunately but
yeah this is the powerful collaboration
so together we are stronger I think
we're
just in the next perfect I'll be uh more
quiet or more uh concise I guess we
already drank together what's up man
uh you mentioned earlier about how the
different vertical markets are
fortunately you know historically not
homogeneous which has worked out to
everybody's benefit in this sense
I'm curious if the code assists
uh software which I guess you could
probably accurately describe as sort of
middleware pretty solid systems yeah
it's like an entire operating system in
a protocol right yeah do you think that
that that made the uh pipe dream attacks
more economically feasible because it
was like this research had been done by
a company that was you know had a
for-profit motivation obviously and they
had put all the research and development
into it so that it could have done but
you're nodding just answers this
question yeah yeah so long story short
we've now gone homogeneous codices is
one of a couple things that have done
that I don't advertise out what the
other ones are because the adversaries
do listen
um and that goes to your other point
about research and development be
careful on that too please researchers
there is a university in the United
States it doesn't matter which one that
published a paper a year before the
crisis attack of exactly how to modify
that exact version of triconics and do
exactly what the attack was and they
made a coding error in it which the
adversaries even copied so we know for a
fact they read the research paper so
when you put out like here's new ways to
take things down like people read please
be careful all right yeah sir yeah so
this is actually a good uh timing to
actually follow up so I have a follow-up
question so I am a academic cyber
physical assistant security or it plus
OT security researcher so I really
appreciate the uh the like the the
pitfalls that you mentioned to avoid
especially for academic research and
wheels that we should not reinvent uh in
the area of cyber physical or it plug OT
so I wonder if you might have any more
like uh constructive uh kind of a
suggestion like which like what areas or
what directions that we could you know
we in Academia could focus on given our
you know the way the mode of our
operation the resources that we do have
here uh yeah yeah I would say
um there's just a lack of knowledge in
General on how to deal with certain
systems so take the latest Honeywell
Experian distributed control system cool
if something were to happen to it what
are the relevant data sources on it
where are they stored how long they
stored what's the time stamp uh
considerations on a UTC plus whatever
um uh what data would you pull in
different scenarios like hey if this was
attacked in such a way that copy the
blueprint of what we saw with prices but
in this type of environment what would
you do so the can I detect it that
markets got it um can I try to prevent
it that Market's got it can I respond
and recover nobody's doing that stuff
nobody's developing like the playbooks
and then here's how to do those things
that's one of them another one is just
the general research on building
materials I think is important because
I've looked at every single IC so
there's a bunch of different ways to get
bill of materials and software
um broadly speaking you do binary
analysis to kind of like decompose like
what's in it or you do like source code
analysis source code analysis already
done don't waste your time sneak and the
others have got that market on the
binary analysis there's like five
companies in the ICS space that are
doing it because you're doing it in it
doesn't make a ton of sense because just
wait for the next version ICS we live
with that stuff for 30 years it's a
market
um the problem though is I I like the
idea so much that I went to buy one of
those companies so we got in advanced
discussions and Tech due diligence with
all five so I won't break any ndas by
saying just like a higher level
statement I looked at all five and
walked away thinking there's snake oil
um but the other thing is worth noting
if you're doing bill of materials
correctly you're actually solving the
antivirus problem
can I accurately identify what this
software is regardless of what it tries
to tell me is the same problem which is
why the bill of materials companies in
my opinion don't have very good
viability chances because I just don't
believe that they'll solve with the
antivirus industry over the last 30
years hasn't been able to solve an
unsolvable problem in a market that
somebody hasn't figured it out sounds
like a good research opportunity
excellent point Thank you Rob I'm also
glad to know that we are on track you
know kind of consistent what you what
you just yeah perfect thank you perfect
absolutely all right sir over to you
thank you