thing that I'm helping other folks to
build them across a lot of different
types of collections a lot of different
problem sets everything from you know
databases of cable messages to you know
industrial or like supply chain data
even some really pretty cool stuff like
crop reports and agricultural research
and weather reports mashing up that kind
of stuff or even up you know media
exploitation outputs and really kind of
hairy stuff to try to handle search on
top of so you know so I'm you know I'm a
builder you know that's that's how I
look at multi go and multi go kung-fu
and how I come at this topic so you know
it's it's a I think the last panel was
terrific I'm very excited that I've got
something pretty different to share with
you it's not a cloak and dagger story
you know it's not a takedown it's a to
look at multi go kung-fu and you know
open source intelligence for that
perspective you know we've been looking
at this for about a year and we've been
getting a lot of a lot of our customers
to thread into analysts that we work
with saying hey we're using multi go a
lot in our environment it's a terrific
capability we think you guys should have
a strong integration doing what you're
doing a record of future in this product
and so from my perspective trying to
figure out exactly what are they doing
with multi go
you know why is that important what
problems is that solving how can we you
know help them solve that with our
integration as best as possible so you
know over the course that year a lot of
analysts have been willing to talk with
us you know show me what they've been up
to look at what we've been doing with
the integration critique it say hey this
/ - this part's bad don't you improve it
you know and this in this way so in
today's talk what I want to do is sort
of you know pay that forward to all of
you and I think you know there's some
some lessons that we've figured out and
learned and the integration between
multigo and our product that are really
pretty general you know it doesn't
really matter I think in terms of
looking at these whether you're using in
a multi go or some other link analysis
product or whether using record a future
or not or I mean we're all using Google
so everybody's doing open source
intelligence to some extent so so that's
what I have for you I'm actually you
know it's because we're talking about
integration patterns and user experience
I didn't want to just give you a screen
shot demo so I'm actually gonna give you
a live demo which is either brave or
foolhardy we'll see how it works out
and so I'm just going to set up the
topic a little bit and then we'll jump
into that and I'll show you in action
what we're talking about here so I'm
going to circle around to multigo but
I'm going to start with the you know the
open source exploitation and I start
here if you're you know an extreme
skeptic about the value of open source
information as part of threat Intel I
hate to disappoint you but I'm I'm not
gonna try to disprove that with some
sort of evidence that you know X
percentage of the information that you
need to do your work is an open source
or there's something like that I think
at least in this community we're mostly
past that conversation and that there is
value it's not the totality of value you
know it's it's it's a tool and the
toolkit you know I think when I'm at a
cocktail party and I say you know we're
collecting information from the web
running it through NLP you know
producing information that's useful for
security people say wow you can you
actually do that but not so much in this
community but there is skepticism still
and I think when I think it on here you
know this is the this is the one
football reference in the presentation
right so this is the logo from a bunny
Morning Quarterback they had a great
quote from there you know all twenty-two
film analysts on Monday and after the
Superbowl he said everybody's favorite
hindsight is everybody's favorite
perspective and I think this actually
came up in yesterday's discussion in one
of the sessions that a lot of the
information you find an open source is
actually reporting about something that
that has already happened you know
somebody observed an event they went to
the web they wrote it down maybe some
device you know observe some technical
behavior they've published that out so
you know by definition that for that
particular situation is not Intel you
know Intel us before it's news that's
now an observation of what happened in
the past now you know you can take that
information and say you know here's the
context here's what's been observed
you know therefore the trend is this and
you do that that type of analysis work
and drive Intel for that information but
that's you know something quite
different and more complicated so I
think this is the the second quote I
want to pull up here this is it's a
tweet from medical ACS and I think this
sums up this this response to
open-source Intel free well says it's
not anecdotal evidence it
artisanal data which is you know so yeah
you can find some information in open
sources you can you can you know
extrapolate a trend you can figure
something out but what are we really
talking about here is this some you know
like small handcrafted your small
batches of threat intelligence that that
craftsmen can produce or is this really
something that can be you know done over
and over again you know by by mere
mortals you know I said it's to put it
in business terms as you know consistent
and repeatable process so I mean this is
the question that that I think about a
lot like how can we take exploitation of
public you know open source information
and make it something that you can do
over and over again do it with a level
of confidence that I've done enough of
this and now I can pivot over into other
other sources other methods or just you
know to say we're done with that let's
move into action so you know why would
this be hard and I guess you know don't
get me started right but I'm gonna pick
a couple of things I think one of the
obvious you know things ought to say and
we know this about there's no end of the
internet you don't get to that page and
say okay my collection work and open
source is done you know I've done enough
and now I can wrap it up and move to the
next it just doesn't work that way and I
think you know the you know the other
point here that you probably have heard
before it's a fairly obvious one is that
it's not a static web right you know
maybe the piece of information that you
needed for your research free analysis
was there last week but it's not there
anymore you know maybe it's it's going
to be there you know five minutes after
you get done with your search you know
it's a very dynamic environment and I
think as you start to look more closely
at this you know there's each different
type of data that's publicly available
and a lot of the speakers have talked
about their discipline and domain you
know the the slice of that information
that they work with and we specialize in
text written by human beings for the
most part it has its own sort of
challenges around collection processing
understanding you know what's what's
novel what's interesting what's abnormal
you know what's interesting from an
Intel perspective so you know you get
into this whole set of problems around
exploiting public information that are
pretty specialized I think from the the
analyst standpoint that's good I I don't
want to be doing that you know
don't work and a huge organization where
there's a whole other staff that's
charged with collection and processing
and I'm just the analyst this is also my
job too maybe that's what it's like in
government but this is industry you know
it's it's a different different playing
field so you say you know are there are
people that I can work with around that
and there are there are a lot of folks
like us that are specialists with you
know one area or another of open source
collection and exploitation and then you
get into the whole question of you know
what's my loadout you know what what
matters are these things overlapping one
of the speakers yesterday Alex talked
about this you know from the threat IQ
so this set of challenges with open
source exploitation this is you know
sort of where I spend most of my you
know waking hours working and trying to
figure out how we can bring a better
solution but I want to pivot for this
talk there's something different which
is you know the I think there's a whole
other set of challenges which relate
relate us back to multi go and that's
that you know this open source stuff
it's not that it doesn't exist in a
vacuum and it's not the whole the whole
set of information and even if you you
know say that 90% of the information
that you would need is available in
public and open sources you still need
the other 10% I think we have opened out
enough software projects to know that it
can be 90% ever complete and 0%
delivered and you know there's no value
there right so you need the rest and so
you know this is what it actually looks
like you know the the speakers in the
panel before we're talking about this
how they had tips they had information
from private collections and they were
looking at other you know open source
and other other media this is how it
actually works you know the the
investigation usually doesn't start from
open source it starts from other more
targeted information it's an iterative
process you say I've got some you know
got an incident that's that's been
observed I've got some observables ipys
domains hashes something like that don't
really have much context around that can
I look at open source and get some more
information and then maybe I dip it
right back you know look at private
collections maybe I take that and look
at some of the research discretion
subscriptions that I have access to so
this is what you know if you want to
have a repeatable consistent you know a
process for open source exploitation
it's going to have to look like this and
then I think the question that you get
to is so you know it's iterative but how
many iterations am I going to be able to
to pull off you know what is what does
it actually practically look like and so
I wanted to pull up this kind of a cool
picture it's a little dim but I think
you know you can probably still make it
out anybody recognize that very close so
it's I it's from homeland it's a picture
of Carrie mathison's sort of wall of
intelligence you know full of all sorts
of I guess in the story classified
information that she's brought home and
in a second
hopefully we'll come right back into the
slideshow so that's fiction and that's
great that makes me happy
but I think you know when I look at it
oh we were having fun this is the great
part about live demos okay so the UH the
other part that strikes me is this
really fictional is if you think about
the I'm sorry kinda never rains but it
pours huh are we back okay
if you think about the sheer amount of
time that she would have had to put into
sourcing all of this you know
information processing and collating it
organizing and so forth and how many
investigations actually get this level
of focus very few and you know the the
amount of work that would have needed to
go into bringing all this information
together it's very I think I find it
very impossible it just doesn't it's not
gonna happen
very often to go to that degree it's
because you you know again the panelists
in the last session we're talking about
how there's a friction there's a
coefficient of friction in getting
access to this information sharing is
great we're gonna do more of it we're
gonna automate more of it it still takes
time to bring information in and out of
the the Intel space that you're in the
analysis space that you're working with
so if we're gonna make this o cent work
you know repeatable effective
predictable etc we're gonna have to
drive down the coefficient of friction
and so we look at that you know we got
our bobsled uh-huh it's an optimization
problem right so we got to focus
somewhere and you know and what we've
been doing with our integration we're
focusing on the operations that happen
very frequently and the operations that
happen at the the inception of the
investigation right cuz they always
happen and it's these sorts of things
it's uh you got a set of in front you've
got a data point or a name or a key and
you want to probe the collection see if
there's even anything there to start
pulling on the piece of string and
following it down maybe you have a set
of observables that you have right now
you want to see what else might be
linked to it to potentially expand the
set of observables and then you know
bring it back against log data or
network data or other collections and
obviously there's there's going to be
pivoting you know got your in a
particular area you want to pivot the
investigation to go in a different
direction and at the end of the day it's
not just about you know of course
collecting more data and so forth it's
about bringing it down to the set of
observables that are actually indicators
to compromise their indicators of recon
or they're going to have value for a
detection or block rules so here's our
gonna bring it back around to multigo so
I think there's a link between all of
these sorts of issues and and why we've
been hearing so much interest in
excitement around multigo and actually
want to pause right here and ask Rick
was you know you asked about multi go
use I'm curious uh in terms of you know
link analysis tools in general it could
be you know about take oh and let's
notebook Palantir centrifuge Thetis even
like a like efi a graph you know tool
just show of hands how many people are
are currently or have work with a tool
like that it almost good so when i jump
into the demo i'll do the first path a
little bit more slowly to sort of
situate it for the the folks who aren't
familiar with those tools but you know
we'll go pretty quick so with multi go i
think the first reason i mean there are
a lot of great tools out there that are
link analysis capabilities one of the
things about multi go is you know we
it's it's very available you just go to
the website you download it it's pretty
affordable you're not gonna have to go
you know three levels up in the
organization to get sign off on the
purchase order which you know I mean I
think that's just in that accessibility
is an advantage and why it keeps popping
up you can learn to do these operations
in multi go go work in a different
organization you can take those skills
with you so that's cool I think that's
what's driving some of the interest all
of the conversations in the last panel
about the difficulties of you know you
here's some information and confidence
and you got to figure out how you can
share it and bring it into this analysis
space there are a lot of you know human
judgments that the analyst is making and
I think you know for better or for worse
multigo is not part of some great big
system with a database on the back end
of it the investigation files sitting on
your hard drive right so I think to put
it this way it's sort of analysts owned
and operated you can make those choices
you can put the data into the
investigate
whatever form whatever redacted form
that you think it's useful without
having to get somebody else to do an IT
integration or whatever else might you
know delay the process by until it's no
longer relevant when you're putting that
information in I think this is pretty
cool you know you can almost like
sneakernet but you just copy-paste i'll
Teague will guess at what you you know
what type of information you've just put
into the investigation might get it
wrong you can correct it but that level
of you know there's not much friction
right there which is really useful
and then lastly from the perspective of
of us as an int Oh information provider
it's got a good set of interfaces around
plugging in so that we can make our
information really accessible there now
I wanted to pause here and say obviously
the the folks that I'm talking with are
interested in multigo because they're
talking to me about multigo because
they're interesting getting a record of
future data into it but you know aside
from using the built-in stuff and you
know interest in what we're doing what
do you think is that the most that the
top collection that I always hear people
talking about wanted to integrate with
their multigo deploys any hazard a guess
yes
so for everybody who is saying yesterday
that your own logs your own data is that
the most you know sort of highest value
most untapped source of information that
you have out there that's what I keep
hearing people talking about in terms of
what they want to hook into multigo but
I think it's pretty cool you know it's
some of that the messages are getting
out but you know in terms of wanting to
demo these interaction problem to you
the interaction patterns to you have put
me a little bit of a pickle because I
don't have access to their internal log
data to show you how you know what we're
doing and what they're doing mixed
together in mash up which is really the
point of the talk so it's lucky enough
Mark Hendricks from domain tools gave us
access to his transform set for you know
for purposes of the demo so what I'm
going to show you is sort of using our
information using his information you're
the to collections to pivot back and
forth to show you the patterns of what
you know the folks weren't working with
are doing between our transforms their
internal transforms that they're
building and other transform sets so you
know think about it both for the
perspective of what you're seeing with
East your transform sets but from that
because that's that's what I'm seeing
people doing
and with that we're gonna leave the
trouble with slides part of the
presentation behind and I can actually
jump into the live demo yeah okay ah
excellent
I'll take his reset and I need to turn
on screen mirroring so you can actually
see what I'm doing that'll be helpful
and how's the contrast event okay good
okay so let's see and I'm gonna see if I
can make this a little bit bigger yeah I
think big enough you can you can see
most of what's going on so so for
anybody who has a user link analysis
tool I mean the main workspace here is a
graph it's a set of entities that are
related together in ways that that you
as the analyst you know have observed
have validated you can add notes you can
you know describe the links that's the
basic basic concept and you can reach
out into all sorts of different
collections like ours or your own data
to pull more information back you know
increase the set of information that
you're working with probe some areas of
it you know if it turns out to be
irrelevant prune it delete it out that's
that's what this this type of capability
is all about so I thought it would start
with in a sort of a simple path here to
get us rollin this is a vicinity
represents a vulnerability you know CBE
2014 Oh 497 so let's say that you're
gonna get a an alert this one's a bit
old of course right let's say that you
get alert through this one it says you
know quite dangerous and you need to
look into it so if you drop it into a
tool like this one of the things that
you would do with with OSINT is just you
know reach out to Google they if you
hadn't already there's the the second
hit here is the NPD you know there it is
impact ten exploitability ten so this
one must be really bad so you know
informations accurate let me dig in and
the next thing you would do is then want
to reach out into another collection to
see if you can get more information
about it so the way we're hooking in in
the way multigo structures this each
tool is a little bit different but they
call it a transform and that's where we
plug in so you know we've got a little
piece in here where it's our recorded
future you know data connections you can
say you know what's what's available out
there pull back first let's see
I am going to have to adjust the window
here sorry about that
one sec because there's some interesting
stuff going on on the right hand side of
the screen that you can't see let's try
that
and
okay
so 703 documents there's a lot of
information out there
I got a slider here you know maybe I'll
pull let's say that the most recent
fifty so I'm going to run another you
know data innovation step here and say I
actually want to get the documents back
from recorded future whatever we picked
up you know through harvesting open
sources on the web where people are
talking about this particular CBE and
you know there's a lot that's the the
top fifty I can select them all let's
see I you know grab this node select the
children look at the details so you can
scan across the the titles of these
different things there's quite a few
different pieces of malware that are
being talked about here so maybe a
useful thing to do next in this
investigation would be to grab you know
all of those documents and say why don't
we reach out and and say if there are
references to malware that could be
malware family names it could be
categories of malware it could be you
know sort of it attack steps you know
that are often referred to in the same
sort of ways let's pull all of that
information back and sure enough there's
a lot you know there's a lot more that
I've just expanded and I'm not gonna
read every single one of these obviously
it's way too much but it gives me a way
to screen the information that's
available to me very rapidly without
without having to do the reading the
Machine already did it so let's say look
at these here all that the pieces of
malware that are referred to one way or
the other and the information that I
pulled back you know exploit kits so
forth let's say that you know angler was
a specific interest some folks that we
work with have been you know looking at
that one is a specific investigation so
maybe that's the place that I want to
dig in so find that particular node
there's angler move up in the graph you
know so what are the parents let's take
a look at that again in the overview you
know what do we have going on here right
and I'm actually gonna
so here's a particular document let's
say this one looks interesting and so
now I've got some information over here
publication information the title you
know fragments I can understand why it
is that the transform pulled this
information back and you know obviously
I'm going to want to look at this in its
original form so I'm out linked right so
and I can read back here on the the
discussion of how this this particular
vulnerability and this exploit fit
together so I want to stop here you know
that's the basic orientation to you know
what link analysis tool looks like what
it looks like when you've plugged in a
set of intervention interfaces to a
collection like ours now a couple things
that we can have learned along the way
there's a lot that we do that's you know
sort of more advanced analytics or other
analysis of the information and we
started out trying to fit a lot more of
this into the product but as we as we
work with the integration I think that
the transparency and the simplicity
really came through is something that we
needed to focus on that you know some of
the people working with this information
are gonna know our product but a lot of
aren't you know it's going in here and
in large part so it can be portable and
move to IR or move to forensics or move
to a lot of other places so it's really
important you know in this integration
to keep it simple so that's I think one
thing that we you know initially thought
we should head in a different direction
but then you know I have pivoted that's
a lesson I think you know another thing
here is if you think about how these
these actual you know pieces of
integration functionality are plugged in
you know so how is it that we ended up
you know in this right right menu here
as a as a transform so the way multigo
plugs this in and then this is where it
is you can manage your transforms you
can you know discover some out on the
web you can create a local transform and
we actually started out here where a lot
of folks were saying hey you know
they're different ways to do this but if
you do local transforms like literally
what's happening is this tool is going
to shell out you know onto the local
computer it's gonna run code right there
it'll do whatever it's gonna do and from
the analyst perspective they can see
exactly what it's doing they want to see
what's going on they can read the code
if they want to tweak it they can adjust
it you know they have fulcrum control
and transparency
that sounds like a great idea let's go
down that path turns out your you know
and this is kind of one of these in
hindsight it's obvious you go into
porting right this is Mac a lot of
people are running on Kali Linux some
people are running on Windows some
people have Python installed other
people are running on a box where they
don't have local ad band they can't
change in this lock down so turned out
to be you know sort of right concept but
wrong implementation so have moved back
toward the transform server where you
know we have got this staged out on a
server
our partners at malformities Labs
provide this where you know it's the
analyst all I need to know is the URL
let's go pick those up connect it in all
of the code lives there I don't have to
patch it you know its platform neutral
for my perspective which is you know
really optimizing for the fact that
people want to have that level of
transparency and and control around how
the integration works but they're way
too busy too busy to install it too busy
to maintain it so even though it's a
different optimization this is what's
worked out much better so that's you
know a lesson learned to share I think
then the you've also noticed I think as
I was stepping through here that it was
very much like step step step you know
here's the CVE what documents do we have
what pieces of malware the operations
are really fine-grained which you know
seems tedious you know why would I want
that why would I not want an operation
that takes six steps instead of one that
there's six steps individually so that's
another interesting thing that it's
turned out that by us taking our
capabilities and providing them in a
really fine grained atomic way that way
it actually gives that same flexibility
that we're trying to do with the source
code distribution in another way because
multigo has in it a way to build macros
so you can take you know a set of
operations that we provide a set of
operations that you've built yourself a
set of operations sydney or something
like domain tools offers and mash them
up to do exactly what you need to do and
speed up that that loop where you're
probing one collection and pivoting to
another and probing that so let me show
you that so I've got here you know it's
an example let's say about an incident
report these three IPS are doing
something fishy might be c2 not really
clear now I'm actually likely to get the
information in a different format your
nots not as a multi go graph but just do
some text
you know email some other database
system so I'm just gonna show you I
grabbed that in the clipboard opening up
a new graph pasting it and multigo
figured out you know it's uh it's got
the IP nature in the string so it
classified as IPS so the first thing
that I'm gonna do here is say and I
built these you know these macros these
mashups for you know for the sans event
here that's why they're all labeled sans
I'm gonna say you know let's just probe
directly into record a future see what
we can get around these IDs and you know
I get a little bit so here's where it's
taking multiple steps you know I got a
document pulled the source hold some
domain names that are inside of it you
know w3 not to interesting these two
look fishy so that's you know a little
bit more interesting but you know not
not a ton of information yet another
thing that I could do is do a whois
lookup right try to figure out what the
the net block is above this IP and use
that as a way to probe record a future
and see you know there's some reporting
around that a s number so you know let's
do that went out and got the net blocks
actually that close that so you can see
it more clearly and these three nodes or
the a s numbers right so this one here
you can see you know about 400 reports
looks like a kind of dodgy neighborhood
this one here yep sorry there we go
this one here only two so that's pretty
interesting this latest report from Fred
expert you know with these these domain
names is from a net block that doesn't
have much you know history of
exploitation at least in our data set
and this one over here has got over a
thousand so that's that's definitely a
bad neighborhood so I don't think I
really need to go deep into those I've
gotten a sense of what's going on but
maybe pop back up to those IPs and then
here I'll show you how you can move
across so you know we don't have a ton
of information around them but maybe I
can look over in the domain tools
collection and say those domain tools
have domain a domain information on
these site keys and sure enough they got
a couple so now I've got some new
observables Chateau mal com alpha Moby
calm kind of interesting record a future
didn't have anything directly around
those but domain tools actually has
historic information as well so you know
let's see look into their information on
historic IPs mapped to the
domains now we're getting somewhere
right so about you know four more around
chateau mall you know some more from
alpha Moby actually I've got another
document from a recorded future that it
pulled out to write so here's something
titled ABC of e-commerce comm reposted
by host Anna calm they report who's
hosting it another domain name so let's
say you know let's look back into that
and do we have some historic IDs around
that and now here we got something we're
interesting right
not only is that IP domain lame name
historically linked back to one of the
same IPS that was shared by alpha Moby
it actually connects over into Chateau
mall com if there's an IP that was
shared in common between those two so
I've now found a non-obvious linkage
between those two IPs and well a whole
lot of information down here around
exploitation observed around this
particular IP which is pretty nasty so
or at least has been behaving badly for
quite a long time so I can get a better
sense by looking at those children of
you know what kind of what kind of bad
information has been observed on that
same infrastructure right so this is you
know it's one example of mash up the
putting together these these little
macro scripts that run between you know
markztools the domain tool stuff and our
stuff is uh it's really pretty trivial
you know there's a simple scripting
engine in here there's a WYSIWYG builder
for it and I think that's where this
gets really really interesting where you
can say you know this work of taking a
set of information and casting it
against a bunch of different collections
and seeing if there's a way to advance
the investigation and then pivot back
may be useful but you know it's not
something I'm ever gonna do by hand
unless I can apply this sort of
automation so I'm just going to show you
one last one as a way to wrap this up
the example so far I've been showing are
places where you at the starting point
you had a pretty good notion of what the
you know what the observable was what
the entry point was for the
investigation you know it turns out
that's not often not always the case
so I wanted to show you a couple of
examples
first you know here one of the
challenges we come back to lowering the
friction between you know moving between
our open source collection other
collections your internal data said you
know everybody's got a slightly
different way of classifying and
describing the information right so here
and if I grab you know some phrases here
one of the things that we've recently
built into the integration and I think
is turned out turn out to be a pretty
useful pattern so I can say you know
your clandestine Fox
you know Occidental annan ghost you know
my head they're all different types of
groups you know once a company once said
an operation or campaign and others
activist group let's say I can't
remember exactly how record a future
organizes its information around that
and I don't want to go and have to
hassle with figuring that out to be able
to run a transform here and just say hey
your map it tell me how you describe
that entity and that gives me a better a
better entry point into your collection
so you know there you go it's been
mapped out turns out we have you know
two ways I'm talking about Occidental
you know the parent company and its
subsidiary and they're you know there
you have it and then you know obviously
there's gonna be a limit on whatever it
is the information that we've been able
to organize and classify in advance and
then you'll pick up something else in
your investigation and it's not really
clear what it is right so here's you
know let's say it's a string that came
out of a piece of malware you know that
you have malware sample you're on the
strings on it like you know what is this
to be able to take this and run it
against recorded future or other
collections right and say have you got
any sort of information on this I'm not
really sure what it is well this
particular case it turns out to be a
marker string that piece of ransomware
and the Sinha locker is right again so I
can you know pick that up and now I've
got a way to pivot into a more useful
investigation or on what this you know
this piece of malware actually is that I
picked up you know another one in here
they came up in a conversation the
customer a while ago it's like this is a
user agent string or a fragment of a
user agent string that they were seeing
doing some fishy you know miss behavior
on their network so again you know throw
that against an open source collection
see what you're going to get
and turns out a lot of people were
observing that I wanted to show you this
one because although you know sort of to
drill into in deep detail probably not
interesting most of what I'm getting
here are black lists from people's
reverse proxies so there a lot of other
people out there they've seen this crawl
are doing nasty things that are decided
to block it so initially immediately
gives me a sense of you know whether
this thing is benign and you know
something if somebody else in my
organization is set up or something that
other web webmasters have been blocking
out turns out to be somebody who is
collecting email addresses for spamming
so so I think I'm almost out of time if
not completely out at five minutes so I
wanted to jump back with that into the
slides and wrap it up on this I hit most
of the patterns in the demo I think
there's one that I ran out of time so to
speak to it so the first one you know we
thought initially we should optimize for
giving the implementation to an analyst
in a way that gave them the I think
maximize their their control and
transparency and at exactly what the
machine was doing turned out that we
needed to optimize for them being too
busy so that's you know why that the
remote transform server the TDs and
multi gos terminology turned out to be
really important you know other other
tools have similar capabilities and I
think you need to lean in that direction
keeping the transforms really atomic
this is one that you know it was it's
counterintuitive because as you're
building the user experience it seems
like you're building a really tedious
step-by-step type of capability but then
when you see it in context of transforms
that people have built against other
collections and what you're making
possible by mashing up those those you
know fine grained operations that's when
the magic starts to happen so this is
another one that seemed you know
counterintuitive but you know as we've
gotten further into it turned out to be
you think a really good practice
I think transparent results you know
work you know everybody every good
analyst is a skeptic so anything that we
can do to put more clarity around where
we got the information why we think it's
relevant why we think it's a match make
it very easy for you to drill back you
know obviously that's that's a virtue so
I mean that's something where we're very
focused on it and there's you know more
to do
what and when is the one that I skipped
in the demo path the turns out the one I
was going to show you what's around
shellshock how you know the day that it
broke there were a lot of people you
know tweeting about a free code or IP
address and then within about an hour
you know this there the amount of data
that piled up around that was was so big
there was hard to tell if anybody had
ever talked about that IP before that
day but you know if you can slice the
collection by publication date or by
medium or something like that all of a
sudden becomes useful so figured out
that actually you know that that ability
to apply filters to a very narrow spot
in the analysis chain obviously pretty
critical if you're interested in seeing
how we fit it into the multi go
interface you know that I can show you
that afterwards but you know being able
to make those sort of fine grained
filter choices at particulars places in
the in the operations turned out to be
very key otherwise you get the kind of
ketchup bottle problem right where you
shake it and shake and shake it you get
no matches and all of a sudden in your
graph is a hairball and the last piece
in about expecting mystery meat really
sort of saying in our integration take
the burden away from the analyst to
understand how we map information you
don't probably care all that much give
you a set up place in the information in
the integration just say I'm looking at
this I don't know what you call it you
tell me and make it very easy to step
into probing our collection in that way
and the last thing I just want to say is
uh you know we you've gotten to this
spot by you know a process of lot of
analysts being being generous with their
time and saying you know here's what
we're doing well with multigo you know
here's what we recommend critiquing our
integration and saying I like this part
about it this parts not so much so we've
been grinding away at it there's a lot
more than I'm very excited to do with
this integration but thank you very much
to those folks in particular I wanted to
mention Keith Gilbert a malformed buddy
Labs he's our partner in delivering this
integration and the folks on Andrew and
roll F ed-up Iturbide you know the the
folks that build the multi co-product
been very open and very helpful
responsive and in helping us move
through any sort of tech questions or
use case questions and working with
their product so we have time for a
couple questions if there's any from the
audience yeah and I actually wanted to
invite mark from domain tools since he's
also got a great integration with
off you go if he if you want to address
any questions to him or if you would
like to help answer them I see something
in the back yeah so I wouldn't ask how
much of your collection is general
business information and how can you
distinguish between threat information
associated with say an IP and legitimate
business information associated with
that same IP it's so this is you know
not so much about the integration but
about our product a lot of it is general
business information and it can be
difficult to distinguish it frankly I
think it's also difficult to distinguish
sometimes between an IP that looks
benign and nobody has reported it it's
being malicious but you know may be
important for you versus you know
something that everybody is you know as
observed to be doing suspicious or nasty
things so you know you we're not always
going to be able to tell the difference
with machinery that's why there need to
be analysts what we're trying to do is
create you know aggregates analytic
metrics around that that are useful for
you know sorting the sand from the salt
but at the end of the day you know
somebody's going to have to look at the
information to make that judgment any
other questions
okay guys then as far as lunch goes
thank you Matt thank you
so for those that are doing the threat
stream lunch-and-learn that's in cafe
DuPont emerging threats is going to be
in Fox Hall a enjoy yeah thank you yeah