one of the um things to really think
about that has transpired over the
last close to a decade now is the
revolution of the social media ecosystem
has gone through since really around
2016 although obviously places like
forchan and other um forums have been
around for much longer I like to compare
it to the change that we saw in our
information landscape we move from three
main channels so equate ABC NBC and CBS
to um Facebook Twitter SLX and Tik Tok
right where these are places where most
people were getting their news their
Sports their celebrity gossip all of the
kind of um content all centralized in
one
place and instead of going to one place
to get all of your information you're
now picking and choosing from the
incels.is or you're going to read uh the
latest and greatest stuff on Truth
social right you've moved away from
centralized information on the main
platforms to a world where you're
picking and choosing um your the kinds
of content that you want to see specific
to your interest so if you want to watch
24 hours of the crocodile hunter you can
do that right you can do that on cable
with the nature Channel I think um but
instead if you wanted to read white
nationalist tracks or um be immersed in
extremist ideology you can now spend all
of your time in specific telegram
channels and get that into your
bloodstream all the
time and that change has been um
accelerating over the last let say five
years and some of you may be familiar
with some of these sites um I you know
most of the time when we talk about the
alternative social media ecosystem many
folks will say oh I never knew even knew
miwe existed or I didn't know what dive
was
um there are so many places now where
you can get content right if you're a
militia ader you can go to mym militia
decom and and um hang out with other
militia folks right so it's it's both a
factor of the community building that's
happening in those places as well as the
echo Chambers developing in terms of the
belief systems that are
there and it's not just the fact that
there's so many of the sites it's the
number of folks who are on those sites
and getting all of their information
from those sites so if you look at P
research
polls over the last um two or three
years they've been really deep diving
into the communities on gab on Truth
social on some of the smaller platforms
what they've discovered is something on
the order of four to five% of American
adults get all or almost all of their
news information from those sites now
four to 5% may not seem like a large
number but when you start to extrapolate
out how many people that actually is 10
to 15 million folks who are only getting
their information from some of the
poorest I would say um poorest in terms
of their enforcement of Their Own terms
of services as well as doing any kind of
content
moderation it's no wonder that we're
getting more and more extreme uh
polarization in our country in terms of
the kinds of belief systems and and we
just
saw an event happened in Butler
Pennsylvania the former president Trump
was um it seems like injured in some way
uh we don't know I'm getting a lot of
questions from folks asking me we saw
anything in the leadup to that event on
these platforms I don't know I haven't
seen uh any evidence of that
but the amount of um
extreme acceptance of political violence
of what of basically legitimizing
political violence that's happening on
these platforms that's of concern um and
it's happening not just on on the far
right we are also seeing this and and
more and more folks are talking about it
it's happening on far left even on some
of the more Centrist um you know places
ideology wise where people are saying
things like bring it
on I'm ready for this Civil War or event
to happen um
and yes a lot of this is just
shitposting
but when it starts to become a fever
pitch I think that's when people should
really be concerned about how many of
these people actually believe this um
and I'm going to talk a little bit you
know more on mass shootings um here are
some of Statistics in terms of
how users who now have at their
fingertips a Content delivery Network
that's really really hard to take down
um more and more people are being
exposed to extremist ideologies to
hateful content um something our our
company helps our NGO clients our our
human rights clients monitor try to
understand threats to their communities
threats to ethn ethnic groups um ever
since October 7th you know things have
exploded uh both in terms of anti-muslim
and uh anti-semitic content on these
platforms you know and and I would say
yes there's a correlation between the
rise and this type of content and the
rise in actual real world physical hate
crimes and things like that we can't
draw a CA causor you know Arrow between
those things but it's not hard to
imagine
that being immersed 247 in hateful
content is potentially going to impact
your mental state in terms of
willingness to you know conduct a attack
against somebody on the street um and
this is more and more I think a concern
when it comes to um you know especially
on the anti-semitic and
anti-us now deliberately left out the
names of many of these individuals on on
this slide but I think one thing to
highlight here is the pathway through
which pton gendin the shooter at the
Topps grocery store in Buffalo New York
got to
he was first and and
self-reportedly exposed to video of the
Christ Church shooting by Brenton Terren
who had live streamed on Facebook live
before Facebook decided that it was
probably not a good idea just to let
people willy-nilly stream things live
stream um on their
platform uh that video which I
unfortunately had to watch more than a
dozen times just really sticks with it's
a massacre uh in a mosque in Christ
Church and gendin viewed a clip of that
on forchan became curious about what
that clip was
about started to self- radicalize went
down the rabbit hole um eventually when
he conducts his attack at the Tops
Grocery Store live streams it as well um
clips of that are still readily
available right it's really hard to take
all this content down now um even with
all of the hashing techniques that are
out there uh and also tries to mimic
Terren Manifesto
and if you look at the basically the
Playbook
now um you know before they conduct
their attack they'll go on one of these
smaller platforms they'll post something
and a bunch of people will just download
it right and just lives forever you can
right now go on you know probably once
you go through a couple layers of Google
searches you can find Brenton Terren
Manifesto and someone out there is going
to read that
and it's also what inspired gendin and
you know um Andress braic inspiring BR T
right it's it's a a cycle that is
enabled by the fact that it's so hard to
get this stuff off of
the all this is leading to higher threat
environment than I think we've seen
obviously the presidential election
that's coming up is a big Catalyst of
this
um but we have seen many of these
platforms be used by nefarious actors
for other purposes you know that are not
related to political violence right
there's child sexual exploitation
happening on places like this they
understand that it's a major problem
still having problems internally trying
to stamp this out um gab which was uh
the host of you know some of these
manifestos as well right Tree of Life sh
Compu um you know again many of the
platforms that we're pulling in data
from are not even enforcing their own
terms of service they're stated terms of
service when it comes to what's
allowable and not
allowable and I'm going to talk a little
bit more about telegram telegram's role
in um a uh Adam wafin plot to conduct
attacks against electrical substations
in Baltimore to essentially cause um a
uh
accelerationist race war happened faster
in Baltimore um you know but
again the platforms are enabling a lot
of
this very very difficult uh you know
content that should be removed and I
think that's a major part of why we need
you know AI essentially to help sift
through all this content right our
platform we pull in something like 6
million posts every 24 hours
if you think about what a human is able
to look through on a daily basis never
mind sleeping and getting um you know
some kind of mental health break for
having to look through all this terrible
stuff something like maybe 600 posts a
day is the maximum somebody can actually
credibly look
through 6 million is orders of magnitude
more than that and even if you have
hundreds and hundreds of analysts
looking at all this content you really
do need to have some you know artificial
assistance to basically help you look
through that content to make sense of of
what's going on understand the
TRS so one way of thinking about how
these threats are manifesting um is to
is to
connect how virtual content can
potentially lead to physical Real World
um
events
and is that better yes okay
um wow that's really
loud the kinds of cyber security threats
that we're used to seeing as an industry
as a community happen purely in the
virtual realm right these are um
basically bits and bites that are then
leading to uh you know more bits and
bites being deployed where a breach
happens and that data is being hosted
somewhere and um everything happens at
the virtual
layer where social media comes in and
user generated content being a threat
that's really happening at that virtual
to physical level right where you see
mign organizing on the kinds of events
such as what happened in Ottawa Canada
right and got the trucker Convoy there
one of the really interesting things
with the trucker Convoy was watching how
that threat evolved over time and
deciding to franchise essentially a
trucker Convoy in the United States
whose first threat was against the Super
Bowl in Los Angeles that year eventually
those folks made their way from Los
Angeles all the way to um Washington DC
and try to drive around the Beltway a
bunch of times and people just honked at
them kind of ridiculous um and then
there's things that happen in the
physical world that lead to Virtual
World propaganda so to the um the war in
Ukraine and the kinds of false
information October 7th as well um and
you know what's happening in Israel
Palestine these are physical world
events that are then leading to people
essentially
recrating the the digital ephemera from
those events and posting them on um
virtual platforms to in inform people
misinform people um to disinform people
right that's again an example of how the
physical world events then lead to uh
virtual
content
and I think we shouldn't underestimate
the speed at which some of these things
are happening um and I'll point to for
example what happened to OCTA right
there was a big breach um within I
think hours of the breach happening the
group that conducted the breach was
already going and advertising on
telegram saying hey you know we I here's
some evidence that I've got this breach
data and I'm willing to sell this to the
highest bid right and there's tons and
tons of um user data very sensitive data
uh available there um in terms of
reputational threats we've seen
companies get uh go after like Etsy
Coca-Cola McDonald's um Starbucks um and
then of course there's the threats that
are coming from places like kiwi farms
and incels going after just private
private citizens people who are not used
to or equipped to deal with the Deluge
of virtual threats that they're
receiving and trying to figure out which
of these folks is actually serious about
causing me physical harm right um and
obviously what's happened with budlight
and Target over the last uh uh year or
so a case I'm going to draw your
attention to here is what happened in
Leon Maine um with Robert card Robert
card was a military veteran he was an
Army National Guardsman who um conducted
a mass
shooting and killed 18 people and there
was an incredible amount of speculation
about card's mental state about his
motivation about his whereabouts right
because he was not immediately
apprehended um after the shooting and
one of the things that occurred on
places like true social on places like
forchan where people trying to
dox um Robert card and posting the
information of the wrong robber card on
those sites right so again C causing
harm to somebody who was innocent of
this specific crime um and also posting
you know potential whereabouts of Robert
card and you know false information
about his vehicle and things like that
and and getting National Socialist club
neo-nazis right
posting on their Channel hey we have
information about Robert car where where
where his whereabouts are everybody come
out let's go get them right again
threats to Public Safety happening
organizing happening on on these
platforms right last known location
here's a very specific address right and
what if there was a Subaru there I mean
who's going to take that action into
their own hands um be responsible for
that that's the kinds of things that are
enabled by um these platforms
Adam Waton in their own white
nationalist accelerationist
Channel posting propaganda posting
content about how to take down
electrical
substations right this is part of uh I
don't some of you may be familiar with
the term rhoa racial holy war right it's
part of this white nationalist ideology
of trying to promote um an
accelerationism uh make more likely and
make faster a war between the races they
targeted Baltimore because it's heavily
African-American population um and you
know I think a number of these folks
were just um arraigned uh on this plot
right and again they were promoting this
content on telegram on their
channels so the evolving threat
landscape and I'm going to uh save the
rest of my time for questions because
I'm sure there are some and I'd love to
have a conversation about
this uh we're tracking something like 65
fairly large social networks that are
out there that have emerged over the
last you know decade or so that are not
being monitored by you know most of the
major platforms that are out there um
that is an
incredible burden on an analyst to try
to even figure out like where is the
newest site where people are posting
stuff um we get asked all the time hey
uh what what new places are you seeing
because we want to figure that out for
ourselves and try to go and and access
that data right um and it's not just
text right our system is able to use AI
to categorize images for things like um
KKK type content anti-semitic content
anti-lgbt content um and it's necessary
to do that because sometimes the threats
are coming in the form of a picture
right so an example we had a client who
one of their employees they had uh been
very vocal about um an activist stance
that they took and forchan decided oh
yeah we're g to you know teach this
person a lesson found their photo and
posted their photo along with their
personal information but no name so if
you're only looking at the person's name
you're not going to be able to see that
right you have to be able to scan the
photo and understand you know run facial
r facial recognition on it and match
facial matching to you know an existing
database and say okay this person is now
being attacked in
this
um you know and I think the the way that
these things evolve that's
another change from the
past if you if you go on social media
right now and look at what people are
saying about Trump getting shot in
Butler Pennsylvania there's a lot of
false swag accusations there's a lot of
you know funny memes as well but this is
a serious thing right um no matter what
you feel about the former president and
I'm not a fan of his um political
violence at this level is something that
um for one thing it's illegal the other
thing is it's you know a harbinger of
potentially worse things to come uh
and yes it's might be on one level but
eventually leads to like actual
conspiracy theories nobody took ton
seriously until what 15% of the US
population actually believes The
Conspiracy Theory right a concern um and
then the final thing
is sometimes the call is coming from
inside the
house uh we get a lot of questions and
we don't work directly with law
enforcement or anything like that
sometimes if we see a direct threat we
will let people know because like we
have a responsibility to do that
um but for many of these companies for
many of the organizations we work with
it's sometimes their own employees who
are leaking company information who are
going on to forchan and you know self-
radicalizing right it's coming from
inside the house and so it's not like
you can just go and monitor this and
leave it up to the government to fix
there's also I think a deeper um social
problem that's happening that needs to
be better
understood um
you know and these are some of the
things that we try to educate our
clients on actions they take uh I'll
just point out that I know the title of
this talk was talking a lot about AI um
you know something that we've been
working on I came out of the human
rights
Community I was initially building a
hate speech detector using um Bert and
Roberta uh bidirectional you know coding
Technologies to build a hate speech
model that led us
to basically seeing the runup to Jan 6
because we were monitor monitoring
Telegram and we were warning basically
anybody who would listen like the
chatter is different this time
something's happening it's much more
specific there's videos of people going
from Florida to DC right it's it's just
a it's a different quality to the
chatter um and they would really listen
to us but we took that
technology to combat what we saw as like
one of the biggest threats that's
happening right because
yes like climate change May Doom us all
um you know there's lots of other
problems happening in the world but if
we can't agree on truth and we can't
agree on you know um what's okay to say
um on these platforms I think that
really causes us um major problems for
how we move forward as a society right
because um it's really hard to have
conversations with your neighbors if you
can't even agree on basic facts so with
that um I'm going to reserve the rest of
my time for questions and uh just thank
you all for coming today and and
listening appreciate
it oh and if you have questions you can
just come to the
front if you don't I can
leave man nothing I said was
controversial at
all everybody just agrees with
everything a lot a lot of Reddit a lot
of
forchan um the 4channers are basically
saying stop jerking off the porn our
leader just been shot and then you know
the redditors are saying something along
lines of I can't believe that guy
missed yes
yeah um we're not campaigning for
censorship that's I think a a
misunderstanding of what we do first of
all um second I would say there are
First Amendment protected activities
certainly but a direct death threat
against somebody is not actually a first
member protected it's recognized by the
Supreme Court via Brandon BG that you
know you can't shout fire in a crowded
theater right that's incitement and so I
would say that like a lot of the stuff
that we see you know in terms of like I
want to kill Supreme Court Justice or I
want to kill Dr fouchy or I want to kill
Oprah that really doesn't have a place
to live online
shouldn't I think
we should agree that political violence
is probably not the right solution but
you're free to say that you don't agree
right that's first am
protected okay
I I know it's really late it's like 8:00
pm right check all right excellent so
you're collecting a lot of important
information um two parts like uh who are
you notifying like when you see major
threats like is that is that the product
that you're
offering does that make sense yeah um so
to be clear we
our company has software our clients
access to content and they're the ones
doing for example we work with a number
of different major anti-hate
organizations in the US and abroad and
they're the ones doing the like law
enforcement notification we're not the
ones doing it now we'll often be asked
hey are you seeing this right and then
I'll go log on to the platform and take
a look um and in case is where we do see
a direct threat like we saw one against
President Biden right before the
inauguration uh we basically I asked
friends if they knew somebody at the
Secret Service so I get that over to
them and U next is there a potential
that government will come and and demand
uh your data from
you that's a good question um I don't
think so we only collect open data so
anything that we have the platforms
themselves should have um we do
sometimes get asked by like a
Organization for example something got
deleted off of forchain if we have it um
because it's no longer accessible there
and we collect a lot of data from
forchan so uh you know we retain that
and are able to provide that so yeah
yeah and we you know again we don't work
directly with law enforcement like it's
just if you look go look at our website
or our terms of service things like that
AC acceptable use policy we're very very
strict about who we bring on as clients
um and it's absolutely prohib hibited
and we reserve the right to audit and
part of our contract like we can go and
look at other people's projects
basically um you can't monitor civil
rights organizations protected human
rights organizations we just don't allow
it
so um yeah my question mainly pertained
to mental health um having to dig
through some of the most like repugnant
and nefarious corners of the internet
obviously takes extreme toll on your
mental health like is that
something that you need to you know
doubly prioritize is there anything you
know routines or anything you can
recommend for those of us you know that
have to monitor some of that as part of
our jobs or
research yeah I mean we our last our
last company Retreat we did two hours
with Specialists that work with the
response teams at meta and places like
that where they're seeing really
terrible stuff as well um I don't know
if I got a ton out of that two hours to
be honest
uh it's also like the first time I'm
ever taking medication so like you know
and I mean you look at my bio I did two
combat tours in a rock so you know it's
not I'm not a stranger to stress but
it's it's a different level um I think
you know do everything to keep yourself
healthy get regular exercise eat right
like get sleep you got to do all those
things as a Baseline and then recognize
your own your own limits and what you're
comfortable with like it's one of the
things that we talked to everybody that
we hired into the company was like me
spending a good 15 minutes tell talking
to them and trying to convince them out
of joining the company and saying like
you're going to be encountering terrible
terrible stuff all the time um and I
mean one of the so so one of the things
that we we did uh in the company has
built a special fine-tune model to find
things like KKK hoods for example and I
spent a lot of time drawing bounding
boxes around KKK hoods to find who the
model and I didn't I didn't think that
would affect me deeply but when you you
know look through several hundred photos
of lynchings like it's just not it's not
great so and look that's why we have ai
right the AI can help you sift through
that um media in our platform you have
the option of blurring all of it right
so if you don't want to see any of it
you can just blur all of it um and the
Lesser you counter it I think the better
um it's just yes it exists um and and
it's it's the hidden part of the
internet that nobody acknowledges why
you don't regularly see child
pornography on Twitter or on Facebook
it's because there are humans reviewing
the AI detections and removing that from
those platforms right that stuff all
exists it's just being hidden from you
and there's human cost to it absolutely
for your work we appreciate you and H
thank you for the
talk it would be naive for you to
believe that your stuff won't be
eventually subpoenaed by law enforcement
and by private
actors I work at the internet archive
and we get requests all the
time and our people often have to go to
court not universally and vouch for the
accuracy or the uh Integrity of
documents that we
preserve thanks for that
do you believe that there's a chance
that by restricting um content like this
to these major platforms it might cause
people to go to even more drastic
lengths to get their hateful messages
out
there I'm sorry could you repeat that
question do you think that it is
possible uh
that by restricting access to these more
open platforms people might go to more
drastic lengths which would in turn
cause more damage in order to try and
get their hateful messages out
there I mean I think that's already
happening I think people who
are you know trying to kidnap the
governor of Michigan for example right
they were using
signal
um I don't I don't see a way to you know
stop people from using signal chat but I
think the question really is in order
for and look there's there's a ton of
Nefarious activity happening in dark web
as well right I think the question is
what happens when you have a bigger
microphone to broadcast your intentions
and get people to join in that's how you
get events like January 6 that's how you
get events like what happened in
Ottawa
um these sites that are popping up now a
lot of them came up because the folks
got basically kicked off of Twitter and
those places right like true social
exists because of you know some of the
enforcement actions taken by Twitter but
I think one of the things to recognize
is that true social is much smaller
platform you just get less engagement
and less ability to broadcast you know
terrible stuff
um
yeah so I I seem to understand that
the the main goal here is Public Safety
yes so uh uh so I was wondering in a
sense uh as a private actor that
develops models that then run
continuously um you take you take
responsibilities both like that in a
sense go in a in a mix from the from the
technical to the definition of the task
and to what kind of cases are are
covered so I was I my question is in a
sense what kind of of
responsibilities uh do you take do you
take uh well technically uh politically
and legally uh when you give your your
models your software to to other uh
institutions who use it yeah I mean I
think the first thing to recognize is
that the AI is not perfect and I don't
think any anybody expects the AI to be
perfect um you know right now if you go
and look at extremely violently
extremely violent posts related to the
Trump shooting right you can find a
bunch in in our platform right now um
some of them are going to be sarcastic
some of them are going to be um you know
basically false detections
right but you've necked down the
universe of things you have to look
through rather than just doing you know
a strict Boolean search right um and
you're able to sort by what the AI
thinks is extremely violent um yeah I
think clients are very realistic about
how the AI performs um and we are very
realistic about that too and I think
it's something I say all the time if
somebody comes around and says like oh
the AI can do everything like that's
just not true the best models out there
running um you know the latest and
greatest um Transformer
technology uh with you know immense
amount of fine-tuning happening even
that's only going to be I don't know
maybe 85% accurate
like that's that's a great model in my
mind um you know a first cut getting
something 68% 71% accurate like you're
pretty
happy um but when you're talking about
millions and millions of posts a day I
just don't see a way of looking through
all of that without the assistance of AI
you have to have something help sit
through
it it's not going to be
perfect well thanks