Hello, I'm Matt, sorry but I lack Portuguese. Then this talk will be in English.
As you listened to I am a graduate student at the University of Michigan.
I'm advised by Alex Halderman.
And I'm here to talk about our experience last year
with the United States Presidential Election 2016. On the Recount screen.
So to understand, let's refresh your memory a little bit.
Last year we had an election between these two candidates.
We had Donald Trump on the left and Hillary Clinton on the right.
And I assure you all now what happened.
Donald Trump won the election. But it was not always so obvious.
Going beyond the election, many people expected Hillary Clinton to win.
This is FiveThirtyEight which is a well-known Election Blog in the United States.
It aggregates and predicts elections and gives you an idea of ​​what they look like.
In the last two elections, 2008 and 2012 they designed almost all states correctly.
So they're pretty accurate.
As we can see here, they are giving a 71 percent chance of winning.
And they were not alone in it.
You have the New York Times, giving Hillary 85 percent.
You have the Princeton Election Consortium which is basically this Sam Wang guy.
That gave Hillary a 99 percent chance of winning.
And Sam was so confident, he said: If Trump wins more than 240 votes he would eat an insect.
So, in general the feeling was that Hillary was going to win this race.
As you can see, virtually every newspaper we know of was mentioning it.
Of course we know that was not all that happened.
What really happened was that. (Political specialist eats insect as promised)
What really surprises everyone including someone who swallowed an insect on national network.
Trump won!
So these are the results in fact, as you can see Donald Trump won the electoral votes.
In a second I explain what the electoral votes are. So do not worry about it.
But in addition, Hillary Clinton won in popular vote. She won.
The margin was 3 million more votes for Clinton than Trump.
Where we have this crazy system, it can happen that everyone votes
in one candidate but the other candidate still wins.
And this is a fierce race. Despite what Trump kept talking after being elected.
His electoral victory was not really the best.
In fact, he is the 46th ranked ranking of all elected presidents we have ever had.
So this was a tight race with unexpected results.
So what happened? Even if you have a systematically misspelled explanation,
That would be the ones that spin FiveThirtyEight.
And someone claimed that the score margins and counting scores were wrong.
Or the election system was hacked.
This seems to be the theory that Trump continued to speak to the world.
You know, the election was manipulated, the results of the election were not valid.
So, we know, if it is probably doubtful,
it seems reasonable that once or twice the counts were wrong.
So, what is happening?
What does Donald Trump claim is really doable?
Is at least possible?
And if so, how do we find out if we do?
So that's what we wanted to do. About a year or so ago.
To summarize the overview of the American elections and why all this is so crazy.
First we have a massive scale, here in Brazil it is a big country.
So you understand, and our elections are very confusing as in Brazil.
Then every state, county, small town they will decide how they are going to vote.
These elections are really complex. With long votes from many categories.
There is always the sense of waiting that we need to score on the night of the election.
And again some statistics for you.
Here we have 200 million registered votes.
187 thousand points of elections.
And 52 types of electronic ballot boxes.
Unlike Brazil that only has one, we have many types.
This is the US map, including technology using technology in the 2016 vote.
And you can see here, we have basically all the
possibilities of combination technology you can imagine.
We have in the Pacific Northwest the postal vote in Georgia and South Carolina
we have DRE and paper, the same that they use in Brazil.
In states like Michigan and New York it's just vote to paper. They use bar code.
And places like Nevada, Texas, Florida or Tennesse there are mixtures of these things.
Again, to show the complexity here.
This is a US state. And these are the different counties of that state.
And you can see that even counties do not agree on the technology they will use.
Some will use DRE, others will use paper, and others will post.
The point here is that the election technology is very diverse, very complex.
So I mentioned that our ballot is very complex.
This is a ballot from the city of Dallas, Texas last year.
And as you can see, it has a lot of text.
You have to read it all before you get down here.
Where you can vote for someone.
And you can imagine that this is exhausting for the voter.
But that's just page 1, and you can see all the competitors here.
And it has 8 pages like that, so it's incredibly complicated,
for anyone to vote for.
If the technology changed one of these pages or these candidates
the voter would not notice it.
As we mentioned, we have many types of electronic ballot boxes.
But they really fit into two types of categories.
You have the barcode, where you place your information on a ballot.
As it appears here. And you put it in a reader.
Sometimes it is counted manually.
And then we have the DRE, which is what you are accustomed to in Brazil.
And sometimes they produce paper registration, sometimes not.
This is an example of a DRE that is used in the United States. This is Diebold Accuvote-TS.
It began to be used in 2003 and is still used in the state of Georgia in the USA.
My mentor when he was a graduate student, ten years ago.
He picked up one of these urns and dismounted, and checked how vulnerable he was.
How safe was the system.
Then they dismounted the urn to open, looked at the motherboard,
you know, there are EEPROM chips and several other sources within that.
And what discovered that these machines were not safe at all.
They repeated the same encryption key, they invaded the entire urn.
The way I started and ran any program they put.
Then they realized that these urns are
these memory cards that are inserted.
So if you put a virus or something on the memory card.
And putting it in the urn, the urn becomes infected with the virus.
And what's more, you get another memory card that has not been infected.
And you connect to this urn, it also gets infected with the virus.
So you can spread a virus in many electronic ballot boxes.
All without being connected to the internet.
To give you an idea about this virus it seems.
Here you see, two choices on a ballot.
Where you have the obvious vote in red for George Washington upon.
And not for Benedict Arnold.
But what this virus does is fool the urn you voted for George Washington
But then at the end of the election you clearly see that Benedict Arnold won.
And this is a vote where all votes went to George Washington
This malware is so easy to build that you even have time to create a user interface.
It's not really difficult. As you can see here, it has a scale bar.
Where you can choose what percentage your candidate will win.
This shows how much these urns are unsafe.
But this is just an electronic ballot box. What about the other urns?
I said we used 52 and just talked about one.
Each of the electronic ballot boxes, which have already been
Studied were considered vulnerable.
You have the DREs of that type. The Accuvote TSX was the successor to the Accuvote TS.
You have the Sequoia AVC Edge. That's really easy to break that's for playing Pac-Man.
That you can see here.
You have these barcode readers. Which were also broken earlier.
So certainly none of our urns that have been studied are actually safe.
And like I said, each of those that have been seen are incredibly vulnerable.
So, can you hack an election?
We have these vulnerable electronic ballot boxes, but you know, we have several types.
Counties, cities and states have different technologies.
And it may not be feasible to steal votes when it has a consequence.
You usually have these 3 types of scenarios if you want
hack an election.
First being someone who changes the election results unnoticed, something that no one understands.
Or else that way you can undermine the credibility of the election.
Obviously when something goes wrong.
You can deny services, you can create long queues to vote, you can turn off machines,
you can turn off the electricity.
You can do many different things to interfere in the elections.
And you can also cause political interference.
If you are a formal government, you can create a
misinformation about one of the candidates to discredit him.
The scary thing about all of this is that we have seen this happen in reality in 2014 in Ukraine.
Russia has modified, altered the electoral result and also inactivated the counting system.
And you should know, in 2016, Russia was in a disinformation campaign against Hillary Clinton.
So these 3 hacker scenarios happened.
Some of them in the elections we are seeing now.
But let's say you want to change your vote.
How will you do that?
We have three major challenges here.
First as I mentioned, it's the different technologies, the machines are not connected to the internet.
And 70 percent of US citizens use some paper record.
In case something goes wrong with electronic ballot boxes and counting.
And check everything over again.
So they look like tough challenges. How do we overcome them?
Well looking at the first challenge, as I mentioned we have this crazy electoral system in the United States.
Called Electoral College.
And the way it works, in every state who wins the majority of votes wins the 'electoral vote'.
With a few exceptions, I think Nebraska has a bigger representation.
Then for information Donald Trump won in Texas.
He won all 38 electoral votes.
And the winner is declared by whoever manages to win 270 electoral votes first.
As you can see here in this count, Donald Trump had 360 electoral votes.
Which was enough to win.
But the thing about it, is that states have locks.
You do not need to change status to see the result.
When you see one of these two or three states here, suddenly you
has 40 electoral votes between Trump and Clinton or vice versa.
So you really only have to affect 1 percent of the votes.
And this is especially convenient when states have a tight margin.
I'll talk about the states later, and what they did.
But New Hampshire's margin was 0.3 percent.
So you just have to invert a few thousand votes there.
So the first challenge does not seem to be a big obstacle for a motivated opponent.
How about the fact that the polls are not connected to the internet?
As I mentioned, we already know that we can build a virus and
spread through electronic ballot boxes.
And the other thing to know is that when you program an electronic ballot box,
it is normal for them to use central computers like this.
Just in any office, and often that computer is connected to the internet.
So if you infect this computer, you can actually infect every
go on that computer and then on the electronic ballot boxes. And effectively the election itself.
But you know, this scenario is exactly as it would be. And it really is what it is.
In the US in many states we have many outsourced companies
this is government business solutions.
They want to program all electronic ballot boxes in a county, city or state.
For example in the state of Michigan, three quarters of the electronic ballot boxes
are programmed by one of the two of these small businesses.
Small means, they hire 20 people.
So, what do you want to do here.
Governmental Business Solution has a good website.
It does not have TLS-R, unfortunately. But there are pictures of their stock,
You can go in and hack into their system.
They have email from all employees so you can make a phishing attack.
So in principle it would not be so difficult to put a virus on the network of these companies.
And then spread by electronic ballot boxes,
to electronic ballot boxes that are not connected to the internet.
But what about this third challenge here?
This one seems more doubtful.
You can not exactly hack paper.
It is not that you will persuade the state to stop using paper if you are an opponent.
So, what you do? The idea behind the paper is to give you a physical note of something earlier.
And a kind of metaphor is that in airplanes they use physical compasses in case the equipment fails.
And that's the point of the paper, the paper ballots are good,
even if computers confuse ballots, you can still recount
where you can see the physical paper ballot, you can find out what the voters
really want it to happen.
But the problem with paper is that it is very slow,
and too expensive to work, need many people,
it takes a lot of time to count each ballot by hand,
But computers are pretty good at it.
But the good thing about the role as I mentioned, are checked by the voter.
The voter marks this ballot. The voter does not dribble a count on that memory card.
Or the voter has no way of knowing how they do it.
So paper cells are very expensive, but better in safety.
While the memory card are fast and cheap but are not reliable at all.
So focus here, you can see some papers to indicate if the value
that is on the memory card is actually correct.
And to give you an idea, as I mentioned 70 percent
of the United States has some form of paper registration of their votes.
And that number is currently growing in the state of Virginia.
Just because the government got rid of all paperless electronic ballot boxes.
So as you can see, many places have that role.
So if they used it correctly, their attack would be picked up immediately.
But unfortunately, no state looks at the paper.
Only a few states look at the election balance.
We basically get all this expensive paper, we cut all those trees.
We print all these bills, and they all go to waste.
Once the election is over.
So, sure, hacking a really much easier election
of what you thought.
Than we think. Than the people who come
working in those 20 years they think.
First, all you have to do is use the previous elections
to find some states with similar margins.
So you look at one of these ballot program companies in these states.
And it spreads a virus to the electronic ballot boxes.
This virus is placed in electronic voting machines, and dribbles a small number of votes
which will not be detectable.
And states throw out the only way to detect the attacks.
Easy right?
So, knowing that. That was what we had to say.
We knew that our electoral system was incredibly vulnerable.
We knew that the hacker scenario was plausible
that we had already seen other opposing nations active in that election.
So what we wanted to do was some kind of auditing in the 2016 election.
But how would we do that?
In the United States we have five weeks after the election
so that the results can be confirmed at the electoral college.
So you only have 5 weeks window if you want to change anything.
As I mentioned the presidential race was extremely fierce
and with a challenging score.
We also saw Russia acting in bad faith in these elections.
And we have already shown that it is feasible to infect
enough to affect the consequences.
But even under these circumstances, no state was willing
to take a second look at your votes,
and check if what the electronic ballots said was correct.
As I mentioned, we only have five weeks as a limited time.
And you know, the two questions we started to ask were:
Is this possible?
Or at least we can prove this consequence
with the evidence we obtained in such a short time?
Alex and myself, and other electoral intellectuals gathered
and we try to think about it.
And I watch over the next slides.
Which are several photos of my advisor. Originally this conversation was
nice conversation with him.
So if you look a lot at his face, that's why.
So what happened in the result?
So one week after the election, Alex and I went to a conference
in Los Angeles, we were flying back to Michigan.
And Alex got that call on the plane.
And honest voters say.
What is the evidence of fraud?
How do we pay for the recount?
Sorry, I should mention the call was from Hillary Clinton's campaign.
The Clinton campaign was trying to figure out how to
they really missed that election.
Was it the evidence of fraud?
How was it? How do we detect it? Can we pay for recounting?
The scary part of the Clinton campaign
is that she considered herself victorious. She had already lost.
Since saying I missed it.
Donald Trump is the president.
So if she starts asking for a recount, that's bad.
So to hammer on that point, in the debates during the election,
Hillary ... there was this moment where the moderator asked Donald Trump
if you lost the election, would you accept the results?
And Donald Trump did not answer. He just nodded and kept going.
Hillary Clinton in response said: He is indirectly saying he would not admit it.
He said something horrible ... a direct threat to our democracy.
So Hillary Clinton was hopeless and could not ask for a recount
after this exchange and say that no one can see that the
election is a threat to America's democracy.
So the Clinton campaign basically continued.
We feel a little hopeful here, because the only way
that we could see these roles would be by a recount.
And the only way to get a recount was
using some candidate who was in the presidential race.
Then a week later, Alex was sitting in his office,
and another connection. And it's Jill Stein.
Jill Stein, is the Green Party presidential candidate in the United States.
The Green Party is a small party.
Pretty small, it only got 1 percent of the vote in 2016.
But they have the history of being the most upright in the election.
So it was very good, it was on the ballots of most states.
Especially states that we care about.
Giving her a record of election integrity.
Then it is possible that he will collect the ballots and recount them.
And so it was done.
Unfortunately mixed in all this.
Someone from the Clinton campaign told the press that we were
working trying to prove that something was wrong.
Unfortunately according to Alex, they were saying
that there is persuasive evidence that the election was hacked.
And up to that point we did not know anything. We only knew what we wanted to see.
This basically ruined our holiday.
We had to write a response until five in the morning.
Here in Medium, I still think it's open and you can read it if you want.
But basically we were claiming that
we did not know whether the election had been hacked or not.
But the only way to say yes or no is if we look at the final balance sheet on paper.
And at that point, it became international news. It was a big deal.
And finally the Green Party turned into solution.
What they were going to do was raise funds
and try to find states that accept money and organize.
Like lawyers and volunteers to recount the ballots.
I should mention that Jill Stein raised $ 5 million in two days.
Jill Stein never raised $ 5 million in her entire political career.
So that was a big deal.
We had 160,000 donors in the United States.
Requesting the start of a security audit.
This was really supported.
So what did we do?
We had that money, we had the candidate,
we thought we could continue with this.
The next question is, what votes will we retell?
It seems a bit difficult to recount the whole country. Since there are 2 million valid votes.
Maybe it can be done in some states.
And in the states we decided to go to, they were Wisconsin, Michigan, and Pennsylvania.
The reason for this, all three had projected Clinton to be victorious.
And in fact I believe the three are democratic states for more than 40 years.
Together they have 46 electoral votes, which is enough to change the victory.
And all three ... you must forgive my slides, it was done on GoogleDrive.
You are running in PowerPoint.
All three have margins less than 0.8 percent.
So they were all very fierce elections.
Altogether decided by just over 78,000 votes in a few million.
In addition, Michigan and Wisconsin allow applicants to apply for recount
In Pennsylvania, unfortunately, he needs a court order.
It's a bit more complicated. You need three inhabitants with your records and pay a fee.
I'll talk a little bit more about it.
So looking at the state of Wisconsin they have 10 electoral votes.
The margin there is 22 thousand, almost 23 thousand votes.
The technology was almost all barcode.
So all the votes had a paper record.
In any candidate can request the count if he pays the cost.
Luckily we have a candidate who now has enough money to pay for it.
The bad part of this is that every county in Wisconsin decides how they recount.
Since they call volunteers to recount each record by hand.
Or maybe they can pick up the records and put them back on the machines.
And try to find out whether they were hacked or not.
The state of Michigan was even tougher.
It was one of the hottest states in 2016.
A margin of only 11,000 votes. 0.23 percent.
Their technology was totally bar code.
Then all voters had a paper register marked by the voter himself.
And there is a law in Michigan that agrees that any 'offended'
And that has a vague meaning, but let's talk about it later.
Who pays the state fee can ask for the recount. Also in Wisconsin.
And Michigan is also home to the University of Michigan. Where I live.
In the State of Michigan the state board of solicitors,
which is how the governing body that determines the election works.
That decides how the recount will be done.
And fortunately in Michigan they decided to make a full recount.
They were going to put all the records back in the electronic ballot boxes.
And finally in Pennsylvania, again another fierce edge. 0.72 percent.
And unfortunately in Pennsylvania they use almost exclusively paperless DRE ballot boxes.
Like the ones you use here in Brazil. Only 30 percent of the votes are on paper.
And the Pennsylvania counting law is a little crazy.
It takes 3 inhabitants of each precinct, to pay a 500 dollar fee
and swear it was a fraud. This in every precinct.
To request a recount in the state of Pennsylvania
we're going to have to find something around 9,000 people to swear
that there was electoral fraud and pay to be made a recount.
That seems a little unlikely.
And unfortunately the state has a recount
automatic if the margin is less than 0.5 percent.
It's automatic, it counts at the state level.
But as we can see, we have almost reached that margin.
We choose these three states.
But how did President Trump and other people notice? Why do not we choose others?
Well, here it is focused on Virginia, New Hampshire and California.
And indeed we have other theories about the state of New Hampshire.
The state of New Hampshire the deadline happens 6 days after the election.
So do not give time, we had no idea
of what we were doing two weeks after the election.
In the state of Virginia it is illegal to recount if the margin is less than 1 percent.
And Clinton won in Virginia with more than 5 percent.
Which is also a crazy law.
You can not see the paper unless the percentage is next.
And in the state of California, the margin was 38 percent. So it's quite unlikely.
And in California historically everyone votes 'blue'.
So there are not many questions about the results there.
I should also mention that there was recount of votes in the state of Nevada.
And it was very inconclusive. It's something separate from that.
But with all effort was recounted.
So we have our states, we have our money, we have our candidate.
Let's start the recount.
So in all states we requested these petitions, Jill Stein requested all we had.
Basically saying: Hey, we want to recount, here's the money. Do it!
And unfortunately this angered the elected President Trump.
Claiming it was a blow to fill the pockets of Green Party.
Which was not really the point.
As you can see, we spent five weeks to accomplish this.
We were running out of time.
I should mention that the campaign was really possible.
thanks to Stein's campaign and the Green Party's 10,000 volunteers.
This incredible advocacy Emery Celli Brinckerhoff and Abady.
He organized lawyers all over the United States.
To request petitions, and check the recount and organize people.
Also many e-voters experts in the United States.
They also helped to require and support efforts.
So even if it's me speaking
I'm just a small part of it.
How does a recount look?
Well, it looks like that.
That's in Washtenaw County in Michigan.
Which is where I live, and I'm there in this photo.
In fact the way a recount works, you have a large heap of ballots.
You have two workers with the ballots. And they get about 100 banknotes
and they count in unison to make sure they do not mix.
Or forget a vow.
How many votes are there?
Well, this is happening. Other people are sitting at the table, they are party volunteers.
I was there for Green Party.
This gentleman here was in favor of Trump's campaign.
To make sure these two ladies here are doing exactly what we expect.
So that there is no ambiguity in the recount.
The recount also looks like this.
When we started recounting, Donald Trump sued us in every state we tried to recount.
Basically restricting or claiming
depending on state law this is not legal.
You can not do that.
So we had a lot of lawsuits because of that.
And it also looks like that. In response to Trump's lawsuits
and in response to Michigan residents the people protested.
So Lansing is the capital of Michigan, as you can see there's a big protest here.
They wanted the revised result.
And we have that too. These are members of the US press. They were there covering everything.
Minute by minute. As you can see it was not always fun.
Every now and then they had to stay in the corner.
And it looked like that. This is a press conference in Philadelphia.
You can see Jill Stein in the middle talking to these reporters.
You can see Alex back here. And that was before they finished
to argue with the court, I believe.
And of course this has become newspaper cover story in all these states.
Here's Alex and Jill a few hours later.
Trying to enjoy this incredible moment of stress.
Well the election ... in fact we did not have that time limit of five weeks.
And then the election is set, Donald Trump is declared victorious.
As everyone knows now.
So you mean it's all done? More or less.
The Wisconsin recount was finally over, they recounted every ID.
Not all counted by hand but they all told.
The state of Michigan, I mentioned earlier that there was cause for 'offense'.
Well, Donald Trump lost in the state of Michigan and at the same time claimed
that Jill Stein was not 'offended' because she had only 1 percent of votes.
So unfortunately the Michigan recount was cut in half.
And the recount in Pennsylvania has never really started. Just a formal recap.
In Wisconsin, more than half of the ballots were counted by hand.
Even 51 counties ... Yes 51 counties counted by hand.
11 thousand 883 different votes. I want to remind you that the margin
victory in Wisconsin was only 22,000 votes.
In principle this is enough to dribble the margin.
That's almost 12,000 people whose votes did not count for the first time.
Because the electronic ballot box was so bad to register your votes.
Even so, the exchange of votes over the
was not enough, something around 400 votes.
Just the exchange of candidates.
And we can not say that there is evidence of attack.
We can say that the machines were old. But only that.
And in the Michigan recount I mentioned that it was paralyzed after 3 days.
10 counties in Michigan were finalized.
12 started but did not finish in the total of 83.
Almost 2.1 million banknotes were recounted. That's 43 percent.
And just a change in the network, 200 votes. And again no evidence of attack.
In Pennsylvania we can say that one or two counties began recounting.
In just a few of their precincts. And we've never seen anything published about it.
So we are not clear what happened.
Our lesson here about Pennsylvania is that it is very difficult to do a recount.
Even if nothing has changed, in terms of votes.
We have discovered several interesting things.
Then in Detroit, Michigan.
Originally they found that 37% of the precincts had too many votes.
They had more votes than voters. That does not seem right.
And during the recount they discovered that half of the precincts in Detroit were ineligible for recounting.
And that's because in the state of Michigan there is this law that says
the course of his record, the seal behind,
to the minutes that the clerk has to write everything that happens.
And if any of these have been lost, it's broken, or anything else.
You can not recount. You have to accept the original results.
Which is crazy. Because if you want to hack an election,
you can hack and destroy all evidence and it will be illegal
in Michigan find out what really happened.
So we have all this evidence now. But what will we do with it?
What I want is called Statistical Risk-Limiting Audit
And basically you count a small number of bills and
compares with electronic ballot boxes and you have an idea of ​​how accurate the election was.
And depending on the margin, and the exclamation you chose
you continue to count until you are satisfied with the results.
Or you count all the bills. And you can know what happened.
The idea here is ... the key here is sample alternation of seals.
Unfortunately because this recount was not random.
Because in Wisconsin they used the technology to count.
We had no randomness.
So unfortunately we have this sample incomplete and not random.
What really should update the samples.
We now have seal level information.
But what can we do with it?
So to make sure of the attack scenarios.
That I may be looking for this evidence.
Three types of scenarios, where the attack chokes the vote.
Where they prevent people from voting depending on the candidate they are voting for.
They compromise election ballot sellers like GVS that I spoke to earlier.
Or they individually compromise electronic ballot boxes.
Then the malware goes to the polling place and infects all those
urns with the virus or just an electronic urn.
So if the alternation ... sorry.
If the recount of the alternate counties is done by hand,
so in theory we can detect all this.
We got to see a lot of votes. We can see several balance sheets
on paper that does not correspond to electronic records. Something of that nature.
That's what we're looking for here.
And again I want to talk, we knew that I had an opponent to the presidency in that election.
The US government, the congress and the president had already said they were investigating this.
The intelligence community in the United States had also
'Hey, Russia is trying to elect Donald Trump.'
So with that in mind ... finally we find it much later
that really they actually hacked one of the electronic ballot companies.
One called VR Systems. And it is suspect that this has caused
service errors in the state of North Carolina.
But he did not show anything else either.
Then we know that we were hacked.
And we were trying to figure out how much it had affected and the consequences.
So me and Walter Mebane, he's a political scientist.
and a professor of statistics at the University of Michigan.
And I had this information trying to figure out some way to make sense.
And then we posted a 'Blogpost' in the Washington Post Monkey Cage Blog,
where you can read more interesting things.
And we had the paper on subjection as well.
What is our plan here?
We want to focus on the differences in the signals between the candidates.
Enter the original count against the total recount of each candidate who has changed.
You know, whether they have won or lost, or if they have remained the same.
We want to contrast the differences not shown with those shown.
We want to see the candidates who lost votes with the votes
lost Trump or the lost Clinton votes.
And we want to see if this sampling impacts change.
What we used was multinomial regression to correlate factors.
Like sellers, specific electronic ballot boxes, the candidate that was chosen and other things.
And the goal here was to try to see that the reported samples and the uninformed samples are not the same.
So you have many differences not shown. Candidate A, Candidate B gained lost votes.
And you have many unseen differences that say Clinton and Trump have won many lost votes.
If there were no differences between these two candidates in theory there was no hack.
Or at least the hack was inefficient.
So looking at the results here. In the state of Wisconsin we found nothing significant correlated.
There were no significant differences between hand and machine recounts.
In the state that is good to know that hired the machines that recounted could also be hacked.
Overall we found no evidence that fraud occurred.
Or if there was such fraud did not influence the consequences.
It can not be said with certainty, because of the electronic ballot boxes.
We saw that an electronic ballot box had high error rates. More than others.
And it's the Optech IIIP Eagle.
And in response to this the state of Wisconsin withdrew this urn.
So we do not use her anymore. Which leaves us relieved.
In the state of Michigan, again nothing significant correlative.
We found no evidence of fraud affecting the consequence.
We saw significant problems in Detroit.
In response to this the state of Michigan decided to play out
all electronic ballot boxes and bought new ballot boxes.
So we had some conclusions about all of this.
To conclude here, hack the presidential elections
is currently very easy.
It looks like it happened in 2016 but it does not mean it will happen again.
We continue to use these vulnerable electronic ballot boxes
which has this central point of attack.
States are beginning to worry about these messages.
'Hey, you really should see the papers.'
The state of Rhode Island for example incorporated the law.
But in general most states will not look.
And even if you get far enough with the role, there are still
many legal obstacles along the way.
For example the 'offense' clause in the Michigan Constitution.
Or the Pennsylvania mechanism to ensure its recount.
Or stop at the time to see the paper.
This is bad. We have seen an adversary nation.
Many intelligence officials from the United States have said that Russia will return.
And there's no reason to think they elected Trump and left.
So we need to fix this in the United States.
We need to renew our equipment technology.
We need to stop using equipment with bad encryption security.
Open in SSL version 1.1 for example. With shared keys at every urn that are in use.
And we need to make sure an attack is detectable.
We need paper records.
We need something we can use to check the urns.
And there are still many states that do not have that.
Or at least it does not have it everywhere.
And finally we need to use the evidence.
We have to make these limitations of risks or recounts depending on the law.
If we are going to do risk auditing it is much cheaper than the total recount.
In states with large margins such as in Alabama for example in 2016,
you can retell seventy bills to confirm the result.
And what we can do, but it can not be done because they have no paper
or the law does not allow doing.
These reforms can be used in 2018. When it's our half election, almost a year from now.
We just have to act.
So I want to thank everyone who listened.
And I'll be happy to answer questions.