The last set of security procedures I want to talk about today have to do with ways of testing voting machines on Election Day during the polling place, in the polling place. Can we and can election officials accurately detect if there is some kind of fraud going on in the machines. So there's several elements of these kind of election day tests that are usually written into election procedures. Some of them are basically legacy items that come from the days of mechanical voting machines and even paper ballots. One of these is the idea of the zero tape. So at the beginning of polling before any votes are cast, a DRE voting machine will usually be made to print a tape indicating that there are zero votes in it to begin with. Now this could be helpful to detect procedural error if the officials somehow forgot to reset the machine from the last election. But it's not going to be very good at preventing fraudulent software, vote-stealing software in the machine from operating. Because the machine can just print whatever it wants. Another kind of testing that we can have some more to say about is so called logic and accuracy testing. Now this is an idea that comes from the days of mechanical lever machines, and some of the procedures that were written for those machines still survive in the era of DREs. Logic and accuracy testing involves allowing people at the, at the polling place, poll workers, or even candidates' representatives, to cast a number of votes on the machine before real voting starts. So they'll have a miniature mock election at the beginning. Now on a lever machine, this is a really good idea. You wanna make sure that all those gear mechanisms work, that they actually add a vote every time someone casts one for each candidate, that they don't add more than one. But on a DRE like the zero tape this can have almost, almost no effect in catching actual vote stealing software. Vote stealing software would just recognize that this is the logic and accuracy test, cuz it's a separate mode the machines are put into. And it would just be programmed, the sensible thing for any attacker, thinking like an attacker, is to program your fraudulent software not to cheat during the logic and accuracy test. So at best this could, could catch programming glitches. It could cast errors in the ballot design, but it's not going to catch genuine fraud. A third kind of test that is something that's been proposed but is not used everywhere yet, is an idea called parallel testing. And the idea here is that on election day we're going to take aside some machines and conduct an election on them that's just not going to count. So we're going to maybe randomly select a machine from a few precincts and on those machines we're going to have some volunteers vote according to a script and then at the end of the election day we'll print out the totals from those machines and make sure that it matches the script. So before I talk about whether this is a useful security procedure I want to give you a chance to think about it yourself. If you were programming vote stealing software on machines that might potentially undergo this sort of parallel testing, how would you still make it cheat? What could go wrong? So there's several ways that vote stealing software in a DRE might try to avoid being caught in parallel testing. Even if the machine, some machines are taken aside and carefully watched, where you know what the outcome should be after people vote following a script. There are ways that vote stealing software could avoid being detected. One way is that the vote-stealing software could just try to differentiate between a real election and a scripted election. Probably the simplest way to do that is to look at the total number of votes. In a simplistic parallel testing regime, maybe you just have a few votes, 100 votes. Whereas in a real election, the machine might be counting many hundreds of votes on Election Day. So the vote-stealing software could just be programmed only to cheat after enough votes were cast. A more sophisticated parallel testing regime would have a script that had enough votes that it would be similar to a realistic election. But sophisticated vote-stealing software might still try to detect that it was a scripted action and not a real one. Some things it could look for would be how long each voter took. People who are following a script are going to get very quick at casting the ballots. Whereas real voters are going to sometimes take shorter time and sometimes longer. Another thing that the vote stealing software could look for is errors that voters make. Real voters are going to make errors, go back and correct them. Perhaps people following a script are going to do it correctly with a higher probability. So any kind of behavior that separates rescripted voting from real voting is something vote stealing software can pick up on and decide not to cheat if it thinks it's being tested. Another kind of way that the vote stealing software could defeat parallel testing is by using something called a secret knock. This is a signal that someone operating the machine, say a user can send to the software by some sequence of action, say, touching a particular place on the screen or even just voting a predetermine series of choices. There are many ways to imagine making a secret knock on the machine but there, there were couple of ways this kind of triggering action could change its behaviour. You might program your vote stealing software so that it only starts to cheat if it receives a secret knock. So then you just have to have one person one conspirator go and vote on each machine in order to enable the fraudulent process. Another way is if you have conspirators who are some of the testers there could be a secret knock to tell the machine not to cheat. So this is potentially a more risky strategy for the attacker but it's one that if you know your conspirators are going to be there it would work. Another question about parallel testing though that's probably a trickier one is what do you do if problems are found? It's already election day. The votes have already been recorded on the DREs. If there's no paper record, but you think that some of them might have been tampered with, what can you do? Those votes are already potentially lost and at least not able to be trusted. So parallel testing is has its limitations. It's possible that it would got, it would catch certain simplistic forms of frauds, certain, poorly designed, attacks that didn't anticipated. But it's not without, but it's not going to be any sort of guarantee that the voting machines are, are behaving honestly. So we've talked about a lot of election procedures today that are important for security and if these procedures aren't followed correctly and if they're not followed exactly, that's going to create a possibility for, for fraud to be committed or a potential for votes to be corrupted or lost. But you have to remember that the people who are charged with following these sometimes very complicated procedures are election workers who are, are often volunteers or often given just a basic amount of training and who only have to carry out these complicated instructions a couple of times a year at most, on election days. We know that people aren't great at following complex instructions to the letter. So what are the odds that election officials and election poll workers are going to make some mistakes? It seems like it's just human nature. We've had some evidence of the rate of error and the I can give you just a few data point on that. It comes from a recent study, in which some computer scientists looked at the electronic logs maintained by DRE voting machines in, fourteen counties from South Carolina. And these logs noted different kinds of events and they gave the researchers some insight into the rate of procedural error. So, poll workers made quite a few mistakes across these counties. About two percent of the machines from seven of those counties the researchers found had never had their votes uploaded and so the votes weren't part of the final count. People just forgot a crucial step in tallying the votes. In some machines across seven of those counties, there were other important errors. The touchscreen displays weren't properly calibrated. In eleven of those counties there were machines that had been opened and then closed in an incorrect way. This is another possibility that introduces potential security implications. So the picture isn't very good. Some fraction of machines are going to have procedural errors committed and the question is can we come up with sets of procedures, sets of cross checks that are going to keep these rates to a minimum because it's only human nature that some errors will occur. When problems do occur whether they're the result of human error or just potentially suspicious anomalies like a few broken seals, a few inconsistencies in review screens or VD pad tapes, the question is will those problems be recorded, will they be? Reported, will they be reviewed? Will anyone notice if there's a pattern across a larger jurisdiction or state? This is one of the last procedural component I want to mention but it's a critically important one that there be some mechanisms in place for reporting and explaining any kind of pattern of error that's happening. First this is the only way that we'll be able to correct problems with the procedures to debug that process requires some kind of widespread noticing and correlating of problems. Second though problems that are low probability but widespread are the best indicator that something's wrong with the technology. If that, that might be the only way that we have to observe that there's some bug or even some attempt at fraud happening below the surface. If someone's building a piece of vote stealing software. For instance, a virus that's spread to the DREs across the state. They might have tested it to the best of their ability, but not gotten it completely 100 percent right. Maybe on a few machines, it's going to leave some kind of, it's gonna create some kind of glitch, some kind of observable problem. But the issue is, the problem is if no one is reporting, recording and correlating reports of problems the chances are slim that anyone's ever going to notice that something is wrong. This kind of vigilance is the price of security especially with electronic voting machines. And so it's critically important that election procedures have error tracking and error reporting as part of, of the electoral process.