[SOUND] Now we're going to continue our interview with Lorrie Cranor and talk about the phishing paper and other general lessons in usable security. >> Yeah so, so that paper. [LAUGH], that's an oldy but a goody. [LAUGH] Got great a while ago. I, I think by the time we got to doing that study I think we had a pretty good idea that that we were going to see a lot of people falling for these attacks and we had a good idea that Internet Explorer was particularly bad. and, and the, the data bore that out. I think the surprising thing for us was why people were falling for things. And I think you know, one of, one of the most surprising points which I think we described in the paper was the anecdote where we saw people who would get the warning in the browser, close the browser, go back to their email, click on it again. Again. Go back to the browser, close the browser and repeat this like eight time. And clearly they had a mental model that wasn't matching our mental model. because this made no sense to us. And then when we asked them about it and it was more than one person who did this, you know, they, they basically it became clear that, that the, the browser warning was informing them that there was something wrong with the website. And in their mind that had nothing to do with the email. The, the email was fine, you know there was just something wrong with the website so obviously if you go back to the email and click again, maybe the problem will clear up. and, and that they really had no idea of what, what the browser warning was really trying to communicate to them. Yeah, well I, I, I think I mean, it does speak to the importance of observing users and doing user studies. And, you know, realizing that users don't think the way you do. [LAUGH] You know, you're not your users and, and so I think every time we do a study like this, we do have these surprising things that if we'd only talked to ourselves we would have never, have figured out. And, and it really is important to, to talk to a wider group and not just people like you. So, so I think to improve the usability and security of systems it's, it's really important to think systematically about how users are going to interact with the systems. And I think you want to go through from the beginning, what is the first thing you're asking your user to do? When do they do it? How do they do it? You know, walk through the whole process write down all the steps. And I think when you do that you'll start to see in many systems, wow there are a lot of steps here. And I think then you can start looking at those steps and saying well, are, are these all steps that are really necessary to have the end user do, you know is there stuff that we can automate and just have the system do automatically and not ask the user to do it. And which are the things that, yes it's really important to have a live human being doing. And can we make those as pain free as possible and as clear as possible so that the human is likely to actually, do them properly and maintain the security of the system.