Well, I got to say it's great to be here in the InfoSec skills learning path. We are in the 2021 OWASP Top Ten security risk list and we have made it all the way to video or to security risks number 9, I should say, it's actually video number 10 because we had an overview video. If you remember that if you've been following along this whole time. But this is a security risk number 9, Security Logging and Monitoring Failures. This one's going to be all about, hey, do you log your activities, and if you log them, do you look at the logs? That's what this is all about. Hey, you know what? Let's jump right into it. Logging and monitoring. Security logging, and monitoring it came from the survey. If you remember back to the overview video that I talked about, where the OWASP organization came up with this top 10 list, 8 of the 10 risks in the list came from the community provided data or the application data that was submitted from all these different organizations, over 500,000 applications and all that stuff. But then the OWASP organization said hey, we understand that the data and the tools to be able to provide that data are not always completely up-to-date, and so there could be some risks that are out there that we just can't measure quite yet. We wanted to open up 2 of the 10 risks to a community survey. This is one of the security monitoring and logging failures is one of the two that came from that survey. Having said that, there's not a lot of data for this category, you can see down there that bottom bullet. There's not much CVE/CVSS scoring data for this category, but detecting and responding to these breaches is a critical thing and that's what all of the security practitioners, all of the respondents from the survey, we're saying is it's like, hey, this is still a big deal. Not only is it a big deal, it's actually up at a higher rank on the list and it was in the 2017 list. It was number 10 in the 2017 list, now it's number 9, so it's making its way up the list. You can see there in that middle bullet too. This topic, this risk itself is challenging to test. It's not like you can run an automated script and determine if someone has looked at their logs or not. I guess you could run a script and see if any kind of logging is being done at all, but do the people come back and look at it? I don't know. You have to talk to people. It involves interviews. But anyway, that's the nature of how this risk made it onto the list. Looking at the factors, there were four CWEs mapped to this particular security risk. You can see there, and frankly, you would expect this a bit to see some lower numbers as we're getting lower down on the list of the OWASP Top ten. But the incident rate there, the max incidents and the average incidence rate was 19 and six respectively. Those are the applications that were vulnerable to those CWEs. Then the average weighted exploit as 6.87. That's actually a pretty big number. The exploitability of this risk is pretty high and the impact is right there at just under five. Then you can see the coverage. These are the applications that were tested against those particular CWEs 53,615 occurrences out of just over 500,000 applications tested. That's not a huge number, but still it's a big deal. Then 242 CVEs mapped back to those CWEs that were listed as part of this particular risk on the top 10. There are vulnerabilities out there, maybe not as many as we've seen in other items on this list, but they're still out there 242 of them. Let's look at a few statistics, a few points of data, if you will, on data breach analysis. The reason that we're talking about this is, whenever you don't monitor your application for security events, like an incorrect login or there's been a massive number of logins over a short period of time. Maybe you're being attacked via some botnet or something like that. Or there was data that was exfiltrated and if you have some alerting that can happen on that, then you need to monitor that. You need to log that, but then you also need to monitor that. What happens here with this data breach analysis that we're talking about here on this slide is, there's a report that's conducted by the Ponemon Institute out of IBM and they did on the 2021, Cost of a Data Breach Reports. You can go out there and look at that, it's a fascinating report, it's a big report. You can just highlight with the executive summary, or you can read the whole thing if you want to, but it's a fascinating read. But anyway, the cost of a data breach in 2021 was 4.24 million and that's 10 percent up from the 2019 list that they did, which was 3.86 million. When you're talking about data breaches, people hear that all the time like, hey, so-and-so organization just had a massive data leak or whatever and the question is, how do you determine the cost of that, or what is the cost of that, frankly? The cost can be calculated in a number of different ways, whether it's actual money that was stolen, or maybe it's intellectual property, or some proprietary data, that thing that has been stolen, or maybe the and, or frankly, a lot of these can add on, so it's like all of this together. The organization that had the data leak may have to go out and pay a variety of maybe settle lawsuits, or maybe they have to pay for some credit reporting capability for all of their leaked data, for all the customers whose data was leaked. All of that stuff, that all of these things play a part, and then certainly probably not even included in that is the loss of potential revenue that could happen had you not had the data leak, this data breach. But suffice it to say the data breach issue is a big problem in the world today, and it's highlighted here in this report. You can see there on the second one, health care organizations. They're the highest average cost of a data breach and that's been true for 11 years. Health care is a big target for attackers, specifically in the area of data exfiltration, data breach. That's a fascinating thing and I will not going into that specifically in this video, but just to look across all the different verticals of health care, or technology, or education, or government, or whatever the industry would be. There are certain industries that get attacked in different ways from these attackers. Health care is attacked and they're the highest average cost for the data breach type attacks. If you're in the health care organization, you need to pay close attention to data exfiltration, data breach attacks, which in the context of this video specifically, you need to really be logging and monitoring all of your security events in your applications. You can see there the average cost was 1.07 million higher in breaches where remote work was a factor. That's an interesting thing. Now that we're in this coronavirus, this whole pandemic, or maybe things are starting to loosen up, whatever, but this whole remote work is a new reality for everyone in the world today. The breaches where remote work was a factor cost $1.07 million more compared to those where remote work was not a factor. This gets into the whole, hey how do you secure the remote access from an employee that's sitting in at their house, let's say trying to reach back into the network, into business critical applications that are hosted a variety of places? Maybe they're on prem, maybe they're in the Cloud, but you don't have those employees sitting in an office environment where you can really lock down networks and all that. They're sitting into their house on their cable modem, or on their DSL, maybe they got fired or whatever it is. Maybe it's a 5GE or LTE cell connection that they're dialing into. Those are not company provided, company locked down network, so how do you reach all the way back out to those employees and still provide secure access, that secure capability. Then like we've said, the numbers don't lie here. When remote work is a factor, the average cost of a breach is much higher. Then that last one, the time to identify and contain a breach, 287 days, which is up. It's not up huge, but it's still up from the 2019 report that they did, which is still crazy,287 days, there's 365 days in a year, so you're just under 80 days short of an entire year. This is, again, time to identify and then contain the breach. It's not necessarily how long does it take to find it. But find it and contain it. Although frankly the how long does it take to find it is a very long time. This is at the heart of people are not logging their events, and they're not monitoring. I was an Air Force officer and did communication stuff and I've worked for a variety of different consulting firms for the Air Force and Department of Defense, and we would see a lot of stuff and a lot of those systems where the problems would be happening for months and months and just no one bothered to look at it or there wasn't a monitoring capability to look at it, and so then that obviously that's a big problem. So the thing that always struck me is it's like, hey, the evidence is right there. I mean, you can look at these events that are happening right there. It's just no one looked at them, so it was crazy. It's a little frustrating sometimes. But that's the nature of this security risk. Let's look here at a few tools. I just went on and did some different searches around the Internet and I just pulled this off. You can see the source down their tek-tools.com, best logging and monitoring tool packages. Now ironically, you see number one up there's SolarWinds, and the SolarWinds we talked about that in the last video where they were the victim of this big attack and they had software that was pushed out to different customers and all that. Now, having said that, SolarWinds has done a lot of work to clean that up and try to be as secure as they possibly can, and again, no one is completely immune to an attack. But nonetheless, in terms of capability and what feature set. Does this tool gives you to be able to look at your network, look at your applications, and see what's going on? SolarWinds PaperTrail is a really good one, but you can see the rest of them. These, frankly I have not used all of these, but you can just go out and look at a variety of different logging and monitoring tools that you may want to check out for your specific application. Anyway, I just thought it'd be good to put a list up there, and again, I just found this off the tek-tools.com website there. Anyway, that's just a bit of information. Here's a scenario this is taken off the OWASP Website, and they just said, hey, here's this is a fictitious scenario that could happen if you will, but it's very believable, it's very realistic. Let's say that a major health plan provider's website operator couldn't detect a breach due to lack of monitoring logging. Here you're the victim of an attack and you're like man, things are slow or maybe things are not slow because not every attack slows everything down. But nonetheless, you're the victim of attack, you couldn't detect that because you're not monitoring, you're not looking at the logs, you're not logging, so there's nothing to look at frankly. Then here's an external party informs the health plan provider that an attacker had accessed modified thousands of records of more than 33.5 million patients. Then you go through this whole postmortem review and let's you're interviewing people and all that stuff. Website developers had not addressed some significant vulnerabilities. This is keeping your systems up to date and patched and all that. Then it says there was no logging or monitoring of the system. The data breach could have been in progress for many years, and that is a very realistic scenario of something that could happen. The point there there's a couple of things to highlight, I guess one is you need to log the security events. You need to monitor those, and then you need to update your vulnerabilities as much as you possibly can and keep your software up to date. The other interesting thing here, and this happens as well. It says here, an external party is the one that informed the health plan provider that an attacker had access to all this stuff,. So you don't want some third-party, some security researcher that's not even part of your organization to knock on your door, proverbially, knock on your door and say, hey, we've noticed that you guys had this massive problem. So it would be much better to catch it yourself and start to fix things yourself. Then have someone down the street come in and say, hey, you've got a big problem because who knows who they've told or whatever, so it's not a good situation to be in. That's a scenario based around this particular security risk. Some things that you can look at to say, are you vulnerable to monitoring and logging? This is a little bit of a joke, but maybe not. It's one way to know you're vulnerable is if you're not logging anything and if you're not looking at your logs. That's the big one we're going to look at, but we'll go through this little list here very quick, so you can see the logging detection, monitoring all that can occur when error or if these things are true, auditable events, and all that logins failed logins high value if they're not logged, obviously. Warnings and errors generate no log messages or inadequate or unclear log messages, so even if you do log, you need to make sure the log messages are understandable that they're meaningful. Don't just say error and then that's it. We did talk about error messages with respect to applications giving an error message back to a user. If a user has a failed login, the error message you can send back to the user that they see maybe like, hey, username and password not entered correctly or maybe just there was an error trying to log you in or something, something pretty generic. You want to give that information back to the user. But on the backend, the thing that you're logging needs to be very specific, needs to be as detailed as you can be so that you can know exactly what's going on. Again, you don't want to give the user a bunch of details because that user could be an attacker and you don't want to lead them on and give them more information than they need. But again, for you, you need to have good error messages, good warnings, good log messages. The next one there, logs of applications and APIs are not monitored for suspicious activity. You need to monitor the logs. You need a log, but then you need to monitor. There's different tools to say, hey, is there anything weird going on in these logs? You can see their logs are only stored locally, so you want to have a backup of the logs if they're stored locally, then this is some of that disaster recovery type discussion where that if something happens if the building burns down if there's a big tornado or whatever, you want to make sure you've got logs stored, not just locally, but off-site as well. Make sure you're doing that. Then appropriate alerting thresholds and response escalation are not in place or effective. This is where if you don't have these thresholds, so let's say you're logging events, let's say someone types in the wrong password. Well, you don't want every bell and siren to start going off if one user types in the wrong password one time. Having said that there is a threshold that you need to set and say, hey, if they've entered the wrong password 50 times in a matter of about two seconds, then that's a problem. That's not even a human. That's a bot. There needs to be some alerting threshold that sends messages or sends some notification to the administrators that says, hey, this is a problem. If you don't have those thresholds in place, then you could be vulnerable to this. Anyway, that's another one. Penetration testing and scans by DAST, Dynamic Application Security Testing tools do not trigger alerts. These are again, as you do pen testing if you conduct those tests which is good and scans by DAST tools. If those are not triggering alerts, then it's like, hey, what are you doing here? You need to monitor, you need to test, you need to scan, but it needs to trigger alerts and let you know when something bad is happening. You can see there that last bullet, the application is unable to detect or escalate or alert for active attacks and real-time or near real-time. You need to have a capability via your logging and monitoring tools to detect an attack or alert for an attack in either real-time as it's happening or very near real-time. You certainly don't want to be the victim of an attack that these attackers come in and then you don't get alerted until you look at the logs the next day or maybe it's a couple of days or even several hours later, you could have significant downtime or degraded performance. A number of things for your application that your legitimate users are trying to gain access and they're just having all kinds of problems and you do not want that. Then it says down at the very bottom, you're vulnerable to information leakage if you make logging and alerting events visible. This goes back to what I was saying before. There are things that you want to show the user in terms of, hey, I've put in the wrong password or whatever, then you want to give them an error message that says, hey, invalid, invalid login or something like that. But then the information on the backend, the stuff that you can look at as the administrator needs to be very meaningful, more detailed, more specific. But you don't want to make those things visible to the public. Kind of a thing. Make sure you don't do that. Make sure you don't give the logging and alerting events. Make sure you don't make those public are visible to the public because then it's now you're either showing attackers that, hey, I am grabbing all this stuff or, and/or you're telling them, hey, this is exactly what's going on on the backend and it just gives them more information or more ammunition to be able to come after you via an attack. That's not good. How do you protect yourself against this? You can make sure that all login, access control failure, server-side, input validation failures, all that, are logged with sufficient context. Just like we were talking about. Then the last part of that first bullet ensure that they're held for sufficient time to allow forensic analysis. What that is talking about is, if you are the victim of a data breach or an attack frankly whether it's a data breach or not. If you're the victim of an attack, then potentially the police or the FBI or some law enforcement may need to get involved so that they can track down the attacker. If there's been any laws broken, then they can go get those people. You need to make sure that the logs are kept, that they're held for a sufficient amount of time to allow for that analysis that the law enforcement may have to do on it. Don't destroy your logs immediately. I mean, you could have a business discussion about how long do I keep the logs. There may be a point in time where you say, we're not going to keep logs forever. If these are ten years old or something like that, then I'm going to get rid of those. Maybe you could even look at the average time to detect and clean up a data breach. You can look at that and say, that's about the average time, so I certainly don't need to get rid of logs prior to that amount of time. Maybe once that amount of time has passed, then I can think about looking at some drawdown in terms of logs, storage and all that, because you're going to pay for storage for these logs so there's a business discussion that you can have around that. But certainly you want to make sure you hold them for enough time for the law enforcement to come if they need to and then you can see there the second one make sure that logs are generated in a format that can be consumed by centralized log management. A lot of times people will send logs. You've got like syslog servers, or you send it to a service like Splunk is a really popular technology to be able to read logs and do a lot of analysis on logs, all that. But nonetheless, it needs to be in a format that one of those log management solutions can consume the logs. If you're generating logs and you're not storing them in a format that any of these tools can use, then it doesn't do you a whole lot of good. Then that next one, ensure high value transactions have an audit trail integrity controls to prevent tampering, those things. You can see they're append-only database tables, things like that. Of course, going back to that law enforcement example, if law enforcement has to get involved and they're going after a potential criminal then the criminal could start to lawyer up and say, I didn't actually do that or those logs were changed or whatever. You need to have integrity controls in place that prevent tampering with the logs. Because as soon as you start to introduce doubt and confusion as to whether a log entry is legitimate or not, it's hard to go after people and prove that they're the ones that did this bad thing to you, right. Make sure that you have those integrity controls in place. Then you can see there establish effective monitoring and alerting so that suspicious activities are detected and responded to. This gets back to the heart of just what we're talking about. You need to have alerts like some of those thresholds we talked about making sure that suspicious activities are detected and they're responded to, and then establish or adopt an incident response recovery plan. This is all about if and when we get attacked, if and when there's data that's exfiltrated or breached from our application, from our organization, what are we going to do about it? How do we step through our plan and respond to this incident, and then how do we recover from that, right? Let me go to this next slide. I found this on Cisco.com and you can find a variety of different incident response outlines or plans out there. You can search, talk to your friends, search, Google, or search the internet, whatever, and find some best practices from other organizations. But these are a few key things that you would want to make sure that your incident response plan has in it. The first one, there, very clear roles, responsibilities and approval processes for all the players involved. You need to sit down and figure out who those people are and establish clear roles and responsibilities and approval like who's got the power to make certain decisions and those types of things. Regular training and assessment exercises. You need to exercise this plan. You need to run through it. I mean, you can even plan these things out. We did this in the military a lot. You would have exercises where we knew that on day x we were going to exercise a certain maybe chemical warfare type environment or whatever and so then is everyone on the base prepared to do what they're supposed to do. Very similar here if you run through your organization and say, next month or whatever it is, we're going to run a scenario where data is exfiltrated or data is breached. We're going to have to walk through our either people making the right decisions, and these things. These exercises always highlight massive amounts of information and spotlights on areas that you can improve on. Then you just keep doing this. As you've got turnover in the organization, you've got new employees, you've got people that have left. You need to keep doing this on a fairly regular basis. Then the next one, a regular means of communicating incident response, planning efforts and successes with upper management. The very high level, the C-level people in your organization are not necessarily going to be on the ground. The tactical level in the weeds as it were, boots on the ground kind of a thing with the actual response. But you need to inform them because like we've seen on a variety of newspaper articles or online articles that have clipped on these various videos you don't want your organization to be the highlight of one of these massive data breach online articles or whatever, so you need to make sure the upper management is informed and you communicate the response efforts with them. Then the next one, a firm understanding of organizations infrastructure where the crown jewels lie. This is all about some of the planning and some of the prioritization of data and processes and applications of your organization and so you need to know what is the most important all the way down to the least important. Or you could even do a tier, tier one is super important stuff. Maybe there's not a single most important application or single most important data but these particular applications or this particular type of data falls in tier 1 let's say and then you got tier 2, tier 3, whatever it is. You can organize this however you need to, but you need to have a good understanding of the prioritization of data applications, that thing within your organization so that you'll know when I respond to this, then if tier 1 has been impacted at all, let's say that's the way that you organize things then we need to clear that up and then maybe you don't spend as much time and resources on the lower tiers at first. Some of the organization and infrastructure understanding that's going to lead you to the way that you respond to these incidents. Then that last one, a feedback loop to ensure that incidents are not just simply cleaned up. You need to not just clean up the problem. Let's say you have data that was leaked, data breach and people's credit card numbers or their social security numbers or whatever, got leaked or whatever, then you certainly want to clean that up. You want to provide maybe credit monitoring resources to your customers. You want to reach out and say, okay how bad has the problem been and you want to clean up the spill as it were. But if you don't come back and feed back to the way that you monitor and log and protect your applications and your data then the whole thing can happen again, so you need to come back and say, okay, well what defensive processes or technologies or whatever have security components have we put in place to make sure that this does not happen again, certainly not in the same way it did. Anyway, so that feedback loop is good. Don't just clean it up but fix like you're going after the root problem there. Those are five different things that you could make sure are parts of your effective incident response plan when you talk about disaster or when you talk about incident response and disaster recovery, that type of thing. Those are just some practical things that you hopefully could make part of your plan. A few other resources to look at here. You can see up there the OWASP. Actually all these are OWASP resources, but the proactive controls with security logging. Of that top one, there's that ASVS again, application security verification standard. I know we talked about that many times. That's an actual standard you can use against applications to make sure that you're monitoring and logging properly. Then you can see there's cheat sheets down there for logging and vocabulary, all that stuff. These are just some good resources to have in your tool bell, if you will, on how to properly log incidence and then also how to properly monitor them. Again, it's a big deal that came from the survey that was done. The practitioners that are out there actually doing this every day have said, hey this is a problem and we need to put it on the top 10 lists so here it is, higher in rank than it was on the last top 10 list. Anyway so with that, let me say thanks for hanging in there and we've got one more video to go. I'll look forward to seeing you on the next video, which is the last and final. Thanks for being here and we'll see you next time.