Welcome to the cybersecurity, leadership and management course. Today we'll be discussing cybersecurity, enterprise, quantitative and qualitative key risk indicators and key performance indicators, better known as KRIs and KPIs. My name is Cicero Chimanda and I am your instructor for this course. Cybersecurity enterprise, quantitative and qualitative, KRIs and KPIs. We will be discussing the importance of risk management with metrics that are quantitative, and qualitative. In this course, we will cover the following topics. Quantitative and qualitative, KRIs and KPIs and overview. We will then delve in, into talking about risk management with quantitative key risk indicators or key performance indicators. Then we'll cap off by discussing risk management with qualitative key risk indicators or key performance indicators. Let's begin. The overview. Cybersecurity is primarily made up of three elements as we know. There are people. Obviously you cannot have an organization without people. There are processes. This is what that organization does. This is the key sauce and the interaction and the processes, the working habits of that organization. It must be formalized in order to be able to be efficient. Then lastly, the technology. Obviously we are in a modern technical world, information age and AI, artificial intelligence, internet of things, business intelligence, cloud computing. These are all part of that technology that enables the organization to be more efficient and more innovative. But again, these three process or elements, they need to be measured. There needs to be metrics around how it's operating today, compared to how it operated yesterday, so that we can make better decisions on how we will operate tomorrow. That's where your key risk indicators for cybersecurity comes in. Then we will also talk about key performance indicators. But in this case, key risk indicators, simply means you're looking at indicators, that can potentially compromise. They have critical predictors. They can have a negative impact to our organization's mission, vision, strategy or goals. These are events that have levels of risk, exposure, they can contribute to early warning signs to enable the organization to be able to act. Whether it's the detect, whether it's to report, whether it's to mitigate, to prevent, to predict. We'll look at some of those components as well. But that's the objective of a key risk indicator. What is a key performance indicator? Well, in cybersecurity, we know that they are risks or vulnerabilities that have been acknowledged or assessed or discovered. Well, we need to have countermeasures. We need to have actions to mitigate or to fort, if you will, those vulnerabilities. These are your key performance indicators. These are actions that you take in order to mitigate your key risks indicators. They can also be called countermeasures or actionable events. It can be done in a person. Your people can have these key performance indicators. It can be within a process, and it can also be with technology. Again, the whole goal is to mitigate and to prevent an actual breach when it comes to cybersecurity. These are key. KPIs or KRIs can be measured in two types of formats. It can be in quantitative indicator. In this case, if you have a malware block, for example, in this graph, you can have a pie chart. It can be done basically with a number, with a ratio, with an index, a percentage, it can even be chronological date differences. But ultimately it wants to answer the how many, how often, what portion, how much? It can be depicted in a graph, in a pie chart, as we see here, or in a bubble graph or other types of pictural format. There is qualitative indicators and that's also depicting elements in a non numeric measure. It's seen and such as experiences, attitudes, perceptions, and words. It can also be done in pictures as you see here, or in stories. This qualitative indicator at times can also be done especially at the academic in terms of case studies, or interviews, or white papers, or focus groups. There are many ways to have indicators of qualitative indices to show your progress or lack thereof in your risk. The questions that typically wants to answer is how do you feel and what do you perceive? As we have talked all throughout this course, I want to just reiterate that information security has an objective. The objective of the Information Security Cyber Task Force is to assure that data or systems have the confidentiality in place, have the integrity, that the data has not been tampered, has the availability, whenever those resources are need to be available, they're available, or they might not be available to those who should not have access. That's the CIA triad, as we all know. That's the deliverable of a cyber security task force. Well, the chief information security officers that translate those CIA, the triad, into deliverable alignment that senior management can understand and all the stakeholders can understand. This is done through security. That's your deliverable as a security officer, is to make sure that your systems, your data, your infrastructure, your people, your process, your technology are secure. You need to convey that there's trust that you obviously as an organization, whether you're a private, a public, government, nation-state, or a non-for-profit. You have regulatory obligations. You have specific rules that you must adhere to. In pursuing systems that will allow you to do those things, builds trust. That's your deliverable; to convey the trust in your organization. Lastly, but not least, is that stability. Your organization must convey that it keeps the lights on, that operate optimally, and that it can be reliable. Stability is the deliverable of your cyber security program, and that is your responsibility, Chief Information Security Officer. Lastly, what is governance interested in more? They're not necessarily going to look at the details of confidentiality, integrity, availability. They might understand and they need to understand that security, trust, and stability is aligned and conveyed, but their language is really organization strategy. Is my risk, is my cyber security program, my information governance, is it align with my organization strategy? Are we going to achieve what we are going to want to achieve? Are we meeting regulatory systems? That's what they want to know. Are we compliant? Are we safe? Are we operating in the lane that we should be operating? Obviously, do we have the operational excellence that our customers, our stakeholders, our shareholders, can count on. That's the language of governance. You can see the alignment between the CIA, the STS, bringing into the governance model. Now, when it comes to your key risk indicators and the KPI, you want to have measurement at all phases of your program. You will have phases of detection. You want to convey KRIs and KPIs when there is detection of a breach, of a vulnerability. At the detect level, you want to have KRIs and KPIs associated to it. You want to have KRIs and KPIs that will report, that will convey, that will communicate at all levels of your audience. We've talked about that communication in different chapters. You want to have KPIs and KRIs that communicate your prevention. This is where a breach was stopped. This is where you can convey how successful your program has been. Then the next one is your mitigation. This is where you communicate to your constituents, how you will respond to the actual breaches, to the different vulnerabilities, different attacks, how you mitigate it. You need to have KRIs and KPIs that conveys that as well. Then you want to have recover. You want to convey how your incident response and recovery phases has panned out. You want to have key risk indicators and key performance indicators around that phase as well. But the last one, which as you can detect on my slide, I left one empty at the beginning. I believe this is the most important area where your key KRIs and KPIs need to deliver to your CISO, to your cyber task force, and to governance. That's the ability to predict because you want to use intelligence in order to look at future threats, future vulnerabilities, future problems in the landscape, and be able to predict how to act so that you can mitigate future attacks. That's where you want to be. We've talked about quantitative and qualitative KRI and KPIs at a high level in overview. Let's talk about them individually. Cybersecurity management quantitative key risk indicators and key performance indicators. Well, indications of quantitative measures is one that includes, as we mentioned earlier, numbers, ratios, indexes, percentage, chronological differences, but it ultimately wants the answer, how many? That's one of the questions it wants the answer. The other question is, how often does this happen? Looking at periodically dates and the comparative analysis. You want to answer the question, what portion, percentages? How does it look like in terms of the overall elements? Then lastly, how much? This can be in cost, this can be in effort, this can be again in time. How much is an indicator of your quantitative measures metrics? Let's look at a sample where you're calculating risk quantitatively using at the annual loss expectancy. This is a formula that's used in the cybersecurity industry to measure the cost of loss to assets. The formula that's used is ALE, which is annual loss expectancy equals SLE times ARO. Well, let's look at what these mean. Well, SLE is your single-loss expectancy. That's your single-loss expectancy for that asset. The way that single-loss expectancy is calculated is looking at the asset value at purchase or current value if you want to do the current value, and then you times it by the exposure factor. Exposure factor is, what is the percentage of that asset having a risk happen within that year? What's the percent? What's the exposure factor? The ARO, which is the annualized rate of occurrence, is the estimated frequency of occurrence in that year. It's a little bit different than the exposure factor. The estimate of frequency is actually the event that it's happened. The exposure factor is how do you determine in terms of if an event happens, what is the damage to that asset? It could be 50 percent, 70 percent. It's in percentages time. But the annual rate of occurrence is the estimated frequency of that occurring in one year. Let's look at a specific example. We have the facts in front of us. The security team has procured a security system appliance that was purchased at around $100,000. The risk of exposure factor is about 30 percent when it comes to that asset and then the estimated frequency failure of this particular asset is one every two years. How do we calculate the annual loss expectancy? Well, we take the asset value, which we know it's $100,000 times the exposure factor, which in this case is 0.03 because it was 30 percent, so that's 0.03, and you come up with $30,000. That's the single-loss expectancy for that asset, and calculated in dollar terms. Now, we look at the annual rate of occurrence. We know by the facts that this asset has a frequency rate of occurrence in malfunctioning or failure once every two years. That comes out to be 0.05. Now, with these two data points, you can calculate the annual loss expectancy. Well, we know by the formula that annual loss expectancy is single-loss expectancy, which is 30,000 in this case, times the annual rate of occurrence, which is the 0.05. Our exposure or annual loss expectancy for this asset is $15,000. Again, this is calculating a quantitative metric in understanding your risk. Let's look at additional quantitative risk indicators that you might apply to your organization. In this case, it's the alignment of operational excellence. We know that operational excellence comes from availability and stability and you want to have operational excellence. These are elements or appliances that you can have for operational excellence. The first one is your Network Operations Center. Your NOC and within your NOC you have cooling and HVAC systems. Some of the metrics that you want to surface if all your organization that are quantitative in that bracket would be failure rate of the cooling HVAC systems during the preventive maintenance service. This is a quantitative measurement that you can surface. The other is in power in UPS, which is again a NOC system. You can look at the time duration, the number of days the NOC has not experienced a power outage that impacted mission critical systems. Because you're UPS or your power systems is there to help absorb some of that, a power outage in your building for example. This is a metric that you can surface from that data point. Lastly, you have, in operational excellence, you want to have redundant fiber. Your network ISP, Internet Service Provider, or your SD-WAN, or your MPLS, you want to have redundant cables. A metric, that's quantitative is the annual report of fiber optic cables cut or interrupted by natural or man-made causes, so you want to surface that, and in doing so, you want to show how you are redundant. Actually, you show the number of outages that are interrupted and whether you were redundant or if the redundant circuit was also impacted, you want to show that as well. Again, these are key risk indicators, that are quantitative, that fall into operational excellence in order for you to mitigate some of your risk. Again, let's look at another element. These are organizational strategy. We know that the alignment of organization strategy comes from your confidentiality, your security in an organization strategy. Again, these are administrative elements that you might employ to mitigate some risk. These are some quantitative key performance indicators that you can surface to your board or to your constituents. For example, in your cybersecurity task force, you can show the number of quarterly risks, closed by a cybersecurity task force, that is one of the functions of the cybersecurity task force. They raise up risks, and then it is up to them to make sure that those risks get closed. You can surface the number of closed risk within a quarter. For example. The other is an annual risk assessment. You can bring in a third party. You can have an internal and annual risk assessment. Preferably, you want to have both. So you have internal and external, but you can surface the number of annual high risks resolved. This is really important. You have high risk assessments. You surplus them. You can show the number of risks that you closed that were high risk. Lastly, you want to have penetration test. There are three types of penetration. You've got the black box where it's closed. You got gray box or white box. In gray box is where you disclose some information to the pen tester and some you do not. Then obviously, the white box is where you show or you give the information of scope where the pen tester should be able to operate within. But anyways, the quantitative metric, which is a key performance indicator, is the number of breaches attempt, that were blocked or mitigated. You want to have, we did try to go through this. You don't want to just talk about where you are successful in breaching in the pen test, you also want to show where you blocked it or mitigated and then highlight what performance is in place that blocked that, that is your key performance indicator. We're shifting over to qualitative key risk indicators or key performance indicators. The first one is we want to look at the qualitative metrics questions that need to be answered. These are questions slide such as, what do you perceive? The next question is, for example, what is your experience? This is a qualitative metric answer. Then lastly, how do you feel? How do you feel about the service? Again, these are qualitative metrics that you can ask and answer when you're looking to measuring your key risk indicators and key performance indicators. Let's look at some qualitative metrics around an antivirus and an anti-spam system that was just deployed. For example, you might want to have a survey done to figure out how qualitatively is your antivirus and anti-spam. This is one question that you can ask to your users or to your employees. How is your PC performance with the new antivirus system compared with the previous? Again, this is how you perceive. You do have your measurement, which is a quantitative metrics around the performance, the speed, the CPU, the memory, all the performance indicators that you might have. You might have quantitative metrics, but you also want to know how is it perceived to your users, so you ask them this question in a survey. Most of the time they're probably going to say it's worse, even though it's better, that's just the way a human element works. But nevertheless, you might want to ask, how was your experience of releasing your email from the quarantine? If you have an anti-spam system, it blocks, it quarantines, it notifies. You might want to ask, how was your experience in releasing your email from quarantine, because there's always a process, is it too cumbersome, is easy, it it user-friendly? Again, this is a question that you have a qualitative metric around your question. Lastly, how do you feel customer service has been when you engage the help desk? This is, one, your customer service. The way you might want to measure is simple for the users as having some metrics such as, very poor, poor, average, good, excellent. We've seen this graph in earlier course. But nevertheless, this is the kind of a qualitative metric that you might want to use to your employees. Let's look at some qualitative metrics that you might want to deploy in some of your key functions. We're looking at organization strategy. Again, we're looking at the element. This is security elements. You have your SASE, which is your Secure Access Service Edge. This is your new industry firewall, if you will, that's not geographical based. One key risk indicator that's qualitative, that you might want to look into this, a SASE solution, is when there is an issue, was the root cause by the Internet service provider? Was it the SASE product, or was it a vendor? You might want to have that qualitative information to a specific issue that was caused. Multi-factor authentication part of the IM. you might want to have, for example, what are the gaps in user training in onboarding materials that may have resulted to the users misconfiguration or misusing verification methods. Again, this is all part of the training, are people understanding how to use their MFA systems when their onboarding? Again, this is a host-based firewall. This could be qualitatively, the health state, indicates the health of the firewall based on SNAT port availability in unit percentage. Again, this is the availability in indicators of health of a firewall that shows qualitative metrics. KPI, Key Performance Indicator, we can see here regulatory systems. This is where integrity equals trust, which brings your regulatory systems. Governance Regulatory Compliance, GRC, you might want to measure is, there are centralized communication of single secure communication hub that instantly syncs to the Cloud. In other words, is there a centralized method of communication for your GRC? Yes or no, qualitatively. Business financial systems, this is your back-office systems or it could be your operational systems as well. But looking at the business units for customers, we survey on features and user friendliness of the systems. Then you can also have firm element or training systems, looking at customer surveys, the results and features, user-friendliness of the training systems. Again, these are qualitative key performance indicators, qualitative key risk indicators that one can adopt when you are measuring risk within your organization. This concludes this particular course. Thank you for listening, and we'll see you at the next course.
