The third characteristic of cloud native we want to understand is observability and how it contributes to DevOps teams becoming more accountable for application security. In short, cloud native Observability means providing information, at all stages of an application’s development lifecycle, such that any question about its code, build, releases, deployments, and customer usage can be answered. This data includes: logs and events from the CI/CD tooling and the running application, Auditable access records produced whenever any of the application’s APIs are called with caller trace information, Instrumented metrics for live monitoring of application health as well as telemetry data to track usage. All this data is collected and aggregated continuously and made available to technologies for real-time monitoring, analysis and visualization. Using a wide range of cloud native technologies and products for all use cases, takes advantage of standardized formats for log, event, and telemetry data from the Cloud Native Computing Foundation (CNCF). In practice, observability is about being able to Identify, pinpoint and Fix problems in any part of the CI-CD processes or the running application as early as possible in order to reduce the mean time to recovery. For cloud native, this involves instrumenting not only the application code itself, but also the tooling and technologies used to build, test, package, and deploy it with real-time metrics. So, active monitoring can be applied, and contextual alerts are sent to team members immediately when conditions indicate abnormal readings. Another significant goal of observability is Unified Governance to help identify exposures, improve Confidence that risk is being mitigated, and to improve the overall application’s auditability posture. This includes Proof of Compliance to requirements and policies imposed by corporation or by regulated industries such as government, banking, finance or healthcare. It also includes traceability of access to any part of the application’s software development lifecycle for security audits. Cloud native governance processes should: Employ automated tools for static code analysis and scanning for the latest Common Vulnerabilities and Exposures (or CVEs) as they are regularly published. Examine all abnormal accesses and review access permissions regularly to assure that they are necessary and properly scoped for the corresponding job roles. Lastly, observability seeks to optimize resources and reduce operations costs. Metric and telemetry data are invaluable in providing customer usage data over different periods of time in different deployments. DevOps teams can analyze this data to fine tune scaling and server cluster configurations to only provision and pay for the infrastructure resources needed by each application microservice. A less-often considered benefit of usage data is resource & feature planning. Where histograms of past usage can help determine resources needed for new features, or to justify sunsetting less-used services or expanding on popular services, telemetry data can also be valuable to the application owner as an accurate and trusted means for them to properly bill the application users for their actual usage of each application service. What is DevSecOps? Simply, it is DevOps but with an additional focus on security and compliance. It emphasizes the need for developers to code, build, and test with security in mind as well as provide operators with needed data to make informed decisions about application security readiness and requirement compliance. From a culture perspective, DevSecOps builds on the cultural changes of DevOps to integrate the work of security teams sooner rather than later. This additional focus may mean instituting security education and training for developers and operators, since security and compliance concerns have traditionally not been part of their roles. Continual human processes may need to be put into place to determine risk tolerance and conduct risk-benefit analysis when: integrating new code or tooling or deciding on the acceptance criteria for releasing and deploying the application into a new environment. DevOps teams, working with security teams, should seek to integrate security within all software-development lifecycle processes. Use of automation should be maximized in order to preserve agility in the end-to-end delivery of the application. DevOps teams must assure all tools and code produce the appropriate data to pinpoint problems and identify threats and vulnerabilities within all stages. Conversely, security teams need to actively share their knowledge with DevOps by providing feedback on security data visibility issues as well as insights into known threats. Integrating security into processes, from a coding perspective, can involve such things as, Introducing Test-Driven Development (TDD) where functional and unit tests are written first for a new feature or change and code is written to pass it. Using Pair Programming where two developers work together and take turns where one developer writes the code and the other one observes and warns about potential bugs they spot. Many tools can be automated within the CI/CD build processes to help with threat detection and prevent tampering such as adding code linters and vulnerability scanners as gates within CI, providing unit test coverage scanners can be run against code changes before merging them, and others to sign and notarize application binaries and packages before they are released. When employing Technologies, Development teams might standardize on a modern Integrated Development Environment (or IDE) that can dynamically run tools to assure code is written to team conventions and standards, and also warn of updates in the application stack including language, framework, and libraries. Many popular cloud native technologies are available to manage the collection, visualization, and analysis of the vast amounts of security-related data. One of the most widely used is CNCF’s Prometheus, which is a system and service monitoring tool with a framework for integrating databases and various monitoring and log management services. In fact, most cloud native technologies you may encounter already have built-in integrations that can emit log and event data to it and then be used by compatible monitoring tools, like Grafana with rich dashboards or LogDNA for log management and analysis. A more lightweight alternative for log data collection is Fluentd which also has hundreds of integrations and uses minimal resources. These are just a few examples; many ready-made cloud native technologies are available to help infuse Security into your DevOps processes. You should now be familiar with: The importance of observability data for, Problem determination and recovery, Unified governance, Resource and cost optimization, and What DevSecOps means and how to apply it.
